Comet Opik 2.2.81 (2026-09-25) routes scored entities into annotation queues, optionally provisions an extended free-form SQL ClickHouse user, registers 19 canonical providers so model prices load, and fixes Gemini usage, streamed Mistral reasoning, and slots=True dataclass encoding in the SDK.
Key Takeaways
- ✓Shipped 2.2.81 on 2026-09-25; pip install opik==2.2.81
- ✓Evaluation loop: route scored entities into annotation queues
- ✓Data plane: optional extended free-form SQL ClickHouse user behind a flag; drop spans FINAL on find_trace_stream / find_thread_by_id
- ✓Cost: register 19 canonical providers so model prices load
- ✓SDK: Gemini usage without candidates_token_count; keep streamed Mistral reasoning; encode slots dataclasses as dicts; opik.sh startup timeout 90s
Project Links & Resources
Direct AccessIn-Depth Technical Analysis
Core Background & Industry Pain Points
Agent/RAG teams need scored traces, human review, and cost accounting in one loop. Breaks: scored entities never reach annotation queues, weak production SQL on ClickHouse, missing multi-vendor price tables, and streaming usage schema drift (Gemini/Mistral) crashing SDKs. Opik 2.2.81 (2026-09-25) patches these ops gaps.
Architecture Highlights & Internals
Routes scored entities into annotation queues; optionally provisions an extended free-form SQL ClickHouse user behind a flag; drops spans FINAL on find_trace_stream / find_thread_by_id; registers 19 canonical providers for model prices. SDK: accept Gemini usage without candidates_token_count; keep streamed Mistral reasoning; encode slots dataclasses as dicts; allow % in config values.
Authoritative Benchmarks & Measured Scores
No end-to-end latency numbers in the notes — verify FINAL removal and the 90s opik.sh startup timeout in your env. Do not invent cost-savings percentages.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.