AI Coding Live News — Real-Time 𝕏/Twitter Updates & Breakthroughs

Comet@comet-ml·Collected 42d ago

👀54

🔥91

Comet Opik 2.2.84: online eval rules call Jev via OpenRouter Decisions API; queue automation + provenance

Comet Opik 2.2.84 (2026-09-29) lets online LLM-as-a-Judge rules call TypeSafe Jev end-to-end via OpenRouter POST /api/alpha/decisions (not chat/completions). Each Boolean metric becomes a yes/no question answered in one call; score 1 if probability ≥0.5, with Probability: 0.93 in the reason. Backend adds OpenRouterDecisionsClient (workspace OpenRouter key, 429/5xx retries) for ~typesafe/jev-latest / typesafe/jev-1.13. Also: Annotation Queue automation + item provenance. ~8 commits / 104 files vs 2.2.83.

Key Takeaways

Release 2.2.84; ~8 commits / 104 files vs 2.2.83
PR #8540: online judges call Jev via OpenRouter Decisions API
Boolean metrics → yes/no questions; score 1 if p≥0.5; reason shows Probability
Models: ~typesafe/jev-latest, typesafe/jev-1.13
Queue automation + item provenance; trace stats / thread_id fixes

Read full story

Linked Models & Agents:

⚡ Jev⚡ typesafe/jev-1.13🤖 LLM-as-a-Judge 🤖 online evaluation 🌐 Comet 🌐 OpenRouter 🌐 TypeSafe

#Opik#Jev#Openrouter#Decisions API#Evaluation#Release

Comet@CometML·Collected 46d ago

👀56

🔥90

Comet Opik 2.2.81: scored entities → annotation queues, gated free-form ClickHouse SQL user, 19 provider price tables

Comet Opik 2.2.81 (2026-09-25) routes scored entities into annotation queues, optionally provisions an extended free-form SQL ClickHouse user, registers 19 canonical providers so model prices load, and fixes Gemini usage, streamed Mistral reasoning, and slots=True dataclass encoding in the SDK.

Key Takeaways

Shipped 2.2.81 on 2026-09-25; pip install opik==2.2.81
Evaluation loop: route scored entities into annotation queues
Data plane: optional extended free-form SQL ClickHouse user behind a flag; drop spans FINAL on find_trace_stream / find_thread_by_id
Cost: register 19 canonical providers so model prices load
SDK: Gemini usage without candidates_token_count; keep streamed Mistral reasoning; encode slots dataclasses as dicts; opik.sh startup timeout 90s

Read full story

Linked Models & Agents:

🌐 Comet

#Opik#Comet#LLM observability#Tracing#Annotation#ClickHouse#Evaluation#Gemini