AI Coding Live News

Independent evaluator Artificial Analysis reran VIALS-AA, its life-sciences visual reasoning benchmark, in a one-shot format with one request and no tools. Tool use lifted the cross-model average from about 25% to about 47%. Leader Claude Fable 5.2 (max) scores 67% with tools at $4.20 per task but 44% without them at $0.73, while Claude Haiku 5.5 (max) reaches 49% with tools at $0.07 per task. The same day AA also reported that across the Claude lineup, moving up a model tier usually beats maxing out effort on intelligence per dollar.

Zoom

Independent evaluator Artificial Analysis announced on X on November 4 that its six Capability Indexes (Finance & Accounting, Strategy & Ops, Legal, Healthcare & Medical, Engineering, Economics) are now v2.0, aligned with Intelligence Index v5.0 evaluations and weighting Non-Hallucination and Agentic Tool Use more heavily. Across the indexes, HLE becomes HLE-Diamond, AA-Omniscience moves to v1.1 and GDPval-AA is removed; Engineering gains a new Science capability. Per Artificial Analysis, Claude Opus 5.5 (max) leads all six, with GPT-6 Astra (max) second in Finance & Accounting, Legal and Engineering. Among open-weights models, Kimi K3.1 Pro (weights not yet released) leads all six at #5 to #8.

Zoom

On 2026-10-23 (15:41 UTC) the official @NVIDIAAI account announced Nemotron 3.5 Bolt E4B 4GB, a small open model for local assistants, interactive characters and robots that can hold a conversation and use tools. The official Hugging Face model card says it is post-trained from Gemma-4-E4B-it, has 8B total parameters (4B effective), ships in Q4_K_M quantization, supports up to 64K input and 64K output tokens, and has a switchable thinking mode, plus a DFlash speculative-decoding drafter and a BF16 version. Weights, data and recipes are open under the OpenMDW-1.1 license, with commercial use allowed. On the model card the 4GB build scores 55.60 on MultiChallenge (Gemma4 E4B-it at the same quantization 39.36, Qwen3.5 4B 52.23), but trails Qwen3.5 4B on Tau2-Bench (52.32) and BFCL-v3 (67.31).

Zoom

On Oct 20 ElevenLabs released Eleven Realtime, end-to-end speech-to-speech models where each model is a complete voice agent, removing the need to wire up speech recognition, an LLM and text-to-speech yourself. v1 Mini targets low-cost natural conversation and v1 Max targets support and multi-step tool use. Both cover 90+ languages and 17,000+ voices. Citing independent Artificial Analysis evals, Max answers 98.8% of Big Bench Audio correctly and starts speaking in 0.57s on average. The API is OpenAI Realtime compatible, so migrating means changing the endpoint and model name.

Zoom

On Oct 13 Liquid AI released LFM2.5-24B-A2B, the largest and final LFM2.5 model: 24B total parameters, ~2.45B active per token, 131K context, and always-on reasoning. By Liquid's own numbers it leads or matches Qwen3.6-35B-A3B on 5 of 8 tool-use/agent benchmarks (AutomationBench 27.69 vs 8.72) while trailing on SWE-Bench Verified, GPQA and other knowledge/coding tests. Open weights are on Hugging Face with GGUF, MLX, FP8 and ONNX builds; API pricing is $0.10 / $1.00 per million tokens and it is also on OpenRouter.

Zoom

IQuest Research’s IQuest-Q1 (~320B total / 15B active, 524K context) targets repo-level coding, terminal work, and long-horizon tool use. Weights and serving images are on Hugging Face / GitHub. Self-reported same-pipeline scores: DeepSWE v1.1 64.6, NL2Repo 63.0, CyberGym 84.5, Terminal-Bench 2.1 83.2. Ships with SGLang/vLLM images and Claude Code / Codex integration notes.

Zoom

Large language models are rapidly deployed across cybersecurity workflows to translate analysts' intent into command-line interface (CLI) executions. However, existing benchmarks emphasize either high-level knowledge QA or unconstrained agent rollouts, failing to measure precise parameter binding across real-world security tooling where minor syntax glitches invalidate execution. Researchers from MBZUAI introduce KaliBench, a fine-grained natural-language-to-CLI benchmark on Kali Linux spanning 8,504 query-command pairs across 1,642 tools, 23 capability dimensions, and 5 security phases. Auditing reveals no open-weight model surpasses 42% exact accuracy without tool hints. Furthermore, reinforcement learning with KaliBench's runtime-free verifiable rewards elevates an 8B model to rival a 685B MoE frontier model.

Zoom

KRAFTON and PUBG Studios have released 'PUBG Ally: A Conversational Embodied Agent as an AI Teammate' (arXiv: 2609.29837), unveiling the first voice-enabled, real-time conversational embodied agent deployed in AAA competitive gaming. Addressing the challenge of synchronizing spoken dialogue with high-frequency tactical gameplay under strict latency constraints, Ally couples an LLM meta-agent for tactical reasoning, speech interpretation, and contextual tool use with a decoupled high-speed control layer for movement, combat, and healing. Iteratively trained on nearly 39,000 live sessions with human teammates across 141 countries, Ally boosted player recommendation rates by 25.1 percentage points.

Zoom