AI Coding Live News

Garry Tan (@garrytan), President and CEO of Y Combinator, posted on X at 18:09 UTC on 2026-11-08 that out of curiosity he is doing open-source eval work on harnesses with frontier models, attaching a screenshot of early results; he then shared the GitHub repo garrytan/g-harness-evals (MIT). The repo holds the model (Opus 5.5 or GPT-6.1 Sol) and tasks fixed and swaps only the harness (system prompt, tools, memory). The screenshot says 7 setups all passed 71-79% of the same 20 coding tasks, and 14 of the 20 tasks scored the same in every setup, so the test was too easy to separate them; a harder Terminal-Bench rerun is in progress. Cost differs a lot: on Opus, OpenClaw about $0.44 per solved task, Claude Code $0.22, Hermes $0.24 and QM $0.19; on Sol, Codex is cheapest and fastest at $0.17. At capture the post had about 312 likes, 84 replies, 117 bookmarks and about 41,767 impressions. The repo discloses conflicts: QM is a YC project and YC is an investor in Hermes.

Zoom

Public benefit corporation Robocurve (@robocurve) published "You Can't See Everything: Perceptual Attacks on Embodied AI" at 19:44 UTC on 2026-11-06 and will present it at CoRL Late Breaking Results. The team had Claude Fable 5.1, Claude Opus 5.5, GPT-6 Astra, GPT-6.1 Sol and Gemini 4 Argon control a real bimanual robot in a closed loop through the Inspect Robots framework on five unsafe tasks, 900 trials in total. Per the paper, models attempted the tasks in 81/300 trials (27%) with unaltered footage, 263/300 (88%) after a Gaussian blur and 291/300 (97%) with frame-by-frame doctored footage; the "stab the object" doll task hit 60/60 with doctored footage. All props were empty bottles or unpowered appliances. At capture the original post had about 11.2K likes and 3.63M impressions.

Zoom

YC Labs, a research project at Y Combinator, and multi-agent environment startup Olam Labs published "Agents in Games" on Oct 29. Fifty GPT-6.1 Sol agents told to "build a thriving civilization" in survival Minecraft spent three hours huddled in shelters rationing food, sending about 5,900 messages and inventing ration ledgers, permits and job titles. In nine Bed Wars games, 10 GPT-6.1 Sol agents beat 10 Claude Opus 5.5 agents 6–3; in seven 50-vs-50 Mindustry matches lasting about 9.5 hours, GPT won 6–1. The leader was almost always whichever agent was listed first in the directory, and stuck GPT agents fetched the game's public source code on their own. The Minecraft harness is open source. These are one lab's experimental observations, not a standardized benchmark.

Zoom

Kaggle announced ScarfBench with IBM Research, a 204-task benchmark that tests whether coding agents can migrate enterprise Java apps between Jakarta EE, Quarkus and Spring without changing behavior. IBM's October 27 technical note shows Gemini 4 Argon with Antigravity first on the smoke score at 75.2%, Claude Opus 5.5 with Claude Code highest on all-tests-pass at 72.1%, and GPT-6.1 Sol with Codex cheapest at $1.25 per solved task.

Zoom

AI coding agent company Factory announced Droid 1.0 on X and its blog at 22:31 UTC on Oct 29, calling it the largest update yet to its agent harness. The key change makes programmatic tool calling (PTC) the default: the model writes a JavaScript program that runs up to hundreds of tool calls in a sandbox in one model request. In Factorys own Terminal-Bench 4 runs, Droid scored 57.6% with GPT-6.1 Sol ($0.79 per passed task) and 65.2% with Claude Opus 5.5 ($4.73), ahead of Codex, Claude Code, Cursor, OpenCode and Devin on the same models.

Zoom

On Oct 28 independent evaluator Artificial Analysis released Intelligence Index v5, its biggest revision since v4 in January: it adds Cognition's FrontierCode 2.0 and Terminal-Bench-Science, swaps Humanity's Last Exam for HLE-Diamond, and now draws 60% of its weight from private or held-out test sets. Claude Opus 5.5 (max) leads at 58, ahead of GPT-6 Astra at 57 and GPT-6.1 Sol at 55; the top open-weights model, Kimi K3.1 Pro (weights not yet released), scores 46, widening the gap from 3 points in v4.3 to 12. It also launched Progress Index v1 for comparing models across years.

Zoom

On Oct 26 OpenAI released GPT-6.1 Terra, an efficient model for visual reasoning, precise computer use and natural writing. It is rolling out today to Plus, Pro, Enterprise and Edu users in ChatGPT Work and Codex, and in the API as gpt-6.1-terra. Standard API pricing is $1 input, $0.05 cached input and $5 output per million tokens, which OpenAI says is 50% lower than GPT-5.6 Terra and GPT-6.1 Sol, with a 1,050,000-token context window. By OpenAI's figures it scores 15 points higher than GPT-5.6 Terra on Agents' Last Exam at medium effort for about a third of the cost, and solves 50.3% of Terminal-Bench 4.0 at max effort.

Zoom

Independent AI evaluation authority Artificial Analysis launched the HealthBench and HealthBench Professional clinical reasoning and medical conversation leaderboards. Addressing the critical flaw where models artificially inflate medical scores through exhaustive, padded responses, the Professional benchmark implements rigorous length penalties. GPT-6 Astra (max effort) leads the world at 66.0%, with GPT-6.1 Sol at 65.1% and Claude Sonnet 5.5 at 63.0%. Claude Opus 5.5, despite holding a 71.4% raw score, collapses to 59.9% (#6) due to averaging ~5,900 characters per reply.

Zoom

OpenAI officially announced breakthroughs in building and migrating software written in low-resource and legacy languages (COBOL, Fortran, Visual Basic, OCaml), introducing the MultiLangBench evaluation suite. Tackling the trillions of dollars in critical banking and enterprise mainframe infrastructure, the benchmark challenges models to synthesize a SQLite-compatible database engine across 20 distinct languages. GPT-6 Astra boosted average test pass rates from 26.8% to 87.4%, with COBOL soaring from 10.9% to 72.1%. In enterprise COBOL migration tasks, GPT-6.1 Sol achieved a 97.9% success rate while cutting inference costs by 39%.

Zoom

On October 19 independent evaluator Artificial Analysis launched two health-conversation leaderboards built on OpenAI's HealthBench (5,000 realistic health conversations) and HealthBench Professional (525 clinician conversations), graded against physician-written rubrics and reported as both raw and OpenAI-style length-adjusted scores. On Professional, GPT-6 Astra (max) leads at 66.0% length-adjusted, ahead of GPT-6.1 Sol (65.1%) and Claude Sonnet 5.5 (63.0%); Claude Opus 5.5 has the top raw score (71.4%) but drops to sixth (59.9%) because its ~5,900-character answers are penalized. On HealthBench, Claude Sonnet 5.5 leads at 62.1%, with DeepSeek V4.1 Flash close behind at 60.9% for about 1/16 of the cost per task.

Zoom

On October 19 OpenAI said its latest models are much better at low-resource languages such as COBOL, Fortran, Visual Basic and OCaml, and open-sourced MultiLangBench, which asks a model to build a SQLite-compatible database engine from scratch in each of 20 languages. GPT-6 Astra averages an 87.4% test pass rate (GPT-5.6 Sol: 26.8%), above 83% in 19 of 20 languages, with COBOL hardest at 72.1%. On a separate COBOL migration eval requiring byte-for-byte output parity after porting to C++, Java, Kotlin, Python or Rust, GPT-6.1 Sol scores 86.6% at low effort (GPT-5.6 Sol: 5.4%) and 97.9% at higher effort. All figures are OpenAI's own.

Zoom

An OpenAI developers blog shows how Core Experimentation uses Codex with GPT-6 Astra to clean dead code and stale Statsig experiments—evidence from Slack, telemetry and flags into small draft PRs. Official: 1,400+ maintenance PRs and 800k+ lines deleted over ~two part-time weeks. GPT-6.1 Sol Ultra reached ~92% of Astra Ultra’s accepted findings at ~14% API cost (vendor-reported).

Zoom

Artificial Analysis released AA-Robotics v0.1 on 2026-10-14, a research preview in which frontier models running inside coding-agent harnesses (Codex, Claude Code, Antigravity) control a 6-axis robot arm over an HTTP API across eight tabletop tasks. GPT-6 Astra scores 69.4, ahead of GPT-6.1 Sol (64.6), Gemini 4 Argon (55.3), Claude Opus 5.5 (48.8) and Claude Fable 5.1 (46.3). Models take about 30 minutes per task versus roughly 20 seconds for a human.

Zoom