Kaggle and IBM Research launch the ScarfBench Java migration leaderboard: Gemini 4 Argon with Antigravity leads at 75.2%, and about 72% of failures happen at runtime
Kaggle announced ScarfBench with IBM Research, a 204-task benchmark that tests whether coding agents can migrate enterprise Java apps between Jakarta EE, Quarkus and Spring without changing behavior. IBM's October 27 technical note shows Gemini 4 Argon with Antigravity first on the smoke score at 75.2%, Claude Opus 5.5 with Claude Code highest on all-tests-pass at 72.1%, and GPT-6.1 Sol with Codex cheapest at $1.25 per solved task.

Key Takeaways
- 204 migration tasks and 10 model-plus-harness configurations; top smoke scores are Gemini 4 Argon + Antigravity 75.2%, Claude Opus 5.5 + Claude Code 70.5% and Claude Opus 5 + Claude Code 69.8% (IBM figures)
- Claude Opus 5.5 + Claude Code has the highest all-tests-pass rate at 72.1%, just ahead of Gemini 4 Argon at 71.1%
- Compiling is mostly solved: 8 of 10 configurations compiled at least 97% of solvable tasks, yet smoke scores spanned more than 50 points
- 535 of 745 classified failing runs (about 72%) were runtime behavioral failures, including 210 lost-route/404 failures
- Cost per solved task: $1.25 for GPT-6.1 Sol + Codex, $1.59 for Gemini 3.8 Flash + mini-swe and $10.10 for Claude Opus 5.5 + Claude Code; public submissions are not open yet







