Memdoor reproduces GVS5H: a team of five Qwen3.8-27B agents averages 9.8/10 on the first 10 LiveCodeBench Hard problems, against 8.0 for a single Fable 5 call

Background and the problem

Frontier coding performance is usually bought with bigger, pricier closed models. The GVS5H paper that went viral in late October (Persis Capital Inc. Research Lab, arXiv 2608.26480, #1 on Hacker News on Oct 25) proposes "ledger-based zero-shot self-orchestration": no training, just fresh instances of one model coordinating through shared files. It reports gains of up to 25.6 percentage points on the 100 latest hard LiveCodeBench problems. The open question was whether anyone could reproduce a single paper's single-harness result. On Oct 31, guregodevo, author of the open-source coding agent Memdoor, published one.

Architecture and how it works

Memdoor's team-solve is a YAML workflow. plan reads the task and writes a plan plus a notes.md ledger; ideate-a and ideate-b propose in parallel, A the most direct provable algorithm and B one from a different family of technique; solve picks one and writes solution.py; test runs the samples via a script; review and repair fix the code from the real test output. Every role starts from a fresh context and communicates only through files on disk. The author byte-compared the requests against GVS5H's original with a test script and copied its temperatures, top_p, 128K reply cap and 10-round limit. The difference is hosting: OpenRouter instead of the authors' own FP8 vLLM.

Benchmarks and measured numbers

Project-reported: on the first 10 problems of the GVS5H list, five passes each, Memdoor averaged 9.8/10, the authors' own harness 9.6, and a single Fable 5 call 8.0. 49 of 50 runs solved, for $35.79. The author names two kinds of loss: a manager inventing a wrong counterexample that got a correct program rewritten, and a program that already passed the hidden tests being overwritten in a later round. The limits are spelled out: only 10 problems, contests from March and April 2025 so contamination can't be ruled out, and the Fable 5 column comes from the original authors' files rather than this harness. So it shows the method reproduces, not that a 27B model beats Fable 5 across the board.

Getting started

Install Memdoor with its official script, run memdoor connect openrouter (anthropic, openai, gemini and chatgpt also work, or memdoor connect local for Ollama, llama.cpp, vLLM or LM Studio), copy examples/workflows/team-solve into ~/.memdoor/workflows/, put task.md and paired tests/N.in and N.out files in a problem folder, then run memdoor workflow run team-solve --wait. The workflow names no model, so a local one works. Two ideas are worth borrowing: let a script, not a model, decide success, and have roles hand off only through files so runs are auditable and resumable. A median problem costs $0.39 but takes a long time, so this suits offline batch work more than interactive completion.