San Francisco lab Paper Instruments launched Ultramarine 1 Flash on 2026-10-14 16:02 UTC: GLM-5.3 Flash post-trained with large-scale non-verifiable RL (per-task rubric rewards) for finance, legal, consulting and operations work. Its report claims +4.1 points on average over the base across six knowledge-work benchmarks with 6.5% fewer output tokens, and 59.3 at $0.13/task on its internal PI Knowledge Work suite vs Claude Fable 5 at 57.8 / $8.19. An OpenAI-compatible API (ultramarine-1-flash) is live; the CEO says weights follow in about two weeks and post-training cost ~$300k. The headline cost comparison rests on an internal benchmark and has not been independently reproduced.
Key Takeaways
- ✓GLM-5.3 Flash base + modified CISPO RL (LoRA rank 64, 131K context, up to 250 turns); +4.1 avg over base on 6 knowledge-work benchmarks with 6.5% fewer tokens
- ✓JobBench 62.9 (#1, Gemini 3.8 Flash 60.6); FrontierFinance 53.3 (#1); APEX-Agents 1.1 57.0 vs Gemini 3.8 Flash 64.3
- ✓Internal PI Knowledge Work: 59.3 at $0.13/task vs Claude Fable 5 57.8 / $8.19 and GPT-6 Sol 61.3 / $1.26
- ✓Trade-off: slight coding regression — Terminal-Bench 2.1 83.2 (base 84.3), DeepSWE 1.1 61.5 (base 63.4)
- ✓Curriculum: 4 domains, 19 practice areas, 88 synthetic environment types, fully synthetic tasks graded by rubrics
Key Decision Metrics at a Glance
Turn your technical choice into a development budget
Compare 40 dev plans & simulate token costs vs $20/mo subscriptions
Project Links & Resources
Direct AccessIn-Depth Technical Analysis
Background
Agent RL has leaned on verifiable tasks like coding, where unit tests define reward. Professional work — diligence memos, litigation chronologies, board pre-reads — has no single correct answer, and open models lag closed flagships there. Paper Instruments released Ultramarine 1 Flash on 2026-10-14 16:02 UTC to push RL into this non-verifiable territory.
How it works
- Base: open GLM-5.3 Flash, trained with a modified CISPO objective in an async loop (LoRA rank 64, 131K context, up to 250 agent turns), plus R3 routing replay and length/invalid-call penalties.
- Curriculum: fully synthetic tasks across finance, law, consulting and operations — 19 practice areas, 88 environment types — each with an instruction, a sandbox (files + MCP services) and a weighted rubric of binary criteria designed to accept valid alternative answers.
- Feather harness: a ~50-token system prompt, dedicated read/edit/write/grep/glob tools with content-fingerprint checks, explicit background shell tasks, and open-source
paper-docx/pptx/xlsxforks for safe Office editing.
Results (vendor-reported, max reasoning)
- JobBench 62.9 (#1; Gemini 3.8 Flash 60.6, Claude Fable 5 57.4, base 56.3)
- FrontierFinance 53.3 (#1); AutomationBench 50.2 (#2); DiligenceBench 50.6; APEX-Agents 1.1 57.0 (Gemini 3.8 Flash 64.3)
- Coding regresses slightly vs base: Terminal-Bench 2.1 83.2 vs 84.3, DeepSWE 1.1 61.5 vs 63.4
- The "60x cheaper" claim comes from the internal PI Knowledge Work suite: 59.3 at $0.13/task vs Fable 5 at 57.8 / $8.19 and GPT-6 Sol at 61.3 / $1.26. That suite is not public; treat the cost claim as unverified until weights and third-party evals land.
For developers
- OpenAI-compatible API: base URL
https://api.paperinstruments.com/v1, modelultramarine-1-flash, defaultreasoning_effort="max"(reasoning cannot be disabled). Responses and Anthropic-style Messages endpoints are also offered — see the docs. - CEO Daanish Khazi says weights ship in about two weeks and that a five-person team post-trained the model for ~$300k.
- Use Feather's prompts and the Paper Office packages for Word/PowerPoint/Excel deliverables; prefer coding-specialist models for pure SWE work.
Benchmark side-by-side against alternatives, or calculate monthly token cost vs subscription break-even.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.