Omarchy Local Model Engine omarchy-local-ai 0.5.0 Ships: Nemotron 3.5 Bolt Becomes the 64 GB Default, and DHH Says Omarchy 4.6 Will Recommend the Best Local Model for Each Machine

Core Background & Industry Pain Points

The hard part of running a coding agent locally is often not the model itself but the questions around it: which model this machine can run, which inference engine, and how much context. Omarchy, the Arch-based Linux distribution led by DHH, wants to turn that into one command. PR #17267, merged November 1, adds omarchy agent local use best: it picks an open model from a curated recipe catalog by GPU family and RAM class, and the model then appears in the agents panel next to Claude and Codex subscriptions. On November 9 DHH posted that Omarchy 4.6 will make this "trivial", with a screenshot from the Ryzen AI Max+ 495 machine that AMD AI software VP Anush Elangovan gave him earlier this month, running a local Qwen3.8 Flash Next at 94 GiB.

Architecture Highlights & Internals

The engine and catalog ship together as the omarchy-local-ai package and update with omarchy update; the catalog is never fetched at runtime. Each recipe pins its runtime image by digest and its Hugging Face weights by revision with hash checks, and records measured memory peaks per context size. Models run in rootless Podman containers on an internal network with no egress, a read-only root and all capabilities dropped, with the API key passed as a Podman secret, and a post-start check confirms the server rejects unauthenticated requests. The runtimes serve Chat Completions, Responses and Anthropic Messages. v0.5.0 adds 64 GB GB10-class support, fixes the Bolt image refusing requests past 4,096 tokens once the page cache fills, and adds use --detach for background downloads.

Authoritative Benchmarks & Measured Scores

All figures below are the project's own measurements on its own hardware, not third-party evaluations. On a Ryzen AI Max+ 395 / 128 GB, the first Flash Next recipe peaked around 99 GiB with 1,607 tok/s prefill and 41.3 tok/s decode on a 32K prompt; after moving to Gufo 0.28.1 in v0.3.0 it decoded at 47.5 tok/s and 105 tok/s on code edits. Qwen3.8 27B moved to 4-bit at 20.7 tok/s and 40 GiB. On a 64 GB GB10, Nemotron 3.5 Bolt runs 91 tok/s on short prompts and 82 tok/s at 32K. No speed figures have been published for DHH's 495 / 192 GB machine.

Developer Hands-on Guide

Omarchy users can run omarchy agent local use best, or omarchy agent --local to launch OpenCode or pi on the local model; other agents can connect to the three APIs on the local port. The recommendation table covers only three classes today, Strix Halo 128 GB and GB10/N1x at 64 and 128 GB, so ordinary laptops have no recipe yet, and replies are asking about smaller machines and whether expected tok/s will be shown. The engine reserves 8 GiB of memory for the desktop. No release date for Omarchy 4.6 has been announced.