LM Studio posted on X at 15:32 UTC on Oct 29 that users can now download a new oMLX engine inside LM Studio and run it with MLX models they already have. The official changelog lists it in Bionic 1.1.10, alongside speculative decoding for MLX models and an in-app terminal. oMLX is an open-source Apple Silicon inference server maintained by developer jundot (Apache 2.0, about 23,000 GitHub stars) built around continuous batching and a two-tier RAM and SSD KV cache. LM Studio has not published speed comparisons for the engine.
Key Takeaways
- ✓LM Studio says the oMLX engine can now be downloaded inside the app and used with existing MLX models; the post has about 540 likes (post)
- ✓The Bionic 1.1.10 changelog (Oct 29) also lists speculative decoding for MLX models, a public in-app terminal and llama.cpp 2.66.0 extension packs (changelog)
- ✓oMLX is Apache 2.0 open source with about 23,000 GitHub stars and needs macOS 15+ on Apple Silicon M1 to M5 (GitHub)
- ✓Its core is a two-tier KV cache: hot blocks stay in RAM, overflow is written to SSD as safetensors, and matching prefixes are restored from disk even after a restart
- ✓LM Studio has published no speed comparison with its existing MLX engine; one user reported that restart-to-update failed after launch and LM Studio is investigating
Key Decision Metrics at a Glance
Turn your technical choice into a development budget
Compare 40 dev plans & simulate token costs vs $20/mo subscriptions
Project Links & Resources
Direct AccessIn-Depth Technical Analysis
Background
The main pain of running local models for coding on a Mac is not generation speed but long-context prefill. Agents such as Claude Code and OpenCode resend long system prompts and history every turn, and when the context changes many local servers recompute from scratch, which can mean tens of seconds of waiting. oMLX was built for exactly this, and its author says in the README that the goal was to make local models practical for real coding work. A user asked LM Studio's mlx-engine repo for oMLX's caching back in March; LM Studio has now shipped it as a downloadable engine.
How it works
oMLX uses a block-based KV cache inspired by vLLM, with prefix sharing and copy-on-write, split into two tiers. Frequently used blocks stay in RAM, and when that fills up, blocks are offloaded to SSD in safetensors format and restored from disk when a matching prefix returns, even after a server restart. It also does continuous batching on mlx-lm's BatchGenerator and supports serving several models with least-recently-used eviction and per-model idle timeouts. In LM Studio, oMLX is a separately downloadable engine next to the existing MLX engine, and MLX models already on disk work without conversion.
Benchmarks
LM Studio has not published any speed comparison between the oMLX engine and its existing MLX engine. The oMLX site hosts a community benchmark table; for example, one Oct 30 entry for an M3 Ultra with 96 GB running Qwen3.8-Flash-Next at 4-bit with 8k context shows about 1,050 tok/s prefill and 56 tok/s generation. These are self-reported community runs that vary widely by chip, quantization and context, so they are not official LM Studio results.
Getting started
After updating to Bionic 1.1.10, download the oMLX engine in runtime settings and load your existing MLX models with it. It pays off most in long coding-agent sessions with local models, where repeated prefixes make the SSD cache valuable; keep in mind the cache uses disk space. On launch day one user reported that restart-to-update failed and LM Studio said it is investigating, so restart the app manually if you hit that.
Benchmark side-by-side against alternatives, or calculate monthly token cost vs subscription break-even.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.