Ollama v0.40 Released: Defaults to Apple Native MLX Runner on Apple Silicon for Doubled Local Throughput

Core Background & Industry Pain Points Mac workstations equipped with unified memory are prime drivers for local LLM inference and edge coding agents. Historically, runtime stacks relied on generic C++ wrappers over Metal shaders via llama.cpp, leaving significant memory bandwidth and Neural Engine compute unexploited. Under deep context windows, models struggled with thermal throttling and degraded token velocities. ### Architecture Highlights & Internals Ollama v0.40 overhauls its hardware abstraction layer by directly integrating Apple's open-source MLX machine learning framework. Computations benefit from dynamic graph compilation and lazy evaluation, preventing unnecessary buffer copies between unified RAM subsystems. The loader automatically detects compatible topologies (e.g., Qwen 3.8, Llama 3) to execute custom MLX FlashAttention kernels, reducing 14B model initialization times from 4.2s to 1.1s. ### Authoritative Benchmarks & Measured Scores Across M4 Max and M3 Pro hardware evaluations: streaming throughput on Qwen 3.8 (int4) under an 8k context window advanced from 34.6 to 64.2 tokens/sec (+85.5% speedup). Time-to-first-token dropped by 42.1% (from 480ms to 278ms), alongside a 26% drop in sustained wattage that prevents thermal throttling. ### Developer Hands-on Guide Developers on macOS can update seamlessly via brew upgrade ollama or direct DMG download, then verify native MLX acceleration with ollama run qwen3.8.