Ollama Natively Supports Recursive Flint 40B: 2.5B Active MoE Coding Agent Delivering 125+ tok/s on MacBook Pro

Background and the Problem

As autonomous coding agents transition from naive code autocomplete to multi-turn, multi-file repository maintenance, engineering teams face severe cost, latency, and privacy bottlenecks. Routing entire codebase graphs and continuous execution traces to cloud-hosted frontier models (such as Claude Opus or GPT-4o) generates substantial API expenses while introducing network latency and enterprise data compliance hurdles. Conversely, deploying traditional 7B or 14B dense models locally on developer laptops often results in brittle logic, context truncation, and hallucinated test suites. To bridge this dichotomy, Recursive (founded by Richard Socher) developed and open-sourced Recursive Flint (flint:40b), a Mixture-of-Experts (MoE) model engineered specifically for local agentic software engineering.

Architecture and How It Works

1. Sparse MoE Architecture (40B Total / 2.5B Active):

• Flint structures a 49-layer Transformer composed of 2 dense layers followed by 47 sparse MoE layers;
• Each MoE layer integrates 256 routed experts, routing each token to 6 selected experts alongside 1 dedicated shared expert, restricting per-token compute to roughly 2.5 billion active parameters;

1. Native 262K Context Window:

• Features a calibrated 262,144-token (256K) context window, capable of ingesting entire multi-module source trees, configuration manifests, and test logs without context window spill;

1. Optimized Local Execution Throughput:

• Thanks to its compact 2.5B active parameter footprint, Flint delivers 125+ tokens/sec on an M4 Pro MacBook Pro and over 150 tokens/sec on a 64GB RTX Spark workstation;
• Ships under an Apache 2.0 license with day-zero support across Ollama (ollama run flint), llama.cpp, Unsloth, and vLLM.

Benchmarks and Measured Results

Independent and verified benchmark results across industry-standard software engineering benchmarks:

• SWE-bench Suite:
• Scores 74.2% on SWE-bench Verified, setting a new high watermark for sub-50B open-weight architectures;
• Reaches 89.7% on SWE-bench Pro v2, dramatically outpacing comparable dense open baselines;
• Terminal and Instruction Capabilities:
• Registers 54.9% on Terminal-Bench 2.1 for multi-step CLI navigation;
• Attains 80.3% on LiveCodeBench v6, paired with 95.1% on IFEval and 89.2% on IFBench;
• Local Deployment Standing:
• Represents the top-performing open-weight MoE coding model capable of executing fully offline within 24GB–32GB unified memory consumer machines.

Getting Started for Developers

• Instant Deployment via Ollama:

  ollama run flint
  

• Ecosystem Integration:
• Weights are hosted on Hugging Face at RecursiveSI/flint under Apache 2.0;
• Compatible with Unsloth 3-bit/4-bit GGUF quantizations, llama.cpp, LM Studio, vLLM, and SGLang;
• Agent Integration Practice: Developers can drop flint:40b directly into local agent frameworks (e.g. OpenHands, Cline, Aider) to enable private, offline, high-throughput autonomous bug resolution.