NVIDIA Introduces EdgeWise: Edge-Cloud Collaborative Agent Framework Slashes SWE-Rebench Inference Cost by 62% Without Losing Accuracy

Background and the Problem

Long-horizon software engineering and system administration agents routinely execute 30 to 100+ interaction turns per task. Invoking a frontier cloud LLM at every single step creates prohibitive API costs and latency bottlenecks. Crucially, empirical trajectory analysis reveals that the vast majority of steps in long-horizon workflows involve routine file inspection, directory traversal, or boilerplate verification—while only a small fraction of pivotal reasoning bottlenecks genuinely require frontier model capacity.

Architecture and How It Works

To unlock cost-efficient long-horizon autonomy, researchers at NVIDIA introduced EdgeWise, an edge-cloud collaborative agent framework:

1. Edge SLM Primary Solver with Selective Cloud Delegation: Deploys a lightweight Small Language Model (SLM) directly on the local edge device as the primary execution engine, selectively delegating only high-complexity bottleneck steps to a cloud LLM;
2. Step-Level RL for Joint Solving and Delegation: Eliminates the need for an auxiliary router model at inference time by introducing a step-level reinforcement learning algorithm that jointly optimizes task execution and autonomous delegation decisions with fine-grained per-step credit assignment;
3. Systematic Design Space Exploration: Pairs autonomous RL delegation with an external judge-based analysis framework to rigorously quantify the four pillars of edge-cloud agent collaboration: when to delegate, how to trigger delegation, what compressed context to transmit, and optimal cloud invocation frequency.

Benchmarks and Measured Results

Extensive evaluations across software engineering and terminal benchmarks confirm that EdgeWise matches or exceeds pure-cloud execution at a fraction of the cost:

• Up to 62% Cost Reduction on SWE-Rebench: Matches or outperforms 100% cloud LLM execution on SWE-Rebench while slashing total inference expenditure by up to 62%;
• 34% Savings on Terminal-Bench 2.1: Reduces inference cost by 34% on Terminal-Bench 2.1 without sacrificing task resolution accuracy;
• Verified Workstation Deployment: Real-world deployment on a local workstation demonstrates seamless edge-cloud handoffs and substantial latency and API cost savings.

Getting Started for Developers

Engineering teams building local-first coding assistants and enterprise IDE agents can immediately apply EdgeWise's architecture:

• Hybrid Edge-Cloud Execution Loop: Pair a local 8B–14B coding model (serving via vLLM or llama.cpp) for high-frequency exploratory tool calls (grep, view_file, test runner checks) with selective cloud escalation for multi-file patch synthesis;
• Context Compression on Escalation: Transmit structured state summaries rather than raw verbose terminal histories when delegating a step to the cloud LLM to minimize input token billing.