JevSpawn Accelerates Agentic Inference: Compositional Action Spaces Bridge Language Instructions with Probabilistic Speed

核心背景与行业痛点

Autoregressive token-by-token generation throttles multi-turn agent execution, creating severe latency bottlenecks as interactions extend across dozens of steps. While Jev-style probabilistic models achieve rapid inference over finite state fields, they require static, manually engineered candidate sets. This breaks autonomous agents operating in dynamic open environments where action spaces evolve dynamically from natural language context.

架构亮点与底层机制

Shanghai Jiao Tong University researchers introduce JevSpawn, dynamically bridging natural language task requirements with finite probabilistic exploration:

1. Parallel Action Spawning: Emits structural candidate action clusters concurrently in a single forward pass, eliminating iterative autoregressive turn delays.
2. Feedback-Driven Branch Selection: Leverages runtime environment signals to prune sub-optimal candidates and prioritize leading action branches.
3. Shared Prefix & Structure Caching: Reuses token prefix KV-caches across all candidate actions within the spawned cluster, reducing context recalculation overhead.
4. Instant Recovery from Cached Alternatives: Caches secondary viable candidates; upon runtime action failure, the agent rolls back to alternative branches instantly without re-invoking full LLM reasoning.

权威 Benchmark 与实测跑分对比

Benchmarked across eight interactive navigation and decision tasks against seven leading agent frameworks:

1. Outperforms 7 Prominent Baselines: Establishes consistent superiority in task completion rates over ReAct, Plan-and-Solve, and TypeSafe Jev architectures.
2. 45% Reduction in Inference Latency: Shared prefix attention and parallel spawning slash per-turn decision latency by more than 45%.
3. Training-Free Plug-and-Play Architecture: Gains stem entirely from structured test-time inference management, requiring zero model parameter updates.

开发者实战落地与开箱指南

JevSpawn is open-sourced on GitHub. Engineering teams building high-throughput agent controllers, robotic navigators, or desktop automation assistants can drop JevSpawn into existing inference pipelines to unlock state-machine-level execution speeds with LLM generality.