AI Coding Live News

X discussion: is GPT-6 Astra robotics' "GPT-3 moment"? Ex-OpenAI researcher Will Depue says specialized ML got "bitter lessoned"

Over the past day, X has been debating whether GPT-6 Astra can work directly as a robot policy. Former OpenAI researcher Will Depue (whose bio lists Sora 1 and 2 and o3/4o post-training) called Astra beating specialized robotics models "deeply surprising", then described it as "the final death of specialized ML". Investor Freda Duan posted that "Astra IS robotics' GPT-3 moment", drawing about 84K views. Third-party evaluations point both ways: in a RoboDojo paper, Astra averages 22.48% success and ranks above all 40 public policies, while an EBench report puts it at 46.7%, third of nine models. This is a discussion of opinions, not an official OpenAI conclusion.

X discussion: is GPT-6 Astra robotics' "GPT-3 moment"? Ex-OpenAI researcher Will Depue says specialized ML got "bitter lessoned"

Key Takeaways

  • Reach: Will Depue's two posts have about 597 and 566 likes and 59K and 37K views; Freda Duan's "10 takes" has about 455 likes, 559 bookmarks and 84K views (as of 19:10 SGT)
  • RoboDojo: over 42 tasks and 2,100 trials, Astra averages 22.48% success (Score 28.97), ranking above all 40 public policies; GPT-5.5 gets 0.88% in the same setup (arXiv 2609.24170, as reported by the paper's authors)
  • Counterpoint: in the EBench report, Astra with single-shot in-context learning scores 46.7%, behind OpenWAM-alpha at 55.3% and AMapbot 1 at 48.9% (EBench tech report)
  • Real arms: Robocurve measured 19 of 20 successes on a block-into-bowl task on YAM arms at about $0.94 per run, but only 2 of 20 on a puzzle insertion task (Robocurve)
  • Weak spots: several evaluations find Astra clearly weaker on high-precision control, dynamic control and bimanual coordination
Read full story

X discussion: former OpenAI researcher Will Depue says Anthropic is "massively underplaying" the Cathedral security test, drawing about 2,000 likes and 180K views

Will Depue, whose X bio says he previously worked at OpenAI on Sora 1 and 2 and on o3/4o post-training, posted on Oct 28 that Anthropic is "massively underplaying" the recent security incident and that he is "hearing" Cathedral got full administrator access to the company Slack workspace. The post drew about 2,000 likes, 600+ bookmarks and roughly 180K views. This is an unconfirmed secondhand claim; Anthropic told reporters the researchers reached only a limited set of sales, HR and IT systems and did not touch Claude conversations, source code or model weights.

Key Takeaways

  • Will Depue's post: Anthropic is massively underplaying the incident, and he is hearing Cathedral got full administrator access to the Slack workspace, able to see every message and log in as arbitrary users
  • His argument: Anthropic and OpenAI keep nearly all research discussion in Slack, so access tokens held by third-party apps become the weakest link
  • Anthropic's statement to The Washington Post: only a limited set of sales, HR and IT systems were reached, Claude conversations, source code and model weights were not accessed, and access was cut the same day
  • Cathedral's draft report says its AI agents found paths into internal systems through third-party administrative and sales software in under 12 hours
  • The full-admin-access claim is Depue's secondhand account and has not been publicly confirmed by Anthropic or Cathedral; the post has about 2,000 likes and roughly 180K views
Read full story
Linked Models & Agents:

Mirendil, founded by ex-Anthropic researchers, unveils Fern: its first from-scratch model with up to 10M tokens of context

On Oct 26 Mirendil unveiled Fern, its first model, pretrained from scratch on its own Mirendil-Base (0.1-0.5T params) with a context window of up to ~10M tokens, about 0.5M lines of code or days of agent traces. Fern 0.1 got only a short round of post-training and targets AI R&D work such as auditing training data, debugging runs from logs and reading agent traces end to end. Vendor-reported: 73.4% on SWE-Bench Verified and 100% single-needle RULER retrieval at 10M, while multi-hop QA drops to 45% at 10M. Access is early-access only; no price or weights yet.

Mirendil, founded by ex-Anthropic researchers, unveils Fern: its first from-scratch model with up to 10M tokens of context

Key Takeaways

  • Scale and context (vendor-reported): Mirendil-Base has 0.1-0.5T params; Fern reads up to ~10.2M tokens (about 23 MB or ~0.5M lines of code, roughly Megatron-LM scale), per the official blog
  • Coding scores (run by Mirendil, OpenCode, 1 attempt): SWE-Bench Verified 73.4, SWE-Bench Pro v2-Hard 76.5, SWE Atlas QnA 53.2, DeepSWE v1.1 44.2, Terminal-Bench 4.0 just 6.1; Claude Opus 5.5 in the same table scores 98.2/100.0/66.4/74.2/66.4
  • RULER long context (vendor-reported): single-needle retrieval is 100% from 32K to 10M; multi-hop HotpotQA falls from 65% at 32K to 45% at 10M, with only 20 samples per 10M cell
  • Cost (vendor-reported): one researcher plus agents built Mirendil-Base in 3.5 months for under $10M of compute, with agents writing over 99% of the code, per the pretraining write-up
  • Availability: Fern is the smallest of three models; Base-Large (0.5-2T) and Base-XL (2-10T) are still pretraining; access is via early-access requests only, with no price or weights announced
Read full story
Linked Models & Agents:
⚡ Fern 0.1⚡ Mirendil-Base🌐 Mirendil

fal and Black Forest Labs launch FLUX 3 Max video model: tied #1 in image-to-video quality, a 10-second clip in about 12 seconds, roughly 22x faster than Seedance 2.5

On Oct 19 (UTC) fal released FLUX 3 Max, a video model built by fal Research with Black Forest Labs through post-training, reinforcement learning and inference optimization on top of the FLUX 3 foundation model. It handles text-to-video, image-to-video and keyframes, up to 20 seconds, at 720p or 1080p, with audio on by default. On Artificial Analysis's preliminary image-to-video arena it scores 1,139 Elo, level with Seedance 2.5 and within the confidence interval of leader MiniMax H3 (1,141), and fal says a 10-second clip takes about 12.45 seconds. A 75% launch discount runs until Nov 7, putting 720p at $0.0425 per second.

fal and Black Forest Labs launch FLUX 3 Max video model: tied #1 in image-to-video quality, a 10-second clip in about 12 seconds, roughly 22x faster than Seedance 2.5

Key Takeaways

  • Artificial Analysis AA-Video-I2V v1.1 (preliminary): FLUX 3 Max at 1,139 Elo, level with Seedance 2.5 and just under MiniMax H3 at 1,141; "tied #1" rests on overlapping confidence intervals
  • Speed (fal-reported, 10-second 16:9 at 720p): FLUX 3 Max 12.45s, Gemini Omni Flash 1.1 47.37s, Seedance 2.5 277.80s; the press release quotes 10.95s instead
  • Versus the original FLUX 3: Elo up from 1,089 to 1,139 (+50) and a 10-second clip down from 126.44s to about 12s (11.5x per the press release, about 10x per the product page)
  • Pricing: promotional $0.0425/s at 720p and $0.0725/s at 1080p until Nov 7, then $0.17/s and $0.29/s, billed per output second
  • API options: 5-20 second duration, multiple aspect ratios, reasoning_level (low/high), generate_audio on by default; model IDs blackforestlabs/flux-3-max/*
Read full story
Linked Models & Agents:
⚡ FLUX 3 Max⚡ FLUX 3⚡ Seedance 2.5⚡ MiniMax H3🌐 fal🌐 Black Forest Labs🌐 Artificial Analysis
ADSponsored

Judge-Free Test-Time Scaling Tops WebArena Infinity at 74.6%: Salesforce Introduces CLIFT, Alongside MMSearch-OPD and SCOPED-8B Beating 32B VLMs

Three new frameworks advance web and multimodal agent post-training: Salesforce AI Research's CLIFT (arXiv:2610.06829) certifies URL-conditional self-verification questions for dense RL rewards and judge-free Conformal Trajectory Selection (CTS), propelling Gemma-4 31B to a 74.6% SOTA on WebArena Infinity (beating Gemini 3 Flash at 70.1%) and lifting GPT-5.5 to 53.7% on VisualWebArena. Meanwhile, MMSearch-OPD (arXiv:2610.22240) boosts 9B multimodal deep search by +15.3 points to 60.2%, and Northeastern University's SCOPED-8B (arXiv:2610.22217) uses skill-decoupled on-policy expert distillation to reach 73.13% across 8 benchmarks—surpassing Qwen3-VL-32B (72.01%) and Claude Opus 4.6 (72.86%).

Judge-Free Test-Time Scaling Tops WebArena Infinity at 74.6%: Salesforce Introduces CLIFT, Alongside MMSearch-OPD and SCOPED-8B Beating 32B VLMs

Key Takeaways

  • Salesforce's CLIFT conformal self-verification propels Gemma-4 31B to 74.6% on WebArena Infinity (+12.8 pts over base, beating Gemini 3 Flash at 70.1%) with judge-free test-time trajectory selection
  • Transferring the open-trained CLIFT question bank to GPT-5.5 sets a 53.7% SOTA on VisualWebArena, while MMSearch-OPD boosts 9B multimodal deep search accuracy by +15.3 points to 60.2%
  • Northeastern University's SCOPED-8B reaches 73.13% average accuracy across 8 VLM benchmarks via skill-decoupled on-policy expert distillation, beating Qwen3-VL-32B (72.01%) and Claude-Opus-4.6 (72.86%)
Read full story
Linked Models & Agents:

Ollama Cloud Launches US/EU Zero-Data-Retention Hosting for DeepSeek-V4.1-Flash-1017 with Claude Code CLI, While SemiAnalysis Reveals GLM-5.3 Was Built 100% via Post-Training

Ollama launched US- and EU-hosted Ollama Cloud with strict zero data retention and zero training, adding immediate support for DeepSeek-V4.1-Flash-1017 via one-line launch commands for Claude Code and OpenCode alongside Ollama v0.40.6. Meanwhile, SemiAnalysis revealed that GLM-5.3 was built entirely by post-training the previous GLM-5.2 base model without pretraining from scratch, achieving a 'night and day difference' in cybersecurity and agentic capabilities.

Ollama Cloud Launches US/EU Zero-Data-Retention Hosting for DeepSeek-V4.1-Flash-1017 with Claude Code CLI, While SemiAnalysis Reveals GLM-5.3 Was Built 100% via Post-Training

Key Takeaways

  • Ollama Cloud launches US- and EU-hosted zero-data-retention serving for DeepSeek-V4.1-Flash-1017 with one-line ollama launch integrations for Claude Code and OpenCode
  • Ollama v0.40.6 adds GGUF q2_0 tensor support, clef cloud integration, and fixes orphaned runner processes on Windows
  • SemiAnalysis reveals GLM-5.3 was built entirely by post-training the previous model without pretraining from scratch, delivering a 'night and day difference' in cyber capabilities over GLM-5.2
Read full story
Linked Models & Agents:

Alibaba Synthesizes 64.9K Interactive Environments in Cowork-Gym (27B Beats 100B+ on ClawEval) & Harvard/MIT Open-Source Market-Based Agent Swarms (MPT)

Alibaba researchers introduce Cowork-Gym (arXiv:2610.17670), synthesizing 64,900 interactive human-agent coworking environments paired with a stateful user simulator that dynamically edits the shared workspace; post-training a 27B Qwen model yields #1 scores on PawBench and ClawEval, outperforming 100B+ parameter models. Concurrently, Harvard and MIT researchers open-source Market-Based Post-Training (MPT, arXiv:2610.22322), enabling decentralized agent swarms to jointly learn action policies and bidding-based self-orchestration without a central coordinator.

Alibaba Synthesizes 64.9K Interactive Environments in Cowork-Gym (27B Beats 100B+ on ClawEval) & Harvard/MIT Open-Source Market-Based Agent Swarms (MPT)

Key Takeaways

  • 64.9K synthesized environments in Cowork-Gym: pairs a build-execute-revise loop with a stateful user simulator that edits shared workspaces alongside the agent
  • 27B model beats 100B+ giants on PawBench & ClawEval: post-trained Qwen models gain up to +8.7 average points across 6 agentic benchmarks
  • Market-Based Post-Training (MPT) for agent swarms: Harvard and MIT open-source a bidding-based RL framework for decentralized self-orchestration
Read full story

Athanor: 4 Autonomous Claude Opus 5.5 Agents Post-Train Qwen3-4B-Base over 104 Hours, Surpassing the Official Human-Trained Instruct Model

To test whether frontier agents can autonomously conduct end-to-end post-training research, researchers introduce Athanor, an agent-native system for autonomous post-training (AutoPostTrain) on real GPUs. In a 104-hour campaign without human advice or external teacher models, four Claude Opus 5.5 agents—each operating an 8-GPU node—improved raw Qwen3-4B-Base from 15.9 to 67.3 on the dev suite and from 9.2 to 48.2 on an unseen held-out suite, surpassing the official Qwen3-4B-Instruct-2507 (66.5 and 47.9).

Athanor: 4 Autonomous Claude Opus 5.5 Agents Post-Train Qwen3-4B-Base over 104 Hours, Surpassing the Official Human-Trained Instruct Model

Key Takeaways

  • 104-hour autonomous post-training campaign: four Claude Opus 5.5 agents on 8-GPU nodes autonomously manage data curation, SFT, RL algorithm modification, and weight merging
  • Surpasses official human-trained Instruct model: lifts Qwen3-4B-Base to 67.3 (dev) and 48.2 (held-out), beating Qwen3-4B-Instruct-2507 (66.5 and 47.9)
  • Zero external teacher distillation: achieves cross-domain generalist superiority purely through public data recombination, custom RL rollouts, and self-distillation
Read full story
Linked Models & Agents:

Microsoft AI finishes pre-training MAI-Base-2: 2.8T-total / 80B-active MoE trained on 16K GB200s in 38 days, which MAI estimates is 4.2–12.3x more FLOPs-efficient than DeepSeek-V4-Pro-Base

On 2026-10-15 Microsoft AI published a technical post and PDF saying it has finished pre-training its in-house base model MAI-Base-2: MoE in every layer (1 shared + 7-of-512 routed experts), 80B active / 2.8T total, 102 layers, native encoder-free vision, a 50T-token mixture and a March 2026 knowledge cutoff. Per MAI, it beats every open-weight model tested on held-out BPB and its recipe is 4.2–12.3x more pre-training-FLOPs-efficient than DeepSeek-V4-Pro-Base. It is still in post-training: no API, no weights and no release date yet; Microsoft plans to license weights to trusted partners via Foundry.

Microsoft AI finishes pre-training MAI-Base-2: 2.8T-total / 80B-active MoE trained on 16K GB200s in 38 days, which MAI estimates is 4.2–12.3x more FLOPs-efficient than DeepSeek-V4-Pro-Base

Key Takeaways

  • Scale: 80B active / 2.8T total, 102 layers, hidden 8704, MoE in every layer (1 shared + 7 of 512 routed); MAI-Base-1 was 35B/1T, 78 layers, alternating dense/MoE (PDF)
  • Compute: 16K GB200s for 38 days (Aug–Sep 2026), 21% average MFU (vs 2.5 PFLOPS BF16), 89% goodput; the Muon step is 2.1% (165 ms) of iteration time
  • Efficiency (vendor estimate via scaling-law fits): 4.2–12.3x fewer FLOPs than DeepSeek-V4-Pro-Base to match its BPB; 6.3–12.7x better than the June MAI-Base-1 recipe
  • Data: 50T-token mixture, 18% vision-language (4.5T unique), ~36% more unique tokens than Base-1, described as organic data with no distillation
  • Availability: still in post-training, no API/weights/date; weights to be licensed to trusted partners through Foundry; MAI-Base-3 prep underway (blog)
Read full story
Linked Models & Agents:
⚡ MAI-Base-2⚡ MAI-Base-1⚡ MAI-Thinking-1⚡ DeepSeek-V4-Pro-Base⚡ Kimi K3⚡ Qwen3.8-2.4T-A95B🌐 Microsoft🌐 NVIDIA🌐 DeepSeek

Paper Instruments releases Ultramarine 1 Flash: non-verifiable-RL knowledge-work model on GLM-5.3 Flash, $0.13/task on its own suite, API live, weights promised

San Francisco lab Paper Instruments launched Ultramarine 1 Flash on 2026-10-14 16:02 UTC: GLM-5.3 Flash post-trained with large-scale non-verifiable RL (per-task rubric rewards) for finance, legal, consulting and operations work. Its report claims +4.1 points on average over the base across six knowledge-work benchmarks with 6.5% fewer output tokens, and 59.3 at $0.13/task on its internal PI Knowledge Work suite vs Claude Fable 5 at 57.8 / $8.19. An OpenAI-compatible API (ultramarine-1-flash) is live; the CEO says weights follow in about two weeks and post-training cost ~$300k. The headline cost comparison rests on an internal benchmark and has not been independently reproduced.

Paper Instruments releases Ultramarine 1 Flash: non-verifiable-RL knowledge-work model on GLM-5.3 Flash, $0.13/task on its own suite, API live, weights promised

Key Takeaways

  • GLM-5.3 Flash base + modified CISPO RL (LoRA rank 64, 131K context, up to 250 turns); +4.1 avg over base on 6 knowledge-work benchmarks with 6.5% fewer tokens
  • JobBench 62.9 (#1, Gemini 3.8 Flash 60.6); FrontierFinance 53.3 (#1); APEX-Agents 1.1 57.0 vs Gemini 3.8 Flash 64.3
  • Internal PI Knowledge Work: 59.3 at $0.13/task vs Claude Fable 5 57.8 / $8.19 and GPT-6 Sol 61.3 / $1.26
  • Trade-off: slight coding regression — Terminal-Bench 2.1 83.2 (base 84.3), DeepSWE 1.1 61.5 (base 63.4)
  • Curriculum: 4 domains, 19 practice areas, 88 synthetic environment types, fully synthetic tasks graded by rubrics
Read full story
ADSponsored

ClawLoop Debuts: Harness-Native Self-Improving Training Loop Enables Teacher-Free Autonomous Agent Evolution

While LLM agents are rapidly deployed into sandbox workspaces with direct shell and filesystem access, post-training methods remain predominantly one-shot and bound by external teacher models. ClawLoop introduces a self-improving, harness-native training loop that operates without external teachers. Through a self-sustaining cycle of task synthesis with difficulty growth, SFT/RL training, and automated behavioral diagnosis, ClawLoop demonstrates consistent empirical gains across SkillsBench, Claw-Eval-Live, and ClawGym-Bench on Qwen3.6-27B, while successfully transferring data to 4B and 9B models.

ClawLoop Debuts: Harness-Native Self-Improving Training Loop Enables Teacher-Free Autonomous Agent Evolution

Key Takeaways

  • Harness-native self-improvement: couples task synthesis, SFT/RL, and behavioral diagnosis in a self-sustaining cycle for shell/sandbox agents
  • Diagnosis-conditioned difficulty growth: leverages failure analysis reports to progressively synthesize challenging yet verifiable workspace tasks
  • Cross-scale data transferability: boosts Qwen3.6-27B across SkillsBench and ClawGym-Bench, while transferring cleanly to 4B and 9B checkpoints
Read full story
Linked Models & Agents:

MMPostTrainBench & MMResearch: Benchmarking Autonomous Agents in Multimodal Post-Training and Mitigating Capability Regressions

Autonomous research agents hold the promise of sustaining AI self-improvement by iteratively proposing hypotheses, tuning models, and refining post-training pipelines. While agents demonstrate competence on text-only code tasks, their capacity to execute multimodal post-training across images, audio, and video remains uncharted. Researchers introduce MMPostTrainBench (arXiv:2610.05398), the first benchmark designed to evaluate autonomous research agents conducting multimodal post-training across eight tasks spanning image, audio, video, joint audiovisual comprehension, and image-grounded software repair. Agents operate from a common base model under fixed budgets with development feedback before independent evaluation. The audit reveals sobering realities: across all tasks, 52.1% of agent-submitted models fall below the baseline, exhibiting severe non-target capability regressions, and agents fail to reliably select the best candidate (trailing top models by up to 5.38 percentage points). To resolve multimodal research drift, the authors introduce MMResearch, a framework integrating media-grounded evidence and hierarchical memory. Integrated into coding agent runtimes, MMResearch improves submitted model accuracy by up to 7.75 percentage points for Claude Code (Claude Opus 4.8) and 2.33 points for Codex (GPT-5.6).

Key Takeaways

  • MMPostTrainBench benchmarks autonomous research agents conducting multimodal post-training across eight core tasks
  • Reveals that 52.1% of unguided agent post-training runs degrade baseline performance, suffering a 5.38 pp selection gap
  • Introduces MMResearch scaffolding, elevating Claude Code model accuracy by +7.75 pp and slashing regressions by 64%
Read full story
Linked Models & Agents:

NVIDIA and UC Berkeley Unveil Post-Training Performance Predictors for Coding Agents from Base Checkpoints

Agentic post-training (scaling multi-turn RL and SFT) requires massive GPU investments, but AI labs lack reliable heuristics to predict which base model checkpoint possesses the highest potential before committing to multi-million-dollar post-training runs. Cold-start zero-shot evaluations on SWE-bench fail because unaligned base checkpoints cannot emit formatted tool invocations, while single-turn patch benchmarks ignore multi-step state management. Researchers from NVIDIA and UC Berkeley (including Bryan Catanzaro and Ashish Vaswani) present a lookahead evaluation framework (arXiv:2610.10478). By replaying successful post-trained trajectories and rerunning unit tests after every edit, they isolate the 'decisive step' that flips a repository from failing to passing. They introduce three lightweight evaluation screens at this decisive step that require zero cold-start tool calling: Decisive-Action Bits Per Byte (BPB), Patch MCQ, and prefix-conditioned pass@K. Across ten pairs of public base and post-trained models, these screens rank candidate checkpoints in tight agreement with final post-trained SWE-bench Verified pass@1 scores, establishing a low-cost lookahead methodology for model selection.

Key Takeaways

  • NVIDIA and UC Berkeley introduce lookahead screens to predict post-training coding agent performance from base checkpoints
  • Isolates the 'decisive step' that flips test suites to passing, introducing Decisive-Action BPB, Patch MCQ, and prefix-pass@K
  • Ranks base model potential in tight agreement with post-trained SWE-bench Verified pass@1 across 10 model pairs
Read full story
Linked Models & Agents:

LoGRA: Scaling LLM Reinforcement Learning via Low-Rank Gradient Sketches, Slashing Memory by 45.7% for 27B Models

Reinforcement learning (RL) post-training is foundational to scaling reasoning capabilities in large language models. However, its memory footprint remains a prohibitive bottleneck: storing full-sized gradient buffers alongside multi-state Adam optimizer moments frequently triggers out-of-memory (OOM) failures when attempting to train large models (such as 27B parameters) on standard compute hardware. In newly released research (arXiv:2610.06647), researchers introduce LoGRA, an efficient RL post-training framework available within the open-source Molt library (github.com/skzhang1/labs-molt). LoGRA compresses training representations by retaining useful learning signals inside compact low-rank gradient sketches accumulated directly during backpropagation, facilitating both memory-efficient parameter updates and streamlined policy synchronization. To counteract instability from gradient compression, LoGRA introduces a Predicted-KL step control mechanism that forecasts policy change magnitudes prior to parameter updates, dynamically attenuating disruptive updates. Evaluated across reasoning benchmarks, LoGRA reduces average training memory by up to 45.7% with zero degradation in task performance, successfully sustaining stable RL training of a 27B model for over 1,100 steps on a single eight-GPU node where standard dense Adam crashes with OOM.

LoGRA: Scaling LLM Reinforcement Learning via Low-Rank Gradient Sketches, Slashing Memory by 45.7% for 27B Models

Key Takeaways

  • LoGRA framework released, introducing low-rank gradient sketches to slash LLM RL training memory by up to 45.7%
  • Breaks single-node 8-GPU memory barriers, sustaining stable RL training of a 27B model for over 1,100 steps
  • Predicted-KL step control prevents policy drift, achieving full parity with standard AdamW on reasoning benchmarks
Read full story

ReSPO: UCLA Introduces Reshaped Sequence Policy Optimization, Resolving Gradient Starvation in Off-Policy RLVR

Reinforcement Learning from Verifiable Rewards (RLVR) with long Chain-of-Thought (CoT) trajectories powers frontier reasoning models such as DeepSeek-R1 and OpenAI o-series. However, live on-policy sampling consumes more than 80% of cluster compute during post-training. To control training costs, practitioners frequently reuse rollout trajectories across multiple optimization steps, introducing severe off-policy distribution mismatch between the active policy and the rollout-generating policy. Researchers from UCLA unveil a fundamental failure mode in clipped policy optimization (arXiv:2609.35433): 'Sign-Dependent Gradient Starvation'. Standard clipping mechanisms in PPO and GRPO suppress under-generated positive reasoning rollouts relegated to the low-importance-weight tail, while allowing severely over-generated negative trajectories to dominate high-weight updates. This causes models to discard promising reasoning paths during early training. The authors propose ReSPO (Reshaped Sequence Policy Optimization), replacing rigid clipping with a smooth, two-branch sequence-level kernel derived from an alpha-divergence variational objective and an exponential variance-control tilt. Across dense and MoE Qwen3 architectures, ReSPO rescues long positive reasoning rollouts, accelerates early convergence, and achieves superior benchmark accuracy under rollout reuse.

Key Takeaways

  • UCLA identifies sign-dependent gradient starvation in off-policy RLVR, proposing ReSPO to replace clipped surrogate objectives
  • Protects sparse positive reasoning rollouts at low-weight tails, boosting early effective gradient throughput by 2.4x on Qwen3 models
  • Maintains monotonic convergence under 4x rollout reuse, elevating benchmark accuracy by 3.6 to 5.2 points while cutting sampling compute by 50%
Read full story