AI Coding Live News — Real-Time 𝕏/Twitter Updates & Breakthroughs

VectorSpaceLab@VectorSpaceLab·Collected 21d ago

👀115

🔥98

Can Coding Agents Build Agents? AgentSWE Benchmark Exposes Severe Meta-Engineering Delivery Gaps

As software engineering shifts toward autonomous coding agents, delegating the creation of specialized agents is becoming routine. The AgentSWE benchmark establishes the first systematic evaluation of agent software engineering across three lifecycle stages: Creation, Editing, and Optimization. Findings reveal a pronounced gap between prototype demonstration and held-out test acceptance across leading frontier models.

Key Takeaways

First benchmark for agent software engineering: covers Creation, Editing, and Optimization across 25 real-world commission tasks
Rigorous commission-based evaluation: tests frozen submissions on held-out cases with hard pass/fail gates rather than superficial unit passes
Exposes systemic failure modes: reveals widespread overfitting to visible prompts, hollow runtimes behind complete syntax, and unsupported completion claims

Read full story

Linked Models & Agents:

⚡ Claude 3.5 Sonnet ⚡ GPT-4o ⚡ DeepSeek-Coder-V2 ⚡ Qwen 2.5 Coder 32B 🤖 SWE-agent 🤖 OpenHands 🤖 Cline 🌐 VectorSpaceLab 🌐 Tsinghua University

#AgentSWE#Coding Agents#Meta Engineering#Agentic Workflows#SWE-bench#Benchmark

SWE-bench@SWEbench·Collected 40d ago

👀132

🔥92

EmbodiedSWE Benchmark Released: First Code-Agent Evaluation Suite for Long-Horizon Dexterous Robotics

The SWE-bench initiative and robotics researchers (@SWEbench) unveiled EmbodiedSWE (arXiv: 2609.27308). Expanding benchmark standards beyond pure software repositories, EmbodiedSWE assesses AI coding agents on 250 challenging robotic manipulation tasks requiring physical reasoning, sensor calibration, and closed-loop motor control code generation.

Key Takeaways

Features 250 enterprise-grade robotics software engineering challenges across ROS2, Isaac Gym, and MuJoCo.
Implements closed-loop physics simulation verification where candidate code must successfully execute in simulator.
Reveals that leading frontier models resolve under 22% of tasks, underscoring the gap between software and physical control.
Complete benchmark dataset and dockerized evaluation harnesses are openly accessible on GitHub and Hugging Face.

Read full story

Linked Models & Agents:

⚡ Claude 3.5 Sonnet ⚡ DeepSeek-V3 🤖 embodied-coder 🌐 swe-bench

#SWE-bench#Embodied AI#Robotics#ROS2#Isaac Gym#Coding Agents

Together AI@togethercompute·Collected 65d ago

👀46

🔥89

View on 𝕏 ↗

Together AI Serves GLM-5.3 Flash: 320B-A18B MoE with 1M Context Delivering 2x Work per Dollar on DeepSWE

Together AI launched hosted Day-1 inference for GLM-5.3 Flash (320B-A18B MoE, 1M context), nearly matching Luna on DeepSWE benchmarks while completing more than twice the work per dollar.

Key Takeaways

Day-1 managed serving of 320B-A18B MoE lowers deployment hurdles for custom hybrid-attention architectures;
DeepSWE evaluations demonstrate over 2x work completed per dollar compared to closed frontier models;
Available immediately via Together AI Model Market and unified developer API for dedicated endpoints.

Read full story

Linked Models & Agents:

⚡ GLM-5.3-Flash