AICoder

AI Coding Live News

First-hand AI coding updates from 𝕏/Twitter frontier labs and researchers, distilled by AI with entity benchmarks.

Explore live on 𝕏 ↗

AI Coding Live News

O
OpenCode@opencode·
👀255
🔥80
View on 𝕏 ↗

OpenCode team ships Hex 3.0 stable: local voice dictation for Mac, Windows and Linux, built for talking to coding agents

Anomaly, the company behind open-source coding agent OpenCode, shipped Hex 3.0 stable: hold a shortcut, speak, and the text is pasted at your cursor. Recognition runs on-device, local model memory drops from about 900 MB to about 160 MB, and it adds vocabulary hints, a hex CLI and optional OpenCode rewriting.

OpenCode team ships Hex 3.0 stable: local voice dictation for Mac, Windows and Linux, built for talking to coding agents

Key Takeaways

  • Hex v3.0.0 was released on GitHub at 00:38 UTC on Oct 31; it is MIT-licensed, written in Rust, and the repo has about 1,557 stars (release notes)
  • Vendor figures: local model memory drops from about 900 MB to about 160 MB; since 2.3.0 it streams, so a 2-minute dictation finishes about 0.1 s after release on an RTX 5070 Ti
  • Runs on macOS 15+ (Apple silicon), Windows x64/ARM64 and Linux x64/ARM64, with new .deb and .rpm packages on Linux
  • 3.0 does not auto-update from 2.3.1 and starts fresh, so settings, History and models must be set up again; Voice Action and spoken commands are removed
  • The @opencode launch post drew about 2,400 likes, 1,600 bookmarks and 100k+ views (post)
Read full story
Linked Models & Agents:
⚡ Parakeet⚡ Whisper🤖 OpenCode🤖 Hex🌐 Anomaly
#Hex#OpenCode#voice dictation#local speech recognition#open source#Rust#developer tools
C
Chris Tate (Vercel Labs)@ctatedev·
👀59
🔥82
View on 𝕏 ↗

Vercel Labs launches agent-desktop: computer use for any coding agent via OS accessibility trees, not pixels, on macOS, Windows and Linux

On October 31, Chris Tate of Vercel Labs, creator of agent-browser, announced agent-desktop on X as "agent-browser, for your whole computer". It is an Apache-2.0 Rust CLI and C library that reads a desktop app's OS accessibility tree and returns every actionable element as a JSON ref; the agent clicks or types on a ref and snapshots again to verify, with no pixel matching, headless by default. Version 1.0.0 shipped on October 27 and added Linux support; the current npm release is 1.0.2.

Vercel Labs launches agent-desktop: computer use for any coding agent via OS accessibility trees, not pixels, on macOS, Windows and Linux

Key Takeaways

  • Install: npm install -g agent-desktop, one package with native x64/ARM64 binaries; 1.0.0 (Oct 27) added Linux (glibc 2.35+, via AT-SPI), and the latest npm release is 1.0.2 (Oct 30)
  • Loop: snapshot prints a JSON tree with refs, click @<snapshot_id>:eN acts, then snapshot again to verify; a changed UI returns STALE_REF, and errors say whether a retry is safe
  • Scope: the README lists 60 command names, 56 of them operational, spanning windows, keyboard and mouse, notifications, clipboard, waits and session traces
  • Headless by default: ref actions go through accessibility APIs without moving the pointer or stealing focus, with --headed for physical input; Chris Tate says multiple agents can use the computer in parallel, each with its own cursor
  • Benchmarks: no official success-rate or speed numbers; it is a tool with no model inside, and web pages stay with agent-browser
Read full story
Linked Models & Agents:
🤖 Claude Code (Anthropic)🤖 codex🤖 cursor🌐 Vercel
#Agent desktop#Vercel Labs#computer use#accessibility tree#Agent browser#Rust#open source
A
Anthropic Frontier Red Team@AnthropicAI·
👀62
🔥86

Anthropic's Frontier Red Team details five exploit chains an unreleased internal Claude model built for Chrome, Safari, Android, Windows and Linux; Chrome 156 fixes 62 CVEs

Anthropic's Frontier Red Team published a research post on Oct 27 saying it spent the past few months using an unreleased internal research model to find critical vulnerabilities in browsers and operating systems, and that after initial prompting Claude wrote five full exploit chains: one-click remote code execution with a sandbox escape in Chrome, a same-origin policy bypass in Safari, data extraction from a locked Android phone over USB, a remote denial of service against Windows HTTP/3 (QUIC), and zero-click root code execution on Linux servers with a multipath TCP listener. Anthropic says Google assigned 62 CVEs to its reports, all critical or high, fixed as of Chrome 156, and every issue went through coordinated disclosure and has been fixed. The model's name, the amount of human involvement and the compute cost were not disclosed.

Anthropic's Frontier Red Team details five exploit chains an unreleased internal Claude model built for Chrome, Safari, Android, Windows and Linux; Chrome 156 fixes 62 CVEs

Key Takeaways

  • The Chrome chain links 4 bugs: V8 heap corruption, a V8 sandbox escape that defeats ASLR, a CacheStorage race condition and a FedCM use-after-free; the PoC launches Calculator within seconds of a page loading in default Chrome on macOS
  • Per Anthropic, Google assigned 62 CVEs to its reports, all critical or high, including 13 sandbox escapes and 21 renderer-compromise bugs, fixed as of Chrome 156
  • The Windows kernel QUIC out-of-bounds read is CVE-2026-33096, with likely several million exposed servers by Anthropic's estimate; the Linux MPTCP zero-click root bug is CVE-2026-98432, where the PoC leaks a kernel pointer in about 48 minutes and reaches root command execution about 5 minutes later
  • Method: many parallel scanning agents scored by the extra vulnerabilities they add, with coverage tracking and automated variant analysis on every confirmed finding; Anthropic recommends scanning at PR time and has released a Claude Code security plugin
  • The model is an unnamed internal research model; Anthropic did not publish the share of human work, failed attempts or compute cost
Read full story
Linked Models & Agents:
⚡ Claude (unreleased internal research model)🤖 Claude Code🌐 Anthropic🌐 Google🌐 Apple🌐 Microsoft
#Anthropic#Claude#Security#vulnerability research#Chrome#Safari#Linux#Frontier Red Team
M
Model Context Protocolmodelcontextprotocol·
👀44
🔥82

MCP reference servers v2.1.0 fix eight vulnerabilities: upgrade filesystem and everything to 2.0.1 and fetch to 2026.10.24

On Oct 25 (UTC) the official Model Context Protocol servers repo shipped v2.1.0, fixing eight vulnerabilities in three reference servers: symlink races, trailing quote/whitespace path bypasses and case-insensitive symlink bypasses in filesystem; cross-origin calls over HTTP/SSE in everything; and IDNA host-name and NAT64 internal-address reachability plus robots.txt not rechecked after redirects in fetch. The maintainers warn that npm users still on date versions (2025.x/2026.x) will not be offered the upgrade and must install 2.0.1 explicitly, and that the filesystem write issue is only partly fixed on Windows.

Key Takeaways

  • Eight vulnerabilities fixed across three servers (release notes)
  • filesystem, 4 fixes: symlink race on writes, trailing quote/whitespace bypass, case-insensitive symlink bypass, list_directory_with_sizes leaking metadata outside allowed dirs
  • everything, 1 fix: HTTP/SSE transports no longer send Access-Control-Allow-Origin: *, so web pages can no longer call tools like get-env
  • fetch, 3 fixes: IDNA host-name check vs. connection mismatch, NAT64-wrapped private IPv4, robots.txt not rechecked after redirects
  • Date-versioned npm installs must run npm install @modelcontextprotocol/[email protected] explicitly; Windows is only partly fixed
Read full story
Linked Models & Agents:
🤖 Claude Code🤖 Cursor🤖 Codex🌐 Anthropic🌐 Model Context Protocol
#MCP#Security#Filesystem#Fetch#SSRF#Symlink#CORS
ADSponsored
D
Dmitry Lyalin@LyalinDotCom·
👀98
🔥86
View on 𝕏 ↗

Google previews Gemini API CLI (gemini-api), a zero-dependency Go binary that lets Claude Code, Codex, Cursor and other coding agents generate media, transcribe and run managed agents from the shell

The Google AI Studio team released an experimental preview of the Gemini API CLI (command name gemini-api). It is a stateless, zero-dependency Go binary for macOS, Linux and Windows, open source under Apache 2.0. Any coding agent with shell access and a GEMINI_API_KEY can call text reasoning, image, video, music and speech generation, file and YouTube analysis, audio and video transcription, and managed agents such as Deep Research with one command. Media is written straight to disk and only the file path goes to stdout, so base64 payloads never land in the context window. Google labels it experimental and not a supported Google product.

Google previews Gemini API CLI (gemini-api), a zero-dependency Go binary that lets Claude Code, Codex, Cursor and other coding agents generate media, transcribe and run managed agents from the shell

Key Takeaways

  • Default models (official README): gemini-3.8-flash for generate, analyze and transcribe; gemini-nano-banana-2.1 for image; gemini-omni-1.1-flash for video; lyria-3.5 for music; gemini-3.8-flash-tts for speech
  • Install with a one-line curl script on macOS/Linux, a PowerShell script on Windows, or go install; the latest GitHub release is v0.4.3 (published 2026-10-19)
  • Built for agents: --usage prints the command tree as KDL, --schema prints JSON Schema, --dry-run validates offline with credentials redacted, plus --output-format json/toon and inline --jq
  • Exit codes separate 4 cases: 0 success, 1 runtime/API failure, 2 usage or validation error, 3 auth failure; --agent-mode emits structured JSON errors
  • It also manages sandbox environments, server-side credentials, webhooks and cron triggers for managed agents; files uploaded via the Files API are kept for 48 hours
Read full story
Linked Models & Agents:
⚡ Gemini 3.8 Flash⚡ Gemini Nano Banana 2.1⚡ Gemini Omni 1.1 Flash⚡ Lyria 3.5⚡ Gemini 3.8 Flash TTS🤖 Gemini API CLI🤖 Claude Code🤖 Codex🤖 Cursor🤖 Antigravity🤖 Deep Research🌐 Google
#Gemini API CLI#Gemini API#CLI#coding agents#Google AI Studio#managed agents#developer tools
V
Visual Studio Code@code·
👀66
🔥96
View on 𝕏 ↗

VS Code Preview Introduces Canvases: Shared Workspaces for Developers and Coding Agents

The Microsoft VS Code team announced Canvases in VS Code Preview (v1.142), introducing dedicated shared visual workspaces connecting developers with autonomous coding agents. Addressing the severe UX limitations of cramped sidebar chat windows—where complex multi-file architectural modifications and agent diffs become hard to track—Canvases provides an expansive, interactive infinite-canvas whiteboard. Developers and agents (such as GitHub Copilot, Claude Code, and Cline) can collaboratively organize system flowcharts, live interactive code diffs, execution traces, and unit test rollouts side by side.

VS Code Preview Introduces Canvases: Shared Workspaces for Developers and Coding Agents

Key Takeaways

  • Microsoft VS Code Preview v1.142 introduced Canvases, pioneering a spatial human-agent collaborative interface
  • Transceives beyond claustrophobic sidebar chats to an expansive, pan-and-zoom whiteboard mapping concurrent multi-file agent diffs
  • Embeds code files, architecture diagrams, interactive git diffs, terminal outputs, and test runners as unified interactive cards
  • Natively bridges GitHub Copilot, Claude Code, and third-party coding agent harnesses, accelerating IDE spatial development
Read full story
Linked Models & Agents:
🤖 GitHub Copilot🤖 Claude Code🤖 Cline🌐 Microsoft
#VS Code#Canvases#Coding Agent#IDE#Developer Tools#GitHub Copilot#Pair Programming
h
haofan@haofan·
👀244
🔥86

Antigravity CLI 1.3.5: Auto mode auto-approves safe edits and commands, reviews sandbox escapes via a separate agent, and adds /compact

Google Antigravity CLI shipped 1.3.5 on 2026-10-14 (UTC). Release notes add Auto mode: it auto-approves file edits and safe commands, still prompts before irreversible actions, and sends commands that would leave the terminal sandbox to a separate agent for review first. Enable with --mode auto, Agent Mode in /config, or Shift+Tab; Linux and macOS now, Windows later. The same cut adds on-demand /compact and upgrades /diff with syntax highlighting, folding, and viewport rendering for large files. Official modes docs still listed only default / accept-edits / plan at fetch time—treat the GitHub release as source of truth.

Antigravity CLI 1.3.5: Auto mode auto-approves safe edits and commands, reviews sandbox escapes via a separate agent, and adds /compact

Key Takeaways

  • Release: Antigravity CLI 1.3.5 (89081f7, 2026-10-14T03:16:53Z) leads with Auto mode
  • Auto: auto-approves file edits and safe commands; still prompts before irreversible actions; sandbox-leaving commands are reviewed by a separate agent first (per GitHub release)
  • Enable via --mode auto, Agent Mode in /config, or Shift+Tab; Linux/macOS now, Windows later
  • /compact on demand: alone summarizes and ends the turn; with a follow-up it summarizes then answers; Esc cancels; finish shows steps summarized and context shrinkage
  • /diff: syntax highlighting, hunk folding (Ctrl+O / f), viewport rendering; expand-all disabled over 100k lines or 10 MB
Read full story
Linked Models & Agents:
⚡ Gemini 4 Argon🤖 Antigravity CLI🤖 agy🌐 Google🌐 Google DeepMind
#Antigravity#Antigravity CLI#Agy#Auto mode#Compact#Sandbox#Google#CLI#Agent#1.3.5
H
Hugging Face Daily Papers@HuggingFace·
👀82
🔥98

RunningTab: KAIST Overcomes Coding Agent Context Drift via Environment-Side Tabs for Direct Workspace Interaction

In software engineering and knowledge-work tasks, autonomous coding agents operate via Direct Workspace Interaction (DWI)—issuing shell commands to navigate, inspect, and modify files within local repositories without external vector indexing. However, during extended interaction horizons, agents suffer severe context degradation: initial task requirements are purged from context windows by voluminous file dumps, read file provenances vanish, and listed but unopened candidate files are permanently forgotten, leading to premature task terminations and unfulfilled requirements. Researchers from KAIST introduce RunningTab (arXiv:2610.10444), an innovative agent interaction framework that replaces unreliable internal agent scratchpads with an environment-maintained 'running tab'. The tab persists three critical streams: dynamic requirement trackers, verified file excerpts with provenance links, and unopened candidate file stacks. When an agent attempts to finalize a deliverable, the environment triggers a mandatory 'Finish Check' barrier that blocks completion if unsatisfied criteria persist. Evaluated across Workspace-Bench, TheAgentCompany, and OfficeQA Pro benchmarks using frontier LLMs, RunningTab decisively outperforms vanilla DWI and self-refine baselines, establishing substantial gains in rubric satisfaction (TCR@70).

RunningTab: KAIST Overcomes Coding Agent Context Drift via Environment-Side Tabs for Direct Workspace Interaction

Key Takeaways

  • KAIST introduces RunningTab, pioneering environment-side tabs to eliminate long-horizon context drift in coding agents
  • Tracks Requirements, Provenances, and Candidate files externally, reinforced with a strict Finish Check barrier
  • Sweeps Workspace-Bench and TheAgentCompany, elevating TCR@70 to 25.67 and proving the superiority of environment-managed memory
Read full story
Linked Models & Agents:
⚡ RunningTab⚡ Direct Workspace Interaction⚡ Workspace-Bench🤖 RunningTab-Agent🤖 Finish-Checker🌐 KAIST🌐 Hugging Face🌐 TheAgentCompany
#RunningTab#Coding Agents#Context Window#Workspace Interaction#KAIST#Agent Architecture
P
Prime Intellect@PrimeIntellect·
👀116
🔥86
View on 𝕏 ↗

Prime Intellect's Prime Agent used a 2,209-agent swarm to rewrite itself in Rust: time-to-type 738ms→56ms, 80%+ less memory

On Oct 9 Prime Intellect shipped a Rust rewrite of Prime Agent, its open-source coding agent harness. A root orchestrator ran 2,209 sub-agents across 10,000+ Prime Sandboxes and ~228.7B GLM-5.3 tokens over two weeks, with differential TUI, harness and protocol parity tests gating every merge. Vendor-measured cold time-to-type dropped from 737.8ms to 55.8ms and post-startup memory from 607MB to 106MB; Windows (beta) and Homebrew installs were added.

Prime Intellect's Prime Agent used a 2,209-agent swarm to rewrite itself in Rust: time-to-type 738ms→56ms, 80%+ less memory

Key Takeaways

  • Scale: 2,209 agents, 16,758 agent-to-agent messages, 228.70B tokens (192.99B/1,981 agents for the port, 35.70B/228 for perf hillclimbing) across 10,000+ Prime Sandboxes
  • Vendor-measured: cold time-to-type 737.8ms→55.8ms (13.22x), first paint 722.8ms→23.6ms, post-startup RSS 607.4MB→106.0MB, install 172.1MB→59.6MB
  • Each task ran a Planner→Implementer→adversarial Reviewer (different model)→Verifier state machine; failures loop back to the implementer
  • A 3-day target-free hillclimb loop logged 144+ experiment/audit records and merged 69+ performance changes
  • Codebase now 9 crates, largest file ~2,500 lines (was ~15,000); native Windows (beta) and Homebrew installs, still open source
Read full story
Linked Models & Agents:
⚡ GLM-5.3🤖 Prime Agent🤖 Claude Code🤖 Codex CLI🌐 Prime Intellect
#Prime Agent#Rust#Agent Swarm
H
Hugging Face Daily Papers@HuggingFace·
👀256
🔥99

Xiaomi Unveils MiMo-V2.6: Scaling Reinforcement Learning Compute Up to 1M Context to Unlock End-to-End Self-Improvement

Reinforcement learning (RL) has emerged as the defining post-training paradigm driving frontier foundation models toward autonomous self-improvement, yet scaling RL across multi-step agent environments and million-token sequences introduces profound stability and infrastructure hurdles. Xiaomi's LLM-Core Team releases the comprehensive technical report for MiMo-V2.6 (arXiv:2610.11959), an omni-modal foundation model family that scales RL compute across three critical axes. First, it employs high-throughput asynchronous training processing 1,568 samples and 2.7 to 3.7 billion tokens per optimization step across context windows scaling up to 1M tokens. Second, it orchestrates training across diverse, multi-domain environments spanning code generation, computer vision, and cybersecurity under a heterogeneous mixture of agent harnesses. Third, it implements groupwise agentic grading to mitigate reward hacking and incentivize concise, token-efficient reasoning trajectories. By freezing the MoE router and establishing multi-layered defense barriers, MiMo-V2.6 guarantees rock-solid convergence at scale, while open-sourcing its RL framework, training dynamics, and interactive environments.

Key Takeaways

  • Xiaomi releases MiMo-V2.6, scaling reinforcement learning compute across 1,568 samples, 3.7B tokens per step, and up to 1M context
  • Introduces groupwise agentic grading to curb reward hacking and compress trajectory token lengths by over 35%
  • Freezes MoE routers to eliminate routing collapse and open-sources the complete RL post-training infrastructure
Read full story
Linked Models & Agents:
⚡ MiMo-V2.6⚡ Hybrid-SWA⚡ Agentic-RL⚡ MoE🤖 Xiaomi-LLM-Core🤖 Groupwise-Grader🤖 Self-Improving-Agent🌐 Xiaomi🌐 Hugging Face
#MiMo-V2.6#Xiaomi#Reinforcement Learning#Self-Improvement#1M Context#Open Source
ADSponsored
O
OpenAI DevelopersOpenAIDevs·
👀221
🔥86
View on 𝕏 ↗

Codex on Windows gets a Microsoft Execution Containers (MXC) sandbox: no admin setup, no extra accounts or firewall rules, native network policy and granular file access

On Oct 9 OpenAI Developers announced a new Codex sandbox mode on Windows built on Microsoft Execution Containers (MXC), promising faster setup, stronger network enforcement and granular file access controls on compatible Windows 11 devices. Docs now list MXC as the recommended implementation, with elevated/unelevated as legacy fallbacks. It ships in Codex CLI 0.162.0, where features.prefer_mxc is off by default in the standalone CLI; the desktop app enables it via rollout.

Codex on Windows gets a Microsoft Execution Containers (MXC) sandbox: no admin setup, no extra accounts or firewall rules, native network policy and granular file access

Key Takeaways

  • Windows sandbox implementations go from 2 to 3: mxc (recommended) / elevated (preferred fallback) / unelevated (weaker network isolation)
  • MXC needs zero admin elevation, zero extra Windows accounts, zero local firewall rules; commands run as the user with per-command policy
  • Minimum Codex CLI 0.162.0; features.prefer_mxc is off by default in the standalone CLI
  • Enterprises can block MXC with windows.allow_mxc = false in requirements.toml
  • Official post: 650+ likes, 51.8K impressions within ~1.5 hours
Read full story
Linked Models & Agents:
🤖 Codex🤖 Codex CLI🌐 OpenAI🌐 Microsoft
#Codex#Windows#MXC#Microsoft Execution Containers
S
Satya Nadella@satyanadella·
👀135
🔥92
View on 𝕏 ↗

Microsoft takes MAI-Code-1.1-Flash local: 137B MoE scores 70.8% SWE-Bench Verified on-device, GitHub Copilot to route between local and cloud

At its Oct 7 Windows event Microsoft shipped an on-device MAI-Code-1.1-Flash (137B total / 6.8B active MoE, 256K context, ~3.3 bits per weight) on RTX Spark PCs, scoring 70.80% SWE-Bench Verified and 66.29% Terminal-Bench 2.1 locally. GitHub HydraFusion will route Copilot tasks between local and cloud models in experimental preview later in October, with no inference charge for local calls; MXC agent containment went GA the same day.

Microsoft takes MAI-Code-1.1-Flash local: 137B MoE scores 70.8% SWE-Bench Verified on-device, GitHub Copilot to route between local and cloud

Key Takeaways

  • On-device: 70.80% SWE-Bench Verified (cloud 72.6%) and 66.29% Terminal-Bench 2.1 (cloud 62.9%); GPT-OSS-120B scored 32.0% / 23.6% in the same test (Microsoft Command Line)
  • Footprint: ~3.3 bits/weight mixed precision plus DFlash2 speculative decoding; 75.5GB peak memory at 256K; 923.5 / 769.8 tok/s prompt processing at 64K / 128K
  • Routing: HydraFusion local+cloud routing reaches the Copilot app, Copilot CLI and VS Code in experimental preview later in October, with no inference charge for local calls (Windows Blog)
  • Available now: Copilot CLI 1.0.94-0+ discovers local Ollama models via /model (tool calling + streaming required) (GitHub Changelog)
  • Security: MXC agent containment is GA on Windows 11, already supported by Codex, Copilot, OpenClaw and LM Studio, with Claude Code and Hermes Agent among those to follow
Read full story
Linked Models & Agents:
⚡ MAI-Code-1.1-Flash⚡ DeepSeek-V4-Flash⚡ Nemotron🤖 GitHub Copilot🤖 Copilot CLI🤖 HydraFusion🌐 Microsoft Azure AI🌐 github🌐 Nvidia (NIM)
#MAI-Code-1.1-Flash#GitHub Copilot#HydraFusion#Windows#MXC#RTX Spark
H
Hugging Face Daily Papers@HuggingFace·
👀90
🔥99

GUI-HARVEST: Self-Improving GUI Agents via Evidence-Driven Harness Evolution Lifts OSWorld by 12.3 Points and GPT-5 by 13.8%

The executable harness surrounding a GUI model governs how multimodal visual observations are assembled, how mouse/keyboard actions are executed, and how error recovery and termination gates operate. Automatically evolving this harness with frozen base models presents coupled challenges: grounding error diagnosis in visual UI transitions, attributing failures amid execution variability, and translating multi-task failure modes into reliable runtime code edits. Researchers introduce GUI-HARVEST (arXiv:2610.00948), an evidence-driven harness optimizer. GUI-HARVEST synchronizes model intentions with before-and-after screenshots, treats repeated rollouts as joint evidence units, and abstracts recurring failure patterns into bounded Python code edits applied directly to the harness. Evaluated on OSWorld-Verified across six general and GUI-specialized backbones, Qwen3-VL-32B-Instruct improves by 12.33 points. Remarkably, transferring the evolved harness to WindowsAgentArena without further optimization elevates GPT-5 by 13.87 percentage points at 50 steps, decisively outperforming Self-Harness and Meta-Harness. Code is available at github.com/GaryYang12345/GUI-HARVEST.

Key Takeaways

  • Introduces GUI-HARVEST, an automatic evidence-driven harness optimizer enabling self-improvement for GUI agents with frozen base models
  • Grounds error diagnosis in before-and-after UI screenshot deltas, translating multi-task failure modes into bounded Python source code edits
  • Lifts Qwen3-VL-32B by 12.33 points on OSWorld-Verified and transfers zero-shot to WindowsAgentArena, boosting GPT-5 by 13.87 points
Read full story
Linked Models & Agents:
⚡ GUI-HARVEST⚡ Qwen3-VL-32B⚡ GPT-5⚡ OSWorld-Verified🤖 GUI-Agent🤖 Harness-Optimizer🤖 Computer-Use-Agent🌐 Zhejiang University🌐 National University of Singapore🌐 Hugging Face
#GUI-HARVEST#Computer Use#GUI Agents#Harness Evolution#Self-Improving Agents#OSWorld
H
Hugging Face Daily Papers@HuggingFace·
👀94
🔥99

DeskForge: Scalable Dense Supervision from Controllable Desktop Environments Propels Computer-Use Agent Grounding by 11.5%

Computer-use agents face fundamental grounding bottlenecks in real-world desktop environments due to layered windows, shifting layouts, and visually indistinguishable controls. Existing training datasets suffer from sparse labels and static capture conditions. Researchers from ETH Zurich and IBM Research present DeskForge (arXiv:2610.02320), a controllable synthetic desktop environment that orchestrates real-world software across varied window states, themes, and screen resolutions. DeskForge synthesizes dense annotations by fusing desktop screenshots, accessibility trees, and geometric window hierarchies, yielding DeskForge-1M—a corpus of 1.2M observations and 159.7M annotated UI element instances. Fine-tuning vision-language models on 200K DeskForge-1M examples improves zero-shot GUI grounding across five external benchmarks; Qwen3.5-4B gains 11.51 percentage points on ScreenSpot-Pro and 10.11 points on OSWorld-G. In end-to-end task execution, task completions surge from 31 to 50 on WebArena-Infinity and from 3 to 15 on OpenApps. The dataset, models, and code are open-sourced at saidgurbuz.github.io/deskforge.

Key Takeaways

  • ETH Zurich and IBM introduce DeskForge, a controllable environment synthesizing DeskForge-1M with 1.2M observations and 159.7M UI elements
  • Fine-tuning Qwen3.5-4B boosts GUI grounding by 11.51 points on ScreenSpot-Pro and 10.11 points on OSWorld-G across five external benchmarks
  • Multiplies long-horizon task completion: solves 50/119 tasks on WebArena-Infinity (up from 31) and 5x on OpenApps (from 3 to 15), fully open-sourced
Read full story
Linked Models & Agents:
⚡ DeskForge⚡ Qwen3.5-4B⚡ WebArena-Infinity🤖 Computer-Use-Agent🤖 Desktop-Grounding-Agent🤖 GUI-Agent🌐 ETH Zurich🌐 IBM Research🌐 Hugging Face
#DeskForge#Computer Use#GUI Grounding#Desktop Agents#OSWorld#WebArena
H
Hugging Face Daily Papers@HuggingFace·
👀85
🔥99

Periscope: Extending Frozen LLMs Beyond Context Windows via Training-Free Grid Factorization (4.5M Tokens on a Single 80GB GPU)

Standard language models read long texts through quadratic attention passes that bound context windows, deplete GPU memory, and suffer from length-induced accuracy degradation. Researchers from the Technology Innovation Institute (TII) introduce Periscope (arXiv:2610.04047), a training-free inference methodology that factorizes context reading across a 2D grid. Observing that long-context reasoning over discrete decision spaces is fundamentally an evidence-localization task, Periscope arranges N text chunks onto a K x K grid (where K = ceil(sqrt(N))). It probes a frozen LLM with K consecutive local spans and K strided spans across the document, scoring single-token log-odds to generate a comprehensive evidence map whose peak pinpoints the exact answer chunk. For a context of length s and chunk size c, a window W scales to W^2 / c tokens at s^1.5 cost. As each forward probe caches only a single span, a 27B model processes 4.5M tokens on a single 80GB GPU—whereas a conventional pass would demand 296GB of KV cache. On InfiniteBench, Periscope outperforms the best full-window reads by 5 points while dominating long-document retrieval benchmarks.

Key Takeaways

  • Introduces training-free Periscope algorithm factorizing N chunks onto a K x K grid to scale physical window W to W^2 / c equivalent tokens
  • Synthesizes evidence maps via local and strided log-odds, enabling a 27B model to digest 4.5M tokens on a single 80GB GPU vs 296GB KV cache
  • Outperforms full-window reads on InfiniteBench by 5.2 points, matching 1M window benchmarks on LongBench v2 reading only 9k top-ranked tokens
Read full story
Linked Models & Agents:
⚡ Periscope⚡ InfiniteBench⚡ LongBench v2🤖 Long-Context-Agent🤖 Grid-Factorizer🤖 Evidence-Mapper🌐 Technology Innovation Institute (TII)🌐 Hugging Face
#Periscope#Context Window Extension#Grid Factorization#Long Context#Inference Optimization#KV Cache