Researchers from the Computational Linguistics Laboratory (CoLingLab) at the University of Pisa published an empirical breakthrough on mechanistic interpretability for LLMs, open-sourcing their findings and codebase on GitHub (arXiv:2609.29362). Using Part-of-Speech (PoS) morpho-syntactic categories as a controlled testbed, the study investigates whether grammatical structure in Sparse Autoencoders (SAEs) resides in isolated monosemantic latents or distributed feature geometries. Findings reveal that while PoS categories are readily recoverable from SAE activations and cannot be reduced to lexical memorization, they do not map 1:1 to single latents. Instead, categories are localized in compact, stable clusters of sparse latents with distinct behavior between open and closed classes.
- ✓Refuting 1:1 monosemanticity: proves morpho-syntactic categories do not map to isolated single latents in Sparse Autoencoders
- ✓Compact latent clusters: categories are collectively represented by compact, structured groups of sparse latents with clear open/closed class divergence
- ✓Beyond lexical memorization: controlled counter-experiments confirm high recovery rates stem from true structural abstraction rather than word memorization
- ✓Out-of-distribution stability: identified PoS feature clusters remain robustly predictive across held-out corpora with interpretable category overlaps
- ✓Open-source evaluation suite: full extraction pipelines, linear probing tools, and latent visualization scripts published on GitHub
🔗
Project Links & Resources
Direct AccessDirect access to official project resources and documentation🔬
In-Depth Technical Analysis
Core Background & Industry Pain Points Sparse Autoencoders (SAEs) represent a cornerstone of mechanistic interpretability, widely embraced by Anthropic, OpenAI, and DeepSeek to untangle polysemantic representations. A widespread assumption posited that SAE latents map 1:1 onto monosemantic linguistic primitives. However, empirical grounding has been lacking regarding whether foundational morpho-syntactic structures, such as Parts-of-Speech (PoS), decompose into individual atomic latents or structured distributed manifolds. ### Architecture Highlights & Internals CoLingLab at the University of Pisa established a controlled probing framework across open-source LLMs and SAEs: 1. Morpho-Syntactic Benchmark: Standardized on Universal Dependencies tags, evaluating structural differences between Open Classes (nouns, verbs) and Closed Classes (pronouns, prepositions); 2. De-memorization Probing: Controlled word-substitution tests confirm that PoS classifications reflect genuine abstract syntactic geometry rather than memorization of high-frequency vocabulary; 3. Compact Latent Group Topology: Demonstrates that syntactic categories are governed by compact, structured groups of sparse latents with shared subspaces across related linguistic tags. ### Authoritative Benchmarks & Measured Scores - Classification Recoverability: Linear probes on SAE sparse activations reach a Macro F1 score of 94.8%, outperforming dense residual stream baselines with less than 1.2% degradation on held-out test data; - Open vs Closed Divergence: Closed classes condense into 8-15 latents, whereas open classes partition into clusters of 45-80 latents; - Single-Latent Ablation: Ablating individual high-activating latents shifts classification accuracy by under 0.4%, demonstrating robust distributed resilience. ### Developer Hands-on Guide Clone the repository colinglab/pos-sae-latents and execute run_pos_probing.py across SAELens / TransformerLens models. Preprint: arXiv:2609.29362.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.