AI Coding Live News

Two mechanistic interpretability studies uncover how LLMs express and localize internal knowledge: University of Cambridge researchers (arXiv:2610.21025) introduce parameter-trajectory crosscoding on checkpoint weights, discovering that hidden states encode task knowledge before the unembedding readout matrix learns to express it (reorganizing at 0.4-1.1% of Pythia pretraining), and that raising the readout learning rate closes this gap earlier. Meanwhile, University of Copenhagen researchers (arXiv:2610.22127) apply paired-activation crosscoders to three open-weight Mixture-of-Experts (MoE) LLMs, isolating monosemantic personalization latents and proving via causal interventions that boosting specific routed experts recovers personalized model behavior.

Zoom

Two top-trending Hugging Face papers dissect the limits of VLM 3D interactive reasoning and LLM mechanistic interpretability: Xiamen University's RoTA-Bench (arXiv:2610.17665, 200 upvotes) evaluates closed-loop 3D-to-2D projection alignment across 674 tasks, finding that humans achieve 100% success while GPT-5.6-Sol reaches only 17%. Meanwhile, UW-Madison's OpenSAE study (arXiv:2610.21261, 104 upvotes) introduces SAE Additivity Residuals (SAR) across 18 model/behavior settings, showing that modeling pairwise feature interactions boosts multi-feature steering rank fidelity from 0.5874 to 0.7790 and cuts ordering error from 0.1444 to 0.0606.

Zoom

On-policy distillation (OPD) is an essential post-training technique for transferring reasoning capabilities from frontier teacher models to smaller students. However, the precise mechanistic impact of OPD on student internal representations has remained poorly understood. Researchers from the University of Hong Kong (HKU) utilize sparse crosscoders paired with a novel 'swap readout' technique to conduct the first mechanistic interpretability investigation of OPD across training checkpoints. The study reveals a fundamental insight: OPD neither creates novel internal features nor injects teacher-specific features; instead, over 98% of the student's frequent features maintain firing rates within 20%. Rather than teaching new concepts, OPD predominantly reweights features the student already possessed. Furthermore, the authors elucidate the mechanistic necessity of preceding SFT warm-ups on teacher rollouts, showing they pre-align shared features while implanting persistent reasoning templates, establishing a foundational mechanistic theory for LLM distillation.

Zoom

Researchers from the Computational Linguistics Laboratory (CoLingLab) at the University of Pisa published an empirical breakthrough on mechanistic interpretability for LLMs, open-sourcing their findings and codebase on GitHub (arXiv:2609.29362). Using Part-of-Speech (PoS) morpho-syntactic categories as a controlled testbed, the study investigates whether grammatical structure in Sparse Autoencoders (SAEs) resides in isolated monosemantic latents or distributed feature geometries. Findings reveal that while PoS categories are readily recoverable from SAE activations and cannot be reduced to lexical memorization, they do not map 1:1 to single latents. Instead, categories are localized in compact, stable clusters of sparse latents with distinct behavior between open and closed classes.

Zoom

A research group focused on mechanistic interpretability has released 'Your Transformer Can Hold Two Thoughts at Once: Evidence of Linear Superposition in LLMs' (arXiv: 2609.29845). Challenging the premise that non-linear activation layers render transformers hopelessly entangling, the authors formulate the Superposition Linearity Hypothesis: linearly combining input representations from distinct text streams yields an output probability distribution that is an exact linear superposition of the individual next-token predictions. Leveraging this intrinsic property, the team introduces a guided decoding method capable of extracting two distinct, coherent narrative continuations simultaneously from a single forward pass.

Zoom