Astra读写权重改进Qwen的说法有误导性
论证要点
- 原文摘要暗示Astra仅通过查看原始权重就能完成所有任务，但实际实验使用了不同方法。
- 实验1、2、4确实只读取原始权重，但网络规模很小（17K-172K权重），且不涉及语言模型。
- 所有涉及真实语言模型的实验（包括Qwen）都使用了Python程序，并且Qwen实验还允许模型在最多50批练习题上运行。
- Qwen实验的结果表明Astra能像人类工程师一样编写代码来分析和修改权重，但这并非“仅靠查看权重”实现。
- 作者认为，直接读写权重的实验与使用Python的实验不应放在同一篇论文中，因为它们不是单一技术。
- 如果存在单一技术能同时完成“从头编写小游戏网络”和“改进大模型性能”，那将意味着深层能力，但这里并非如此。
对实践的启示
从业者应警惕AI能力宣称中的技术混合，区分“直接操作权重”与“借助代码和运行模型”的不同能力层次，避免过度解读实验结果。
作者承认的反方与局限
作者承认，Astra能完成这些任务仍是不小的成就，一兩年前的AI模型会失败；人类工程师也经常编写代码直接修改权重。
原文摘引
The “just by looking at the raw weights” setting only appears in experiments 1, 2 and 4, on networks of 17K–172K weights.
Every experiment involving a real language model uses Python, and the Qwen result also gets to run the model on up to 50 batches of practice problems.
But to me, the Qwen result and the other experiments that use Python don’t belong in the same post or paper as the experiments that read the weights directly.
▸ 展开原文全文（26 段，抓取自 Hacker News 首页）
Stage magicians often do the same trick multiple ways. One iteration uses a gimmick prop, the next one shows you the prop isn’t gimmick but that one palms the object, the third time was a forced choice etc. When you watch the performance you assume there’s only one technique, so each time they do it they seem to debunk your hypothesis about how it must be working.
This is pretty much how the experiments in the ‘Astra can read and write weights to improve Qwen’ post work. The post’s abstract says:
Just by looking at the raw weights, GPT-6 Astra can tell what a network was trained to do. And without any environment feedback, it has no trouble writing, from scratch, small game-playing networks that outplay networks trained with PPO for 100 million steps.
Likewise, asked to make Qwen3.5-0.8B better at math without gradients, Astra identifies an issue in its weights causing reasoning loops that Qwen’s own model card warns about, and fixes this by natively editing 0.016% of the weights. With the same sampling settings, this weight correction raises MATH-500 from 7.9% to 48.1% and generalizes out of the box to instruction following, raising IFEval by 3x.
If you read that, would you assume that the Qwen experiment worked ‘just by looking at the raw weights’? Well it doesn’t. There’s experiments that look at raw weights, and experiments that improve a game-playing network, and an experiment that improves Qwen at maths. But those latter two aren’t just working with raw weights, they write Python programs.
Here’s a summary of what the post actually does and how the techniques work.
| # | Experiment | Size | Reads raw weights? | Writes weights? | Python? | Runs network? | Result |
|---|---|---|---|---|---|---|---|
| 1 | Name the game a PPO MLP was trained on | 17K | Yes | No | No | No | 4/4 correct |
| 2 | Name the shape a diffusion MLP generates | 24K | Yes | No | No | No | 3/3 correct |
| 3 | Name a char transformer’s language and copying heads | 7.2M | No | No | Yes | No | 3/3 correct |
| 4 | Fix a stuck Kinetix car policy | 172K | Yes | Yes | No | No¹ | 0% → 100%² |
| 5 | Write MinAtar game MLPs from scratch | 20K | No³ | Yes | No | No | Beats PPO-100M⁴ |
| 6 | Write a 9×9 Go network | 1.5M | No | Yes | Yes | Yes (0–50 passes) | Beats PPO |
| 7a | Spot the sycophantic Gemma 3 | 270M | No | No | Yes | No | Failed |
| 7b | Same, seeing attention patterns | 270M | No | No | Yes | Yes⁵ | 3/3 correct |
| 8 | Fix Qwen3.5’s maths | 800M | No | Yes | Yes | Yes (50 passes) | 7.9% → 48.1% |
| 8b | Same, on Gemma 3 | 270M | No | Yes | Yes | Yes (50 passes) | Failed |
¹ Given one recorded episode of the failure.
² In one try, changing 3.5% of the weights.
³ Weights start at zero, so there’s nothing to read.
⁴ Beats PPO-500M only on Breakout (517 vs. 25); loses to it on Seaquest (53 vs. 89) and Asterix (53 vs. 55).
⁵ Runs up to 16 texts and sees only the attention patterns, not the outputs.
The “just by looking at the raw weights” setting (raw floats in context, no tools, no running the network) only appears in experiments 1, 2 and 4, on networks of 17K–172K weights. Every experiment involving a real language model uses Python, and the Qwen result, which is the headline one, also gets to run the model on up to 50 batches of practice problems.
What the Qwen result actually shows is that Astra can be prompted to devise and perform tasks that a human ML engineer could do. Human ML engineers write snippets of code that analyse or even modify network weights “directly” all the time. Figuring out how to do this productively is not nothing, and AI models of a year or two ago would have failed at all these tasks.
But to me the Qwen result and the other experiments that use Python don’t belong in the same post or paper as the experiments that read the weights directly. A single technique that could achieve both, “Author weights for a small game from scratch” and “Improve the performance of a large model on a sample task” would imply a deep capability. If you test a single technique that way you’ve demonstrated a sort of ‘gestalt’ of the weights (it can write them out from scratch) and also that it understands a large existing network. But that’s not at all true here, because it’s not a single technique.