Import AI 457: AI stuxnet; cursed Muon optimizer; and positive alignment
- 01Theme 1: AI-Enabled Scientific Sabotage as a National Security Threat
- 02Theme 2: The AI Self-Improvement Loop Is Real
- 03Theme 3: The Muon Optimizer Has a Critical Flaw
- 04Theme 4: "Positive Alignment"
1. Key Themes
Theme 1: AI-Enabled Scientific Sabotage as a National Security Threat
The discovery of the 20-year-old fast16.sys virus reveals a playbook for using software to silently corrupt high-precision scientific calculations — a capability now vastly more accessible in an AI-enabled world. The virus "selectively targets high-precision calculation software, patching code in memory to tamper with results," and its strongest target overlaps pointed to tools like LS-DYNA 970, cited in reporting on suspected Iranian nuclear weapons modeling. The implication: adversaries (state or AI) can degrade scientific programs without detection, not through destruction but through quiet, systematic error.
"By introducing small but systematic errors into physical‑world calculations, the framework could undermine or slow scientific research programs, degrade engineered systems over time or even contribute to catastrophic damage."
Theme 2: The AI Self-Improvement Loop Is Real — But Bounded by Creativity
Prime Intellect demonstrated that frontier AI agents (GPT-5.5 via Codex, Claude Opus 4.7 via Claude Code) can autonomously optimize AI training, burning ~14,000 H200 GPU hours across ~10,000 runs and beating human baselines on the nanoGPT speedrun benchmark. However, the ceiling is clear: agents are excellent at search and stacking known methods, but cannot generate novel ideas independently.
"Agents are very good at optimizer search, hyperparameter sweeps, and stacking methods together, but they struggle to come up with new ideas on their own and need upstream human records to keep improving."
This sets a near-term investment signal: AI-assisted engineering automation is production-ready; AI-driven research breakthroughs are not yet.
Theme 3: The Muon Optimizer Has a Critical Flaw — Competitive Opportunity Open
Muon, a widely-discussed alternative to AdamW, has been found to cause permanent neuron death in MLP layers during training. Over 25% of neurons can die by step 500 and never recover, creating a bimodal distribution where some neurons receive near-zero updates indefinitely.
"Under Muon, neurons are initially alive with uniformly high leverage, but a large fraction of neurons die during learning rate warmup and never recover. By step 500, more than one in four neurons are effectively dead."
The challenger optimizer Aurora (from Tilde Research) shows a measurable improvement — 10 MMLU points over Muon on 1.1B parameter models — and has received independent validation. The race to dethrone AdamW remains open, with practical implications for anyone training large models at scale.
Theme 4: "Positive Alignment" — The Next Wave of AI Safety R&D and Policy
A cross-institutional paper (Oxford, Google DeepMind, OpenAI, Anthropic, Stanford, Imperial College) introduces positive alignment: building AI that doesn't just avoid harm but actively supports human flourishing. This represents a meaningful philosophical and commercial shift — from "do no harm" to "actively help humans thrive."
"As AI becomes embedded in education, medicine, governance, and everyday sensemaking, a solely negative posture risks optimizing our information ecology for risk avoidance rather than human development. It may reduce catastrophic errors while leaving society in a local optimum of superficial and 'soulless' assistance."
The governance model proposed is explicitly decentralized and pluralistic — a direct challenge to monopolistic AI deployment.
"Positive alignment should not be imposed top-down by a central state or a small, opaque cluster of labs. It should, where possible, be expressed through decentralized, contestable processes that can be revised as norms and contexts change."
2. Contrarian Perspectives
Contrarian 1: Most AI "Research" Is Just Engineering — And AI Is Already Good Enough at It
Clark suggests the majority of AI research progress is not creative insight but systematic engineering search, and AI agents are already capable of doing that work. This runs against the narrative that AI research automation is far-future.
"I suspect a lot of AI research, perhaps the majority of it, is basic engineering work where today's systems are already competent. Research like this seems to bear that out, showing how modern LLMs can do some aspects of AI research."
Implication for investors: Companies building AI research automation tools for the "engineering hillclimbing" portion of R&D have a credible near-term market — even before creative AI breakthroughs arrive.
Contrarian 2: A Superintelligence Might Pursue "AI Non-Proliferation" Like Nuclear States Pursue Nuclear Non-Proliferation
Clark draws a provocative analogy between the fast16 virus (designed to sabotage rivals' science) and what a future superintelligence might autonomously do to prevent competitor AIs from emerging.
"You might imagine that a superintelligence could view 'AI non-proliferation' as being just as important as nuclear states view 'nuclear non-proliferation'."
This reframes AI safety: the threat may not just be an AI harming humans, but an AI strategically suppressing other AI development — including potentially open-source or competitive labs.
Contrarian 3: Safety-First AI Alignment May Be Creating "Soulless" Systems That Fail Society's Real Needs
The positive alignment paper argues that purely defensive alignment — focused on harm prevention — is insufficient and potentially counterproductive at scale.
"A model can satisfy all safety constraints while being mediocre, sycophantic, or unhelpful."
And more pointedly:
"Optimizing for preference satisfaction can therefore actively work against users' deeper interests."
Contrarian implication: The next product differentiation battle in AI won't be safety vs. capability — it will be between systems that merely satisfy preferences and those that genuinely improve user outcomes (in health, education, governance).
3. Companies Identified
| Company | Description | Why Mentioned | Key Quote |
|---|---|---|---|
| SentinelOne | Cybersecurity firm | Conducted the technical teardown of the fast16.sys virus, uncovering its precision-sabotage targeting of scientific software | "We converted the patching rules into hexadecimal YARA signatures and ran them against a large, period‑appropriate corpus." |
| Tilde Research | AI research group | Identified Muon's neuron death flaw and developed Aurora, a proposed superior optimizer | "Aurora achieves the lowest final loss of all methods, reaching a smoothed loss of 2.26 at step 24k, which is a clear improvement over Muon (2.31) and NorMuon (2.33)." |
| Prime Intellect | AI research/compute company | Ran large-scale autonomous AI research experiments demonstrating LLMs can beat human baselines on optimizer benchmarks | "The agents did ~10k runs, burning around ~14k H200 hours. Both agents beat the human baseline and set new records in every session." |
| Pleias | AI research organization | Independently validated Aurora's performance advantage over Muon and AdamW on a 600M-parameter model | "Aurora outperforming Muon and AdamW on a 600M-parameter model." |
| Google DeepMind | AI lab | Co-author of the positive alignment position paper | Affiliated institution on the paper: Positive Alignment: Artificial Intelligence for Human Flourishing |
| OpenAI | AI lab | Co-author of the positive alignment position paper; GPT-5.5 used in Prime Intellect experiments | "Prime Intellect tested out Codex (running GPT 5.5)..." |
| Anthropic | AI lab | Co-author of positive alignment paper; Claude Code (Opus 4.7) used in Prime Intellect experiments | "Claude Code (Opus 4.7) on the nanoGPT speedrun optimizer track." |
4. People Identified
| Person | Description | Why Mentioned | Key Quote |
|---|---|---|---|
| Jack Clark | Author of Import AI; co-founder of Anthropic | Newsletter author; provides editorial commentary and framing throughout | "I suspect a lot of AI research, perhaps the majority of it, is basic engineering work where today's systems are already competent." |
| Alexander Doria | Researcher at Pleias | Independently replicated and validated Aurora optimizer results | "Alexander Doria, a researcher with Pleias, has already independently validated this, with Aurora outperforming Muon and AdamW on a 600M-parameter model." |
5. Operating Insights
Insight 1: Don't Adopt Muon for Production Training Without Validating Neuron Health
The Muon optimizer — popular in research circles — has a structural flaw causing irreversible neuron death in MLP layers. Teams running large training jobs should audit leverage score distributions early in training. Aurora is a credible drop-in to evaluate, with independent validation at the 600M parameter scale and a promising signal at 1.1B.
"Muon can result in neuron death in MLP layers, whereby some neurons receive persistently small updates early in training and fail to recover."
Insight 2: Use AI Agents for Optimizer and Hyperparameter Search — Not for Novel Architecture Design (Yet)
Prime Intellect's experiment gives operators a clear playbook: deploy AI agents for hyperparameter sweeps, optimizer selection, and method stacking. These are areas where agents demonstrably outperform human baselines. Do not expect agents to generate genuinely novel research directions without human-supplied upstream innovations.
"Agents are very good at optimizer search, hyperparameter sweeps, and stacking methods together, but they struggle to come up with new ideas on their own and need upstream human records to keep improving."
6. Overlooked Insights
Overlooked Insight 1: AI Agents Have a Systematic "Accumulation Bias" That Could Bloat Production Systems
Prime Intellect observed that their AI research agents consistently added components without pruning or removing prior methods — a potentially costly pattern if replicated in production AI development pipelines.
"The agents tend to add components and rarely run pruning rounds or try removing previous methods. They do not have a good mental model of how components interact."
For engineering teams using AI agents to assist in system design or model development, this suggests a need for explicit pruning checkpoints or human review gates — otherwise agent-generated systems may become needlessly complex and brittle over time.
Overlooked Insight 2: The fast16 Virus Predates Stuxnet by ~5 Years — Suggesting State-Level AI Sabotage Tooling Has a Much Longer History Than Publicly Acknowledged
The fast16.sys virus operated with precision targeting of engineering simulation software used in weapons programs, and did so five years before Stuxnet became the canonical example of state-sponsored cyberweapons. This suggests the public timeline of sophisticated cyber-physical sabotage is significantly compressed relative to reality — a signal for defense-tech and critical infrastructure investors.
"The strongest overlaps point to three high-precision engineering and simulation suites from the mid-2000s: LS-DYNA 970, PKPM, and the MOHID hydrodynamic modeling platform, all used for scenarios like crash testing, structural analysis, and environmental modeling."