Tilde Research

“The challenger optimizer Aurora (from Tilde Research) shows a measurable improvement — 10 MMLU points over Muon on 1.1B parameter models — and has received independent validation.”

“Under Muon, neurons are initially alive with uniformly high leverage, but a large fraction of neurons die during learning rate warmup and never recover. By step 500, more than one in four neurons are effectively dead.”

“Alexander Doria, a researcher with Pleias, has already independently validated this, with Aurora outperforming Muon and AdamW on a 600M-parameter model.”