Inception

“If you think about inference, now not training, inference generation, autoregressive models are still sequential. The computation is one left to right, one token at a time... That kind of workload does not map well to GPUs. That kind of workload is extremely memory bound.”

“There was an inflection point in 2017 when people switched from RNNs to transformers... But if you think about inference... the equivalent at inference time is a diffusion based model.”

“Our Mercury models are on par with the Haiku models, Flash models, Mini Nano models from OpenAI, if you look at benchmarks, while being significantly faster.”

“The gains that you get from the software, they are multiplicative with the gains you get from the hardware.”

“They were previously using, serving their LLMs on Cerebras... And then they switched over to our diffusion-based LLMs because they can essentially get the same speed as what you would get if you were to run an autoregressive model on custom hardware.”

“They switched over to our diffusion-based LLMs because they can essentially get the same speed as what you would get if you were to run an autoregressive model on custom hardware.”