Stefano Ermon

“If you think about inference, now not training, inference generation, autoregressive models are still sequential. The computation is one left to right, one token at a time... That kind of workload does not map well to GPUs. That kind of workload is extremely memory bound.”

“There was an inflection point in 2017 when people switched from RNNs to transformers... But if you think about inference... the equivalent at inference time is a diffusion based model.”

“Our Mercury models are on par with the Haiku models, Flash models, Mini Nano models from OpenAI, if you look at benchmarks, while being significantly faster.”

“They were previously using, serving their LLMs on Cerebras... And then they switched over to our diffusion-based LLMs because they can essentially get the same speed as what you would get if you were to run an autoregressive model on custom hardware.”

“Open Router has this very nice way of kind of like looking at all the different use cases... they have like a nice hard taxonomy basically of tasks.”

“All of that started from ideas that were developed in academia in my lab. But that's not the only one.”

“We kind of like came up with this idea of let's train a neural network to denoise images... And that basically became the kind of like underlying technology of modern diffusion models... with my PhD student, Yang Song.”