Sun Yutao
Yutao Sun is a PhD candidate at Tsinghua University, advised by Jianyong Wang, and a Pre-Career Scholar at the Shanghai Innovation Institute. He has been a research intern at Microsoft Research Asia since July 2023, mentored by Li Dong. He is best known for co-developing Retentive Network (RetNet), You Only Cache Once (YOCO), and Differential Transformer, with research focusing on efficient attention mechanisms, long-context modeling, and multimodal world models for large language models.
“Using sparse attention naively on every layer has very little benefit on Blackwell... in decode, sparse attention may not even provide significant speedup over full attention.”
Source→“Yoco implemented an architecture where all layers share a single KV cache.”
Source→AI-extracted from podcast / newsletter / paper summaries. May contain errors.