Beam

“Laskin: 'We trained the final model on, you know, the SpaceX cluster that we received in, in July.'”

“To train Beam, which is a 500 billion parameter model total 23B active, it was 6,000 GB300s. We ran it, I think, for a few weeks... Reinforcement learning was a little over 10,000 GB300s for four weeks.”

“I've not seen a 10,000 GB300 for four weeks run documented yet.”

“We trained the final model on, you know, the SpaceX cluster that we received in, in July.”