It’s size makes it basically useless. It underperforms models even in it’s active weight class. It’s nice that it’s available but Grok-0 would have been far more interesting.
Who cares if the code is open source, or pre-training weights are released? Virtually every Masters in CS student in 2024 is building this from scratch. The differentiator is the training dataset, or at worst, the weights after fine tuning the model.
Training something like this costs millions. It’s not going to be useful to many people because of it’s size/the cost of doing anything with it.
I don’t think that giving alms justified the existence of feudal Lords. In the same way, I think the fact that nation states can’t keep up with the research expenditure of a few rich men (think space travel), shows us that we have a problem. That said, it does represent a fairly generous donation to science, start-ups, or whatever.
Who really wants an LLM trained on Musk’s Great Replacement blog?
