• 8 Posts
  • 163 Comments
Joined 6 months ago
cake
Cake day: May 11th, 2024

help-circle
  • I think it’s more likely a compound sigmoid (don’t Google that). LLMs are composed of distinct technologies working together. As we’ve reached the inflection point of the scaling for one, we’ve pivoted implementations to get back on track. Notably, context windows are no longer an issue. But the most recent pivot came just this week, allowing for a huge jump in performance. There are more promising stepping stones coming into view. Is the exponential curve just a series of sigmoids stacked too close together? In any case, the article’s correct - just adding more compute to the same exact implementation hasn’t enabled scaling exponentially.


















  • The “caravan” of scary South American migrants that Trump fear-mongered about back in '15 (Jesus, has it been a decade?), they were largely subsistence farmers that were forced off of their lands by multi-year droughts that were demonstrably an effect of climate change. In just the last few years, it’s almost comical how dramatically present the effects of climate change are. Whether or not we’re able to admit it to ourselves consciously, I think everyone feels it. We’re about to hit a dozen different asymptotes, and we’re clearly not ready. So, turns out, fear-mongering is a particularly successful strategy in this kind of zeitgeist.