TechLich@lemmy.worldtoFuturology@futurology.today•The AI Bubble may be about to burst. LLMs have reached the point of diminishing returns, and there's no sign of scaling leading to independent reasoning, needed for the first steps to AGI.English
3·
5 days agoNot entirely true, the big change was multi-headed attention and the transformer model.
It’s not just being used for language but anything where sequence and context patterns are really important. Some stuff is still using convolutional networks and RNNs etc. but transformers aren’t just for LLMs. There’s definitely a lot of algorithmic advances driving the wave of new ai implementations, not just hardware improvements.
Agreed. It’s a lot of the same tech that powers both, but it’s not like a self driving car contains a language model that’s fine tuned on the adventures of Steve MacQueen or something.