<![CDATA[How do LLMs work?]]>

Large language models can write essays, solve math problems, and generate computer code, but it’s not fully understood how they do it. Researchers can observe the billions of parameters inside these systems changing during training, yet the internal logic of the models remains largely hidden. In a sense, the engineering is ahead of the science. Can science catch up and make LLMs and other deep neural networks mechanistically interpretable?

https://cacm.acm.org/news/can-we-understand-how-large-language-models-reason/

]]>https://wtf.coffee-room.com/topic/3874/how-do-llms-workRSS for NodeFri, 17 Jul 2026 22:01:10 GMTFri, 17 Jul 2026 19:55:50 GMT60<![CDATA[Reply to How do LLMs work? on Fri, 17 Jul 2026 20:06:28 GMT]]>Not sure there's a "science" behind this.
There's mathematics, sure.
We know the exact mathematical operations performed to train the LLMs and to make use of the LLMs (perform inferences).
But I hesitate to say there is "science" behind it because the mathematical operations performed do not attempt to simulate or approximate any physical system.
It's like you execute a series of mathematical operations and plot the output of each step onto a canvas using some coordinate system, then you step back and notice some "interesting patterns." Doesn't mean there is a "science" behind your series of mathematical operations, the output just happen to resemble something you deem interesting when mapped using a certain coordinate system. 🤷

]]>https://wtf.coffee-room.com/post/27148https://wtf.coffee-room.com/post/27148Fri, 17 Jul 2026 20:06:28 GMT