How do LLMs work?
-
Large language models can write essays, solve math problems, and generate computer code, but it’s not fully understood how they do it. Researchers can observe the billions of parameters inside these systems changing during training, yet the internal logic of the models remains largely hidden. In a sense, the engineering is ahead of the science. Can science catch up and make LLMs and other deep neural networks mechanistically interpretable?
https://cacm.acm.org/news/can-we-understand-how-large-language-models-reason/
-
Not sure there's a "science" behind this.
There's mathematics, sure.
We know the exact mathematical operations performed to train the LLMs and to make use of the LLMs (perform inferences).
But I hesitate to say there is "science" behind it because the mathematical operations performed do not attempt to simulate or approximate any physical system.
It's like you execute a series of mathematical operations and plot the output of each step onto a canvas using some coordinate system, then you step back and notice some "interesting patterns." Doesn't mean there is a "science" behind your series of mathematical operations, the output just happen to resemble something you deem interesting when mapped using a certain coordinate system.
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login