Why do LLMs have emergent properties?
Large language models display emergence behaviors: when the parameter count is scaled to a certain value, suddenly the LLM is capable of performing a new task not possible at a smaller size. Some say the abruptness of this change is merely a spurious artifact of how it is measured. Even so, many would like to understand, predict, and even facilitate the emergence of these capabilities.
The following is not a mathematical proof , but a plausibility argument as to why such behavior should not be s...
Read more at johndcook.com