How Do Machines ‘Grok’ Data?
Anil Ananthaswamy,
Quanta Magazine,
Apr 12, 2024
The ultimate objective of teaching and learning is understanding or comprehension. By that we mean something more than memorization. But what, exactly? Usually we'll appeal at this point to something like models, causes or generalizations. Some think it involves "selfhood (memory, beliefs, consistency) and initiative (curiosity, proactivity)." But it feels like it should be something deeper - knowledge, for example, should be something we can't not know, like (say) recognizing our mother at the train station. This article describes a type of learning researchers called 'grokking' after Robert A. Heinlein to mean understanding something "so thoroughly that the observer becomes a part of the process being observed." But what does that mean? Well I'm paraphrasing the article a bit, but it means essentially this: the internal structure of the (neural) network becomes the phenomenon it is attempting to learn. Its understanding of modular arithmetic, for example, is physically arranged into a circle.
Today: 6 Total: 102 [Direct link] [Share]
