Content-type: text/html Downes.ca ~ Stephen's Web ~ How to represent part-whole hierarchies in a neural network

Stephen Downes

Knowledge, Learning, Community

There's a pretty good article about this in MIT Technology Review, but it's behind a paywall so I can't link to it (I have a full text version in my RSS feed reader; also see the comments here). The concept Hinton describes is called GLOM (derived from the slang "glom together"). The idea is simple but the tech is complex. Here's the idea: "Similarities of big vectors explain how neural networks do intuitive analogical reasoning." A vector is an array of numbers that encodes information, for example, the xyz coordinates of a point. Any given perception can be represented as a really long vector - or as sets of multiple (and multidimensional) vectors. These subsets are similar to previously experienced vectors, allowing the neural net to extract parts from the whole. These are "islands of agreement". Why do I think this is a good idea? Because I had the same intuition in 1993 (note that I am in no way claiming to have discovered this; it's a very different thing to have an intuition and to flesh it out as a fully formed idea).

Today: 0 Total: 13 [Direct link] [Share]


Stephen Downes Stephen Downes, Casselman, Canada
stephen@downes.ca

Copyright 2024
Last Updated: Dec 22, 2024 03:13 a.m.

Canadian Flag Creative Commons License.

Force:yes