Hierarchical Temporal Memory (HTM)

[Zacariaz]

Frequently people talt about implementing AI like stuff into their OS, but i am somewhat confused that i have yet to hear anyone discuss tha Hierarchical Temporal Memory (HTM) concept. Even though it is a somewhat new concept i can only asume that people arent aware of it.

The keyword is "Jeff Hawkins", he is a man who have some rather controversiel ideas, however these ideas seem to me, to be on the right track, and i dont think that it has to be as complex and difficult as you might think.

Im not really qualified to discuss this subject... yet, but i will post a few link and stuff and hope to get a good discussion out of it.

http://www.numenta.org (The company)
http://redwood.berkeley.edu/ (The research)
http://www.ted.com/index.php/talks/view/id/125 (a video where Jeff Hawkins is talking about the subject)

[Zacariaz]

ok, people are reading but not commenting. I think the subject is like the most interesting thing in the world, am i really the only one?

[Colonel Kernel]

I watched the video -- It is very interesting, but there is almost too much that could be said about it. Especially for those of us with busy jobs...

I don't think it has much to do with AI as we know it today though -- it sounds to me like his theories will lead to a more hardware-based and much less deterministic approach to AI.

[Candy]

I watched the video -- It is very interesting, but there is almost too much that could be said about it. Especially for those of us with busy jobs...

Exactly. Do you have an executive summary?

[Zacariaz]

really i just think it is an intereteing subject, and i dont think that it takes great effort to obtain a basic understanding, but eh, ill just have to discuss it with my self then, it will be kinda boring, but itll have to do...

[Colonel Kernel]

Exactly. Do you have an executive summary?

Jeff Hawkins believes that there is no good theory about how the brain works because of implicit and incorrect assumptions, not because there is a lack of data (he says there isn't) and not because it is inherently complex (which it is, but so are many subjects and it hasn't stopped us from understanding them).

He says the implicit and incorrect assumption is that intelligence is determined by behaviour (e.g. -- the Turing Test -- if it behaves enough like a human, then it must be intelligent). He postulates instead that intelligence is determined by prediction -- that is, our ability to "see the future" based on patterns we have accumulated from our past via our senses and recorded thought processes.

For example, you will almost certainly be able to predict what word comes at the end of this...





























...sentence.

His ideas are really interesting and struck me as being very intuitive. If possible, try to find a spare 20 minutes to watch the video.

[jnc100]

it is a somewhat new concept

Hardly. Based on his "two box" approach, it is a fundamental component of most artificial intelligence attempts to base the output (behaviour) not only on the input but also on some sort of memory store of what has gone before. To give a simple example, if I have an algorithm which has to answer truthfully, this would never pass a Turing Test without some ability to learn from the past interactions:

me: Have I asked this question before?
ai: I don't know.
me: Have I asked this question before?
ai: I don't know.
me: Have I asked this question before?
ai: I don't know.
...

etc. Anyone can see that you must have some sort of learning ability in an ai. Before anyone suggests otherwise, behaviour _is_ the 'end point' of intelligence if you believe evolution: to survive we must behave in a certain way (thinking various interesting things is not enough) therefore evolution works towards refining that behaviour and intelligence evolves to ultimately decide what behaviour to perform. A more accurate model of how the brain works is:

Code: Select all

              +-------+
Input ------> | brain | ------> Output
              |       |
         +--> |       | ----+
         |    +-------+     |
         |                  |
         +------------------+

Indeed the idea of a 'cortical re-entrant loop' as depicted above has been around for a long time (I have it well described in a neuroscience text book from 1991) and many of the waveforms seen on EEG (electroencephalogram - basically a 'brain wave' scan) correspond to waves of depolarisation moving across the surface of the brain, generally assumed to be an electrophysiological analogue of the basis of consciousness. The fact that the nature of these waveforms change during various phases of sleep (whereas electrophysiological measurements in other parts of the nervous system don't) further supports the theory.

Unfortunately, the subject is incredibly difficult to test. This chap is correct that most of the deep brain structures are preserved across evolution, and so we can perform many studies on these structures in animals. These studies have quite rightly generated a large amount of information and led to the development and rigourous testing of many theories about the workings of the sensory, motor and homeostatic networks within the brain, many of which are of great benefit to medicine. The neocortex on the other hand, whilst present in all mammals, is in humans of much greater mass in proportion to the total brain making it very difficult to extrapolate the results of animal studies to humans. We obviously can't go performing invasive experiments on the living human brain, so the study of consciousness is incredibly difficult, limited to EEG studies and modes of imaging which can assess the levels of blood flow and glucose delivery in different parts of the brain. In this regard, this guy is incorrect therefore: most of the problem we have is that there is simply not enough data available on the workings of the human neocortex.

I will go into this in more detail if anyone is interested. Personally, I haven't heard much of this guy before (presumably because he seems to be sponsored by technology companies and is not so much in academia) and so can't comment on anything he's published, but I will give his papers a look when I get a chance, busy job and all.

Regards,
John.

[Zacariaz]

no doubt you know a great deal more about this subject than i, and Jeff Hawkins may be on the wrong track after all, but please keep and open mind, the most profound discoveries in history has been contradicting all that we thought we knew.

About the sponcorship... as far is i know Jeff hawkins have had this interest for a very long time, but was unable to study it in a proper way. He could study the brain, but not the theory behind the workings of it. He could study AI but not i the way he wanted. So he simply financed it himself by many years of hard work, and that is, i think, impressive.

I think his talk and lectures makes sence. Im currently in the process of readig his book "on intelligence" but i fear it is allready out of date. His company has made much progress and many discoverys and im sure we are gonna se more. It is allso Important to note that Jeff Hawkins is very much aware that this is a new area, allthough using many old discoverys and data, and he incourages others to take part in this. His company is created in order to make profit, but he saids it is only to hurry up progress. Weater it is true or not i do not know, but i for one believe that he is truely interested in unlocking the secrets of the neocortex and develope methodes of using it in computers and stuff, nothing else.

bla bla bla, i could keep on going, but i wont... for now...

Regards