A naive conjecture about computer architecture and artificial intelligence
|
Prev: Lolling at programmers, how many ways are there to create a bitmask ? ;) :)
Next: A naive conjecture about computer architecture and artificialintelligence
From: Robert Myers on 10 Jul 2010 12:14 On Jul 10, 4:39 am, n...(a)cam.ac.uk wrote: > In article <dwRZn.6659$KT3.5...(a)newsfe13.iad>, > Robert Myers <rbmyers...(a)gmail.com> wrote: > > > > > > > > >> Yes. There is a lot of good mathematical work on this, in the area > >> of real (i.e. Kolmogorov) complexity theory. Most of it can be > >> summarised for practical use as "if you have a genuinely random > >> finite state machine, you are up the creek without a paddle, so > >> consider what empirical, semi-mathematical heuristics you can use." > > >The theory and practice of identifying patterns in semi-random time > >series must be very well developed, if only because of those > >front-running supercomputers humming away on Wall Street. Is there a > >related subfield (Andy apparently has developed one for his own use) > >where the available resources are extremely constrained? How much of > >this is genuinely new ground and how much might already exist in some > >other context? > > >The empirical, semi-mathematical heuristics must have themselves come > >from a learning predictor somewhere, even if it is the brain of a > >computer architect. > > No, no, not at all. Firstly, the whole point of such things IS that > there is no theory - if anything works, fine, and trial-and-error is > the only way to find out. And, worse, the very approaches to try > are selected by guesswork. We humans are stunningly good at coming up with inspired guesses, and the more important part of my conjecture here is that the way we learn to do that isn't very different from the most humble learning predictor. That is to say, we have a lot to learn. Somewhere, I read about the difference between pursuing a hunch with the reassurance that what you want to do works well in some context (a promising bet) and pursuing a hunch where what you want to do is just something you want to do (a far less promising bet, in most cases). > Secondly, all that those people who gamble with our money need is a > small margin (say, 5%) - and that would be of no use in this context. Since I know nothing about what the time series gnomes of Wall Street are doing, except in the most painfully general terms, I have no way of responding, except to say that using some single metric of what they need from their work seems like a premature dismissal to me. Since I have only ever worked with stationary random processes (perhaps plus a known signature that might or might not be present), the mere fact that the Wall Street quants are working with a non- stationary random process puts them way ahead of me. Robert.
From: nmm1 on 10 Jul 2010 13:12 In article <da140fe0-b8cf-4406-8df5-70b1f21b7ad8(a)y11g2000yqm.googlegroups.com>, Robert Myers <rbmyersusa(a)gmail.com> wrote: > >We humans are stunningly good at coming up with inspired guesses, and >the more important part of my conjecture here is that the way we learn >to do that isn't very different from the most humble learning >predictor. That is to say, we have a lot to learn. Not quite. We don't actually know how we guess or how we learn, but we have some evidence that those are NOT the same as the computer programs written to do the same! Neural networks aren't. >Somewhere, I read about the difference between pursuing a hunch with >the reassurance that what you want to do works well in some context (a >promising bet) and pursuing a hunch where what you want to do is just >something you want to do (a far less promising bet, in most cases). It's a good rule, and corresponds to my experience. >> Secondly, all that those people who gamble with our money need is a >> small margin (say, 5%) - and that would be of no use in this context. > >Since I know nothing about what the time series gnomes of Wall Street >are doing, except in the most painfully general terms, I have no way >of responding, except to say that using some single metric of what >they need from their work seems like a premature dismissal to me. I think that you have misunderstood. I am not talking about any particular metric, but about the margin (above pure 'randomness') that they need to achieve 'success'. I don't know much more than you, but have been briefed about what they are doing by some of them who were thinking about collaborating. And that statement is a paraphrase of what they said. The same is NOT true of branch and preload prediction. If you can reduce the misprediction by only 5% (relatively), it isn't exciting enough to change to an experimental design. >Since I have only ever worked with stationary random processes >(perhaps plus a known signature that might or might not be present), >the mere fact that the Wall Street quants are working with a non- >stationary random process puts them way ahead of me. Branch and preload prediction aren't stationary, either. I have dabbled a bit with such things, know a LITTLE about the statistics, and my comment about heuristics stands. Regards, Nick Maclaren.
From: Robert Myers on 10 Jul 2010 15:12 nmm1(a)cam.ac.uk wrote: > In article <da140fe0-b8cf-4406-8df5-70b1f21b7ad8(a)y11g2000yqm.googlegroups.com>, > Robert Myers <rbmyersusa(a)gmail.com> wrote: >> We humans are stunningly good at coming up with inspired guesses, and >> the more important part of my conjecture here is that the way we learn >> to do that isn't very different from the most humble learning >> predictor. That is to say, we have a lot to learn. > > Not quite. We don't actually know how we guess or how we learn, > but we have some evidence that those are NOT the same as the computer > programs written to do the same! Neural networks aren't. > Some time back, I had an "aha" moment. It may be a misleading "aha," but for the time being, I'm sticking with it. There was a news story about a very primitive sea creature with too few cells even theoretically to have what we would recognize as a central nervous system. I forget too many details of the story to retell it accurately, but it seemed that the sea creature had "learned" to predict the location of nutrients apparently based on nothing more than periodicity (as I recall). I doubt if the creature would have understood the Wiener-Khinchin theorem, but it apparently knew all it needed to know about autocorrelations. If I think about evolution, the idea that the seeds of "intelligence" were there practically from the very beginning seems very compelling to me, and since the available resources were quite limited, it can't have been anything very sophisticated. I haven't been keeping track of the details of the ongoing failures of artificial intelligence, but we are in complete agreement that, whoever they are, they haven't got it yet. I prefer taking my cues from experience (the sea creature, how I experience learning the piano, and the success of primitive predictors) to the continuing failures of a floundering enterprise. <snip> >> Since I know nothing about what the time series gnomes of Wall Street >> are doing, except in the most painfully general terms, I have no way >> of responding, except to say that using some single metric of what >> they need from their work seems like a premature dismissal to me. > > I think that you have misunderstood. I am not talking about any > particular metric, but about the margin (above pure 'randomness') > that they need to achieve 'success'. I don't know much more than > you, but have been briefed about what they are doing by some of > them who were thinking about collaborating. And that statement is > a paraphrase of what they said. > > The same is NOT true of branch and preload prediction. If you can > reduce the misprediction by only 5% (relatively), it isn't exciting > enough to change to an experimental design. > To quote the learned Andrew Glew, from this same thread in this same august forum: <quote> I warned you: at UWisc, I wanted to work on big instruction windows. But Jim Smith said "big instruction windows cannot be justified, because branch predictors are not good enough". Now, I knew something that Jim didn't: I knew that Willamette's branch predictors were significantly better than any than had been published in academia (I don't know the exact numbers off my head, but: if you go from 94& to 97%, you have doubled the instructions between mispredicted branches on average. Often more so. Also, I knew that main memory latencies of 300 cycles were coming. </quote> I did that not to prove you wrong, Nick, but to show Andy that I am paying attention. ;-) Robert.
From: jacko on 10 Jul 2010 16:22 > To quote the learned Andrew Glew, from this same thread in this same > august forum: > > <quote> > > I warned you: at UWisc, I wanted to work on big instruction windows. > But Jim Smith said "big instruction windows cannot > be justified, because branch predictors are not good enough". Now, I > knew something that Jim didn't: I knew that > Willamette's branch predictors were significantly better than any than > had been published in academia (I don't know the > exact numbers off my head, but: if you go from 94& to 97%, you have > doubled the instructions between mispredicted > branches on average. Often more so. Also, I knew that main memory > latencies of 300 cycles were coming. > > </quote> So with speculative execution in mind, how many bits of branch path are needed to place any speculative instruction within it's right branch prediction group? and why not run the pre-mis-predicted stream too if your going to stall on the other one, just in case. Maybe the whole problem stems from branch uncertainty. :-) maybe this could be helped by a trend register in count up/down situations with backward offset, and a conditional call CCALL with a BUT stall marking the limiting progress without the call taken being known. Cheers Jacko
From: nmm1 on 10 Jul 2010 16:38
In article <oC3_n.6805$OU6.4684(a)newsfe20.iad>, Robert Myers <rbmyersusa(a)gmail.com> wrote: > >I haven't been keeping track of the details of the ongoing failures of >artificial intelligence, but we are in complete agreement that, whoever >they are, they haven't got it yet. I prefer taking my cues from >experience (the sea creature, how I experience learning the piano, and >the success of primitive predictors) to the continuing failures of a >floundering enterprise. Yes, I agree that we have only just realised the potential for emergent behaviour in quite simple systems. It wasn't long ago that most people believed you could say nothing about the behaviour of what are now called chaotic systems, except the very global averages. We now know that is wrong, but haven't really started to understand them. Regards, Nick Maclaren. |