Article:		An Introduction to Artificial Intelligence
Subject:		Great Intro Article
Date:		2005-10-22 17:45:38
From:		MrO
8:3:8 - an "aha" moment for me, as I've been somewhat mystified by neural nets and couldn't connect it back to the bit world. Thanks very much! I was always fascinated by a little demo that accompanied NeXT boxen, that had a simulated seal learning how to balance a ball on it's nose. The learning was based on a neural net, though I'm now curious how the "success" feedback was represented in the system - perhaps another article in this line? Thx, Mark

Showing messages 1 through 3 of 3.

• Feedback
  2005-10-22 23:22:18  austins [View]
  
  
  Usually neural networks are trained by measuring the error in the ouput later and using an algorithm called backpropagation to adjust connection weights to explicitly correct the error. With enough training and a solvable problem, the network will usually converge. Sometimes, though, the network gets "overtrained" and just stores the desired output, so it doesn't generalize to new problems very easily.
  
  Neural networks are still good for some things, but there are now much better algorithms now for face recognition- the eigenfaces decomposition with support vector machines, for example, does a pretty good job. This is still a hot research area, and amazing things are being done.
• Feedback
  2005-10-23 06:16:53  Matthew Russell | [View]
  
  
  Yep. Backpropagation is the standard technique for adjusting the weights and is used here as well.
  
  Good point about there being newer cutting edge ways of face detection than neural nets -- you're absolutely right about that. I chose neural nets here because of the general applicability to solve other problems in addition to the one at hand, and for the coolness factor as well. Lots of people hear about neural nets, but not too many people ever stop to figure out what they are.
  
  As for the overtraining: right again, and that's easier to do than you'd think. And another consideration I didn't mention in the main text is that unlike a tool like a decision tree, a neural net is like a black box -- you might get it to learn something, but there's almost never any way to generalize what it's learned. There are plenty of other factors to consider as well.
  
  Really, I just hit the tip of the iceberg with neural nets here -- like anything else, you could write a textbook on the subject. As a good resource, I recommend taking a look at the excellent chapter about them in Tom Mitchell's Machine Learning (the little blue book) if you're hungry for more than what you'll find on the (inter)net.

• Great Intro Article
  2005-10-22 21:10:40  Matthew Russell | [View]
  
  
  Without seeing the demo myself, I couldn't tell you for sure -- I wonder if anyone has ported that demo or a similar demo?
  
  Anyway, just remember that conceptually neural nets just take an input and produce an output. The input might be the angle of the seal's nose to the center of the ball, a measure of roll on the ball and whatever else you can think of, and at some point there's a threshold that's going to get crossed. Based on the input and output values, you just quantify the error and then the network basically takes care of the rest internally.
  
  If there's an enthusiastic enough response to this intro to neural nets, I'm sure we'll be able to get you more good stuff on AI -- Baysean lets, decision trees, genetic algorithms, Markov processes, etc. -- it's all good stuff worth knowing about and it's becoming more pervasive in society than you'd think -- your car's transmission, that computer game you love playing, your spam filter, ..., and the list just goes on and on.
  
  If anyone has any specific requests on future AI pieces, feel free to mention them below.