Neural Networks Notes

Lectures

1A

Machine learning is good for complicated processes combining weak rules to hit a moving target. The machine learning approach is to mine data for rules and it should work for new cases, not just the cases it was trained on.

Best for learning algorithms:

• Recognizing patterns
  • objects in real scenes
  • facial identities or expressions
  • spoken words
• Anomalies
  • unusual sequences of credit card transactions
  • unusual patterns of sensor readings in nuclear power plants
• Prediction
  • future stock prices
  • which movies a person will like

The MNIST database of hand-written digits is the machine learning equivalent of fruit flies to genetics. TIMIT is the same benchmark for speech recognition.

1B : Real Neurons

A cortical neuron has one axon that branches and dendritic tree that collects input form other neurons. A spike of activity in the axon causes charge to be injected into the post-synaptic neuron. Shen a spike of activity travels along an axon and arrives at a synapse it causes vesicles of transmitter chemicals to be released. There are several kinds of transmitter.

• Each neuron receives input from other neurons (a few neurons are also connect to receptors)
• The effect of each input line on the neuron is controlled by a synaptic weight.
• The synaptic weight adapts so that the whole network learns to perform useful computations.
• You have 10^11 neurons each with about 10^4 weights.

The cortex is modular. Different bits of the cortex do different things. Local damage to the brain has specific effects. But the cortex looks pretty much the same all over. This suggests that there is a uniform learning pattern at work.