Instructor: Emo Todorov

TA: Miro Enev

Text: Neural Networks: A Comprehensive Foundation (2nd Edition)

Lecture Notes (PDF format):

• Introduction (01-09-2007)
• Linear Algebra (01-11-2007)
• Error Minimization and Gradients (01-16-2007)
• Minimization (continued) and Gradient Descent (01-18-2007)
• Perceptron and Gauss Newton Method (01-23-2007)
• Gauss Newton Method Continued (01-25-2007)
• K-Means and Bayesian Inference (02-01-2007)
• Backpropagation (02-06-2007)
• Model Selection (02-08-2007)
• Recurrent Networks (02-13-2007)
• Recurrent Backpropagation (02-15-2007)
• Kohonen Map (02-21-2007)
• Dynamical Systems Controlled via NN (02-26-2007)
• Distributed Representations (03-01-2007)
• Optimal Control (03-08-2007)
• Dimensionality Reduction (03-13-2007)

Assigned Readings:

• Week 1: Read Haykin Chapters 1 & 2
• Week 2: Read Haykin Chapter 3
• Week 2: PDP Chapter 1 - The Appeal of Parallel Distributed Processing
• Week 3: Read Haykin Chapter 4
• Week 4: Re-read Haykin Section 3.10, Review Backpropagation (Ch. 4)
• Week 5: Suggested Readings - Computational Philosophy & Abstraction
  1. David Marr's Introduction to 'Vision'
  2. Large-Scale Theories of the Brain - A Critique of Pure Vision
• Week 6: Your choice of article from IJCNN Proceedings
• Week 7: None - In Class Presentations
• Week 8: Neuro Animator - Hinton 1998 (6.8 MB)
• Week 9: Haykin Section 15.7 - Back Propagation Through Time

Assignments and Projects:

• Assignment 1 (Due 1/25/07) - Gradient Descent (Solution)
• Project 1 - Neuron Locator (Solution)
• Assignment 2 (Due 3/1/07) - Implement the Backpropagation Algorithm (Updated 2-15-07)
• Assignment 3 (Due 2/20/07) - Present Article from IJCNN (work in pairs, email article choice to TA)
• Assignment 4 (Due 3/23/07) - Brain, Body, and Back Propagation Through Time (UPDATED: 3-20-2007)
  

  A helpful reading on weight sharing

  If you have not submitted a solution to the previous assignment (backpropagation), you are expected to have it done by Friday.
  Note that a working implementation of BPTT grants you credit for the missed backpropagation assignment.

Additional Materials:

• Matlab
  • Matlab Primer
  • Matlab Review (includes code)
• Linear Algebra
  • Introduction to Linear Algebra Concepts
  • Linear Algebra Review (Matlab based)
• The Gradient
  • Visual Introduction to Partial Derivatives and the Gradient
  • Another Set of Lecture Notes for Gradient Descent
• Modeling
  • Levels for Modeling
  • Appropriate Model Complexity