Brief course description

CNS/Bi/Ph/CS 187. Neural Computation. 9 units (3-0-6); first term. (Note that CNS 187 is a direct descendent of the former CNS 185, which is no longer taught.) This course investigates computation by neurons. Of primary concern are models of neural computation and their neurological substrate, as well as the physics of collective computation. Thus, neurobiology is used as a motivating factor to introduce the relevant algorithms. Topics include rate-code neural networks, their differential equations and equivalent circuits; stochastic models and their energy functions; associative memory; supervised and unsupervised learning; development; spike-based computing; single-cell computation; error and noise tolerance.

Required background

This course is rather interdisciplinary as it attempts to pull together topics in biology, physics, mathematics, and computer science. However, lack of background in one or more of these areas need not prevent you from succeeding in the course. We do require some basics: familiarity with computer programming in a high level language (ideally MATLAB), and knowledge of basic calculus, probability, linear algebra, and some familiarity with differential equations.

Useful Matlab references:

Textbook

This course has three required textbooks, Introduction to the Theory of Neural Computation by A. Hertz, A. Krogh, and R. Palmer, Information Theory, Inference and Learning Algorithms , by David MacKay, and Spiking Neuron Models by W. Gerstner and W. Kistler.
All these books are available in the bookstore. MacKay's book is also available online. Although these books will be the main references, additional books and primary source papers will be useful on a lecture by lecture basis. Reference material will be available as appropriate.

The following books are also highly recommended:

Arbib, M., The Handbook of Brain Theory and Neural Networks
P. Dayan and L.F. Abbott, Theoretical Neuroscience
Maass, W., and Bishop, C., Pulsed Neural Networks

Anderson, J., Neurocomputing Foundations of Research
Ballard, D., Introduction to Natural Computation
Bishop, C., Neural Networks for Pattern Recognition
Kandel, E. Schwartz, J. and Jessell T., Principles of Neural Science
Koch, C., Biophysics of Computation
Minsky, M., Computation: Finite and Infinite Machines
Nicholls, J., From Neuron to Brain A Cellular & Molecular Approach to the Function of the Nervous System
PDP Research Group Staff, Parallel Distributed Processing Explorations in the Microstructure of Cognition, Vol. 1: Foundations
Rieke, F., Warland, D., de Ruyter van Steveninck, R. and Bialek, W., Spikes; Exploring the Neural Code
Rumelhart, D., Parallel Distributed Processing Explorations in the Microstructure of Cognition, Vol. 2: Psychological & Biological Models
Shepard, G., The Synaptic Organization of the Brain
Strogatz, S., Nonlinear Dynamics and Chaos
Other references that may be helpful.

Instructors, teaching assistants, and web page

CNS 187 is taught by Professor Erik Winfree. If you have a question for the TAs, you are strongly encouraged to use of the cns187 e-mail address cns187_ta@dna.caltech.edu (as opposed to the telephone or personal email). The TAs will hold office hours once per week -- see the main class page. During their office hours, the TAs will be available for assistance on any topic covered in the class but will not be available for assistance regarding computer programming. You are advised to regularly check the class webpage at http://www.dna.caltech.edu/cns187; it will continually be updated with homeworks, syllabus reviews, etc. http://www.dna.caltech.edu/cns187 , so be sure to check it regularly!

Electronic mail

Questions about homeworks or administrative details can be sent to the email address cns187_ta@dna.caltech.edu. This goes to the TAs. There will also be a mailing list including the addresses of all the students in the class, cns187students@dna.caltech.edu. Anything sent to it will be automatically broadcast to everyone. The TAs will send homework clarifications (if necessary) to this mailing list, so check your e-mail regularly!

Problem sets

There will be an assignment due each week covering the material taught in the lectures and covered in the references. It is important to read the references for a better understanding of the lectures and assignments. However, do not consult materials that directly provides a solution to the problem: every problem should can and should be solved using only the information provided in lectures (except where specifically indicated). For example, do not consult problem sets or solutions sets from previous years. The homework sets will include computer simulations in MATLAB, mathematical derivations and analysis. Collaboration is permitted for understanding and discussing all problem sets, but each student must write their own submission in their own words and understand everything they hand in. You should also write your own matlab codes. Names of collaborators must be given on the homework. This will not affect grading.

Problem set grading and lateness policy

Problem sets will be given a grade from 0 thru 100; a failing grade (F) will be given for inexcusably poor or absent problem sets. Decisions of the TA's regarding problem set performance will be final. Answers are expected to be clear, concise, and thorough; if a TA has questions about the answer your provide, you may be required to speak to him for full credit (this will be clearly marked).

You may consult textbooks and web material about fundamental concepts in neural network, but you may not consult such material if it provides a solution to problem identical or nearly identical to the one in the CNS 187 homework set. Furthermore, you may not consult solution sets from previous years, or homework sets and/or solutions by students who took the course in previous years.

Homeworks will be posted on the course webpage on Thursdays and are due the following Friday before 5:00pm in the TA's mailboxes unless otherwise noted. The class mailbox is located on the first floor of Moore. Problem sets handed in late, but before 5pm on Monday will be penalized by 20%. Problem sets will not be accepted after this time, and therefore given an (F), as the solutions will have been posted on the web by Tuesday morning. One free extension to Monday will be granted to each student (don't use it up needlessly!); a longer extension requires permission from the instructor. Further extensions will only be granted for extreme circumstances (e.g. serious medical or family issues). Any non-emergency extensions must be cleared before the due date with the instructor or one of the TA's.

Exams

There will not be a midterm nor a final exam.

Course grading policy

The course will be graded on the basis of the regularly assigned problem sets.

Computer accounts

Computer programming and Web browsing are integral parts of the course. Matlab is available on the ITS and UGCS computer clusters, among others. If you need an account on the UGCS machines, use the UGCS/CS Computer Account Request Form.