Optimization

Here's the Optimization Help for matlab. This is the key function you will need to implement SVMs. Here's another useful document.

More on Kernel Methods

Videolectures has several videos and lecture slides covering Kernel Methods. Wikipedia has nice definition and additional resources for kernel methods. Support-Vector-Machines.org has additional resources and references for... you can guess what.

Legrange Multipliers

Legrange Multipliers allow us to optimize a function (maximize or minimize) subject to constraints (such as the solution must lie on a specific curve. Dan Klein has a nice tutorial on Lagrange Multipliers, including inequality constraints (less-than or greater-than). The MIT Math Department has a nice applet that shows the solution space for f(x,y) and g(x,y). Notice that as you move the point around, the gradient of f and g (shown as arrows) are parallel at solution points.

Handwritten numbers... data!

Our upcoming projects will use the MNIST handwritten digit database as training and test data. Note that the link has classification results for many techniques taken from the pattern recognition literature. Also notice that the dataset has 60K training patterns and 10K test patterns, where each sample is a 400 dimensional vector. Consider how this might affect a naive least square linear discriminant fit.

Linear Regression

Linear Regression is a method for fitting a model to data. The most common method for solving the fit involves the method of Least Squares. We will be using the methods in the context of Linear Discriminant Analysis. See also Regression Analysis.

Integrals, Bayesian Belief Networks, etc...

Wolfram provides the Integrator for symbolic integration on the web! Use it, it's free, saves a lot of time looking up integral itentities in your old calc book.

For more info on topics covered in class, you might consider
Wikipedia. Eric Weinsteins Mathworld is also a super valuable resource for understanding the math we deal with in this class.

Dont forget that there are slides that go along with the book at the book's website.

Math software

Maple software isn't free, but the student edition may be cheap enough to be worth it.

Here is a nice tutorial on plotting in matlab.