Optimization

Here's the Optimization Help for matlab. This is the key function you will need to implement SVMs. Here's another useful document.

More on Kernel Methods

Videolectures has several videos and lecture slides covering Kernel Methods. Wikipedia has nice definition and additional resources for kernel methods. Support-Vector-Machines.org has additional resources and references for... you can guess what.

Legrange Multipliers

Legrange Multipliers allow us to optimize a function (maximize or minimize) subject to constraints (such as the solution must lie on a specific curve. Dan Klein has a nice tutorial on Lagrange Multipliers, including inequality constraints (less-than or greater-than). The MIT Math Department has a nice applet that shows the solution space for f(x,y) and g(x,y). Notice that as you move the point around, the gradient of f and g (shown as arrows) are parallel at solution points.

Handwritten numbers... data!

Our upcoming projects will use the MNIST handwritten digit database as training and test data. Note that the link has classification results for many techniques taken from the pattern recognition literature. Also notice that the dataset has 60K training patterns and 10K test patterns, where each sample is a 400 dimensional vector. Consider how this might affect a naive least square linear discriminant fit.

Linear Regression

Linear Regression is a method for fitting a model to data. The most common method for solving the fit involves the method of Least Squares. We will be using the methods in the context of Linear Discriminant Analysis. See also Regression Analysis.

Integrals, Bayesian Belief Networks, etc...

Wolfram provides the Integrator for symbolic integration on the web! Use it, it's free, saves a lot of time looking up integral itentities in your old calc book.

For more info on topics covered in class, you might consider
Wikipedia. Eric Weinsteins Mathworld is also a super valuable resource for understanding the math we deal with in this class.

Dont forget that there are slides that go along with the book at the book's website.

Math software

Maple software isn't free, but the student edition may be cheap enough to be worth it.

Here is a nice tutorial on plotting in matlab.

Note: Problems Changed for Chap 2.3 2.4

FYI: I have dropped a few of the more tedious problems for 2.3 and 2.4. The assignment is now as listed on the class web page.

UCI Datasets

The University of California Irvine has a nice collection of sample datasets for exparimenting with pattern classification algorithms. Have a look at what's available. We may use these datasets in future assignments; they may also be useful for your final project.

New Reading and Problem Assignments

New reading and problem assignments have been posted on the class syllabus. Please note: I expect written or printed solutions to be single sided. You must show your work, no magic solutions. While hand written solutions are acceptable, I prefer "typed", latex, or .doc home works. If you do not know latex, but have a copy of MS word, you can install the optional equation editor plugin/tool. Several of the problems call for plots of functions, primarily in 1D. Please be sure to include these plots with your home works. Hand drawn plots must be on graph paper with proper annotation.

Project Research & Journals

The last part of this class will involve a project of your choosing, which represents 20% of your final grade. While the project is your choice, it must be approved by the instructor. I will be looking for projects that solve a relevant pattern recognition problem and utilize modern techniques. Some background research on your part will be required. I recommend IEEE PAMI (Transactions on Pattern Recognition and Machine Intelligence) as a good starting point. You might also find scholar.google useful for identifying the most relevant papers to look at. Use the citations number as a (coarse) indicator of the papers impact.

PDF's map homeless population in LA

The "heat maps" at Cartifact are visualizations that show the density of the homeless population in down town LA. This density method is a visual example of probability density functions based on non-parametric estimation; a topic we will cover in chapter 4.

via blogdowntown and boingboing

Books!

Books should be at the bookstore by Friday Jan 19th; the following Monday at the latest. Amazon has them listed for around $100, but you will want speedy shipping too. Please be sure to fix errors using the authors' errata page.

Wecome to Pattern Recognition

Hi, This is the blog for CS 531/ECE 517. Here is where you will find information, news, and assignments for the class. Please do use the comments to ask questions and provide answers for your classmates.