My Review of ML (CS7641) Machine Learning

           Grade: A 
      Difficulty: 8/10 
          Rating: 10/10
 Time commitment: 15 hours/week
-------------------------------

Overall

This course is hard for two reasons.

(1) The sheer volume of topics covered in this course is massive. It feels like you are drinking from a fire hose. The professor says the goal is to get as much breadth as possible with enough depth so students can go out there and dig in any particular field on their own. They cover all of standard ML topics (supervised, unsupervised & reinforcement learning). Then they even cover some theoretical ML (e.g. PAC learning, Haussler theorem, VC dimension)
(2) Assignments are extremely open ended, and they intentionally don't reveal rubrics.

Lecture quality

The lecture format is unique. Two professors (Charles Isbell & Michael Littman) "chatting" in front of a drawing board. They take turns to give a lecture on each new topic: one professor plays the "teacher" and the other professor plays the "student" who constantly interrupts to asks questions. They often derive algorithm/formula by hand live in the lecture, which is impressive.

I thought this interactive style of lecture was fantastic because you get to see their thought process: they start with the problem, then discuss ideas for solutions, and eventually arrive at an ML algorithm. This way, you really get to see the motivation/intuition behind ML models, instead of getting a bunch of formula & definitions thrown at you.

The downside to this style of lecture is that the video length tends to be very prolonged (the total video length is approx. 28 hours). It often feels like you see they just joke around for 2 minutes, and come back to the lecture. Some students hate this format so much that they refuse to watch it and study from external materials, like Andrew Ng's Coursera series. It's really the matter of individual preferences. Luckily, I personally found this style of lecture fun & engaging.

Assignments

Projects: There are 4 big projects (they call them 'assignments'). The goal is not to derive theorems or implement algorithms from scratch, but it's all about analysis. They tell you to apply various ML algos to certain dataset, and present "interesting" analysis. It's extremely open ended. Typically each assignment can take 20 ~ 40 hours.
- A1 : supervised learning
- A2 : unsupervised learning & dimensionality reduction
- A3 : randomized optimization
- A4 : reinforcement learning
Exams: I thought exams (one midterm & one final) were hard. Again the focus is not about reciting theorems or formulas, but about "synthesis" of various ML principles and technique. e.g. given a dataset of certain characteristics, can you intelligibly discuss why model A might out/under perform model B ?

Grading

This course has a controversial grading scheme. It's part of the projects for you to think about what "interesting analysis" means. In other words, the grading rubrics are hidden. So when you submit a report, you don't know whether you are gonna get 30% or 90%. Yes it's wildly unpredictable.

TAs typically give you any score between 30 ~ 100%. I sometimes got 50%, and other times got 100%. It felt very random. Whenever TA docks points, the feedback is usually "It's great that you discussed X, but you didn't talk about Y & Z." (duh)

Obviously it's frustrating. But the idea is they want you to think about what makes an interesting ML analysis. It defeats the purpose if they told you exactly what to explore, as in which model tends to overfit, which performance metrics to use & why, how outliers influenced the models, why you need to standardize features only for some models, what graphs to plots, why training time varies so much by models, so on.

Some students have serious panic & mental breakdown for receiving 50% score after spending 40 hours. They think they are failing the course. They grow intense hatred, and write long angry posts on Piazza & Reddit. But it's clear from official university records (https://lite.gatech.edu/) that every semester more than 50% of this class gets an A. So I never ever had to worry about my grade, because I knew my reports were good enough to score at least the class median, which guarantees an A. So I just focused on learning.

Thoughts

I think students like or dislike this course depending on whether they appreciate the philosophy of the course, which is to synthesize knowledge. I found the assignments rewarding. I learned so much. This course definitely made me a better data scientist / ML researcher.

They probably should cut out the RL (reinforcement learning) module, because there is a separate dedicated RL course (CS7642) which is designed as a sequel.

Reference

Syllabus : https://omscs.gatech.edu/cs-7641-machine-learning