01-22

• IID assumptions
• underfitting: high training error
• overfitting: high testing error
• capacity: ability to fit a variety of functions
• Ocaam’s razor (principle of parsimony)
• regularization: reducing generalization error but not training error
  • reduce overfitting
• bias vs variance tradeoff: “U curves”
• maximum likelihood estimation: find parameters that optimize
• approximate optimization
• stochastic gradient descent