Ch 7 · Evaluating a Model Honestly · Machine Learning & AI

Topic 7 of 13 in Machine Learning & AI — Foundations — 2 lessons.

Accuracy and Its Limits

Accuracy is the share of correct predictions. Simple, but dangerous on imbalanced data: if 99% of emails are not-spam, a model that says "never spam" scores 99% while being useless.

Precision, Recall & F1

When accuracy lies, split the errors apart: precision asks how many of your positive predictions were right, while recall asks how many of the real positives you actually found. Which one matters more depends on which mistake is costlier. The example counts true positives, false positives and false negatives to work out the scores. In the first model, one false alarm and one miss pull precision and recall down to 0.75 each, so F1 is 0.75 too. The second, over-cautious model has perfect precision but finds only 1 of the 4 real positives, and its F1 of 0.4 shows that one great score cannot hide a bad one.

All topics in Machine Learning & AI Beginner

1. What Is Machine Learning?
2. Data — Features, Labels & Splitting
3. Linear Regression — Fitting a Line
4. How Models Learn — Gradient Descent
5. Classification — Predicting Categories
6. Decision Trees
7. Evaluating a Model Honestly
8. Overfitting & the Bias-Variance Tradeoff
9. The ML Workflow
10. Mini-Project — A Spam Classifier
11. Clustering — Finding Groups Without Labels
12. Your First Real scikit-learn Model
13. Words as Numbers — A Taste of NLP