Week 5 Part II · Training Infrastructure

Loss Functions & Metrics

Task-appropriate losses (cross-entropy, MSE, BCE); metrics; the train and eval loop.

Learning goals

Choose a task-appropriate loss.
Track metrics that reveal real performance.
Write a clean train and evaluation loop.

This is the weekly homework lab, completed independently after the lecture and the practice lesson. It follows the course's Build / Predict & probe / Explain & defend model: use an AI assistant freely for the Build; the graded learning is in Predict and Explain. See the AI-use policy and a fully worked sample submission.

⚙Exercise

Part A · AI assistant welcomeBuild

Implement a training loop with loss and metric tracking (accuracy and F1 for classification, or MAE and R2 for regression).

Part B · student reasoningPredict & probe

Predict what happens if MSE is used for a classification task, before trying it.

Part C · in plain languageExplain & defend

Critique an AI-written accuracy metric that is wrong under class imbalance; explain the bug, fix it, and report a better metric.

✓Deliverables

A train and eval loop with metric logging.
The wrong-loss experiment and the metric critique with fix.

Hints.

CrossEntropyLoss expects logits and class indices, not softmax plus MSE.
Accuracy hides minority-class failure; check per-class metrics or F1.

❓Self-check

Answer each before expanding it. If one is unclear, revisit the lab and the references.

Which loss fits multi-class classification, binary classification, and regression?

Cross-entropy; binary cross-entropy; MSE (or MAE).

Why can accuracy be misleading?

Under class imbalance a trivial majority predictor can score high.

Name a metric more robust to imbalance.

F1 (or per-class precision/recall, balanced accuracy, AUC).

What does model.eval() change?

It disables dropout and uses running statistics in batch normalization.

What inputs does CrossEntropyLoss expect?

Raw logits and integer class indices.

Instructor lesson plan (with references)