Week 12 Part III · Architectures & Representation Learning
Representation Learning
Autoencoders and latent representations; contrastive and self-supervised methods.
Learning goals
- Train an autoencoder and a contrastive embedding.
- Probe and interpret a learned latent space.
- Reason about what makes a representation useful.
This is the weekly
homework lab, completed independently after the lecture and the practice lesson. It follows the course's
Build / Predict & probe / Explain & defend model: use an AI assistant freely for the Build; the graded learning is in Predict and Explain. See the
AI-use policy and a
fully worked sample submission.
⚙Exercise
Part A · AI assistant welcomeBuild
- Train an autoencoder and build a simple contrastive embedding.
Part B · student reasoningPredict & probe
- Predict the expected latent-space structure (clusters by class, smooth interpolation).
Part C · in plain languageExplain & defend
- Probe the latent space (interpolate, cluster, nearest neighbors), interpret what it captures, and critique an AI-suggested but flawed contrastive loss.
✓Deliverables
- An autoencoder and embedding notebook with latent-space visualizations.
- An interpretation and the loss critique.
Hints.- A too-large bottleneck just copies the input.
- For contrastive learning, the augmentation choice defines what counts as similar.
❓Self-check
Answer each before expanding it. If one is unclear, revisit the lab and the references.
What are the three parts of an autoencoder?
Encoder, bottleneck (latent), and decoder.
What happens if the bottleneck is too large?
It can copy the input without learning useful structure.
In contrastive learning, what defines a similar pair?
Two augmented views of the same example.
What is the goal of representation learning?
To learn features that make downstream tasks easier and transferable.
Name a self-supervised method.
SimCLR (or MoCo, BYOL, SwAV).
Instructor lesson plan (with references)