Week 2 Part I · Foundations
Tensors & Data Representation
Instructor lesson plan: lecture (3 h) and practice (2 h).
Learning objectives
- Manipulate tensors fluently (reshape, permute, reduce, index).
- Reason about shapes, broadcasting, and device placement.
- Encode real data into correctly shaped tensors.
🎓Lecture · 3 hours
| 0:00–0:10 | 10 min | Recap & retrievalOpen with two quick questions on last week's material (retrieval practice), then state this week's objectives. |
| 0:10–0:25 | 15 min | MotivationEverything in PyTorch is a tensor; fluency here prevents most beginner bugs. |
| 0:25–1:10 | 45 min | Tensors and shapes- A tensor has a shape, a dtype, and a device; most beginner bugs are a shape or dtype mismatch.
- Reshape, view, permute, and flatten rearrange dimensions; reductions (sum, mean) collapse them.
- Indexing and slicing select sub-tensors, with the same start-inclusive, stop-exclusive rule as Python.
- Contiguity: a view shares memory, a reshape may copy; call .contiguous() when an op needs it.
|
| 1:10–1:20 | 10 min | Break |
| 1:20–2:05 | 45 min | Broadcasting and data representation- Broadcasting compares shapes from the trailing dimension; sizes must match or be 1 (which then expands).
- Images are (N, C, H, W); text is integer token ids; tabular data is (N, features).
- dtypes matter: float32 for inputs, long for class indices.
- Move tensors and models to the GPU with .to(device) and keep them on the same device.
|
| 2:05–2:35 | 30 min | Live demo (predict, then run)Show two shapes and ask the class to predict whether they broadcast and to what shape, before running the operation. Shape gymnastics, a broadcasting puzzle, and a deliberate shape-mismatch error with its fix; then encode an image and a table. |
| 2:35–2:50 | 15 min | Wrap-up & practice previewRevisit the misconception and concept checks below, recap the takeaways, and preview the practice lesson. |
| 2:50–3:00 | 10 min | Buffer & questions |
Common misconception to confront. Students often think: Broadcasting aligns shapes from the first (leftmost) dimension.
Set it straight: It aligns from the trailing (rightmost) dimension: (3,) broadcasts with (5,3) but not with (3,5).
Check for understanding (pose during the concept blocks; let students answer before revealing).
Can a (3,) tensor be added to a (5,3)? To a (3,5)?
(5,3): yes, trailing dims match (3==3) and it expands across the 5 rows. (3,5): no, trailing dims 3 vs 5 mismatch.
Why must class labels be long (int64) and inputs float32?
Cross-entropy indexes the target class with an integer; the matrix multiplies need floating-point inputs. A float label or double input is a classic dtype bug.
Key takeaways.- Shape errors are the most common beginner bug.
- Broadcasting aligns dimensions from the trailing axis.
- Know when an operation copies versus views memory.
💻Practice · 2 hours
In the practice lesson the instructor demonstrates implementations, runs code, and works through examples, using the practice notebook linked below. The weekly lab is then set as homework, where students apply this themselves.
| 0:00–0:10 | 10 min | Setup & recapRecap the lecture's key ideas and open the working notebook. |
| 0:10–1:00 | 50 min | Instructor demonstrations- Demonstrate tensor creation, reshape, permute, and indexing in a notebook.
- Show broadcasting on several shape examples; trigger a shape-mismatch error and fix it.
|
| 1:00–1:05 | 5 min | Break |
| 1:05–1:45 | 40 min | Instructor demonstrations (continued)- Encode an image and a small table into tensors and move them to the device.
|
| 1:45–2:00 | 15 min | Wrap-up & lab briefSummarize the patterns shown and brief the weekly lab (homework), which students complete on their own. |
Common pitfalls to pre-empt.- Print .shape and .dtype often; broadcasting aligns dimensions from the trailing axis.
- .view needs contiguous memory; .reshape may copy.
Open the practice notebook in Colab Curated references Lab (homework)