Systems session 1/3 · Instructor guide

Tokenizers and token budgets

Assumptions

Demonstrate one English sentence, one French sentence, and one code sample. Do not present the approximation as an exact tokenizer.

60 min

  1. 10 min — framing
  2. 20 min — concepts
  3. 20 min — measured activity
  4. 10 min — review