Systems session 1/3 · Quiz & review

Tokenizers and token budgets

Can two models assign different token counts to the same sentence?

Show answer

Yes. Vocabulary, normalization, and segmentation rules differ by tokenizer.

What must fit in the context window?

Show answer

The complete serialized request plus the output tokens reserved for generation.

What evidence closes this lesson?

Show answer

A measured result with assumptions, exact revisions, and a quality check.