Systems session 3/3 · Exercises

KV-cache sizing and optimization

  1. Estimate cache memory for short and long contexts.
  2. Repeat at concurrency 1, 8, and 32.
  3. Propose a bounded-context or prefix-reuse experiment and define success metrics.
Show answer

Under the simplified linear estimate, it approximately doubles. No. Provider prefix reuse has provider-specific persistence and eligibility rules.