title: "Claude API: Deep Dive"
subtitle: "Applied AI — Advanced Level · Session 1"
author: "Yann Isola"
theme: "ink #1A2230 / teal #0F7A6C / copper #B4612A / light-teal #E9F6F3 / bg #F4F7F6"

Slide 1 — Title

Claude API: Deep Dive

Applied AI — Advanced Level · Session 1
Yann Isola · Preparation Claude Certified Architect

Slide 2 — Session Objectives

• Build a complete API request and justify each parameter
• Interpret the 4 values of stop_reason → application logic
• Reason about the context window as working memory
• Master prompt caching (cache_control) and its ROI
• Implement SSE streaming (SSE = Server-Sent Events)
• Arbitrate synchronously / streaming / API Batches
• Design error handling: 429, backoff, retry-after

Slide 3 — The minimum contract

Three required fields: model, max_tokens, messages

curl https://api.anthropic.com/v1/messages \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-sonnet-4-5",
    "max_tokens": 1024,
    "messages": [{"role": "user", "content": "Hello"}]
  }'

Volatile model names — always check docs.anthropic.com

Slide 4 — max_tokens: a ceiling, not a target

• The model can stop before the ceiling
• If it reaches it → response truncated with HTTP 200
• Billing = input tokens + output tokens (different rates, output ≈ 5× )

"usage": {"input_tokens": 58, "output_tokens": 1024}

output_tokens == max_tokens → suspicion of truncation

Slide 5 — Calling via Python SDK

import anthropic
client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY

response = client.messages.create(
    model="claude-sonnet-4-5",        # 
    max_tokens=1024,
    temperature=0.2,                   # technical task
    stop_sequences=["END_REPORT"],
    system="Financial analyst. English, concise.",
    messages=[{"role": "user", "content": "..."}],
)
print(response.content[0].text)
print(response.usage, response.stop_reason)

Slide 6 — Sampling Settings

Certif trap: temperature: 0 ≠ perfect determinism

Slide 7 — Three roles, three privileges

1. system — privileged channel (outside the messages table)
   → persona, rules, exit policy. Not “a disguised first message”
2. user — the caller: questions, documents, tool_result
3. assistant — the model… and the prefilling

Rule: strict alternation, first message = user
Two consecutive identical roles → 400

Slide 8 — Prefilling: putting words in the model’s mouth

messages=[
    {"role": "user", "content": "List 3 risks in JSON."},
    {"role": "assistant", "content": "{"}   # ← prefill
]

• Model continues from {
• Remove preambles (“Of course! Here…”)
• The prefill is not in the response → re-prefix on client side

Slide 9 — stop_reason: the 4 signals

stop_reason	Meaning	Reaction
end_turn	natural ending	nominal
max_tokens	truncated (HTTP 200!)	alert / restart
stop_sequence	sequence encountered	read response.stop_sequence
tool_use	requested tool	execute → tool_result → loop

Slide 10 — The canonical safeguard

match response.stop_reason:
    case "end_turn":
        return extract_text(response)
    case "max_tokens":
        raise TruncatedResponseError(partial=...)
    case "stop_sequence":
        return parse_delimited(..., response.stop_sequence)
    case "tool_use":
        return handle_tool_loop(response)
    case _:
        raise UnexpectedStopReason(...)

Slide 11 — The context window = working memory

• ~200,000 tokens (some models: 1M in beta)
• Contains EVERYTHING: system + history + docs + tools + results
• It's not "a limit" — it's what the model knows for this query

3 consequences: linear cost · lost in the middle · latency (TTFT)

Slide 12 — Cost: the conversation that becomes quadratic

API stateless → full history is re-invoiced every round

# 20 turns, 30k avg history tokens, 500 output tokens
total = sum(cout(30_000, 500) for _ in range(20))
# ≈ $1.95 for ONE conversation 

× 10,000 users/day → technical direction topic

Parades: summary · sliding truncation · prompt caching

Slide 13 — “Lost in the middle”

Recall of information according to its position in the context:

START  ████████████  excellent
MIDDLE ██████        degraded  ← bury nothing here
END    ███████████   excellent

Strategic placement:

• Large documents → at the top
• Critical instructions + question → at the end
• Key instruction in the middle of 80k log tokens →

Slide 14 — Prompt caching: the principle

Reuse a prefix already processed via cache_control

Economy (volatile):

• Reading (hit): ~10% of the price → −90%
• Writing: additional cost ~25%
• TTL ~5 min , refreshed on each hit (1 h option )

Profitable from 2 requests: 1.25 + 0.10 = 1.35 < 2.00

Slide 15 — cache_control: syntax

system=[{
    "type": "text",
    "text": GROS_CONTEXTE,          # 50k stable tokens
    "cache_control": {"type": "ephemeral"}   # ← breakpoint
}]

u = response.usage
u.cache_creation_input_tokens   # written (×1.25 )
u.cache_read_input_tokens       # read (×0.10 )
u.input_tokens                  # uncached, full price

Slide 16 — Cache Invalidation Rules

• Prefix exact: a byte modified upstream → invalidated
• Order: tools → system → messages
• Minimum cacheable: 1,024 tokens (otherwise ignored silently)
• Max 4 breakpoints per request

Target design: stable at the head, variable at the tail

Slide 17 — Anti-pattern: the cache that costs more

system = f"[{datetime.now()}] You are an assistant..."  # 

• Leading timestamp → never identical prefix
• 0% hits + 25% write overhead on each call
• Result: we pay more than without cache

Slide 18 — Count before paying: count_tokens

count = client.messages.count_tokens(
    model="claude-sonnet-4-5",   #  tokenizer per model
    system="...",
    messages=[{"role": "user", "content": doc}],
)
print(count.input_tokens)   # without generation

• Each model family has its tokenizer
• Never reuse a tiktoken count (OpenAI) for Claude
• Orders of magnitude: ~3.5–4 car./token (English), a little denser in French

Slide 19 — Streaming: why

• UX: first token in ~1 s instead of waiting 30 s
• Long generations: essential (HTTP timeouts)
• SSE = Server-Sent Events: unidirectional HTTP flow text/event-stream

event: content_block_delta
data: {"delta":{"type":"text_delta","text":"The"}}

Slide 20 — The HSE event cycle

message_start          ← id, model, INPUT usage
content_block_start    ← opening of block 0
content_block_delta ×n ← the text chunks
content_block_stop     ← closing of block 0
message_delta          ← stop_reason + output_tokens ★
message_stop           ← end of stream

(+ ping keep-alive, + error possible during flow)

★ Certificate trap: final metadata is in message_delta

Slide 21 — Streaming: implementation

with client.messages.stream(
    model="claude-sonnet-4-5", max_tokens=1024,   # 
    messages=[...],
) as stream:
    for text in stream.text_stream:      # high-level helper
        print(text, end="", flush=True)
    final = stream.get_final_message()
    print(final.stop_reason)             # always check!

Low level version: stream=True + loop on event.type
→ required for certification (exercise 2)

Slide 22 — API Batches: the 3rd execution mode

Fashion	Latency	Cost	Usage
Synchronous	seconds	100%	interactive
Streaming	TTFT ~1 s	100%	interactive long
Batch	24 hour SLA	−50%	mass, non-urgent

• Up to 100,000 requests / ~256 MB per batch
• In practice: often < 1 hour
• Results unordered → correlation by custom_id mandatory

Slide 23 — Batch: code

batch = client.messages.batches.create(requests=[
    {"custom_id": f"doc-{i}",            # correlation key
     "params": {"model": "claude-haiku-4-5",   #  small model
                "max_tokens": 512,
                "messages": [...]}}
    for i, doc in enumerate(documents)
])

for r in client.messages.batches.results(batch.id):
    if r.result.type == "succeeded": traiter(r.custom_id, ...)
    else: rejouer(r.custom_id, r.result)  # errored/expired/canceled

Slide 24 — Error handling: taxonomy

Code	Cause	Retry?
400	malformed query	correct
401 / 403	invalid key / no access
413	request too big	reduce
429	rate limit (RPM/ITPM/OTPM)	backoff
500 / 529	internal / overload	backoff

RPM = Requests Per Minute · ITPM/OTPM = Input/Output Tokens Per Minute

Slide 25 — Exponential backoff with jitter

except anthropic.RateLimitError as e:
    retry_after = e.response.headers.get("retry-after")
    if retry_after:
        delai = float(retry_after)        # the server WINS
    else:
        delai = min(60, 2**tentative) * random.uniform(0.5, 1.5)
    time.sleep(delai)

1. retry-after bonus on your formula
2. Mandatory Jitter (“thundering herd” effect)
3. The retry SDK already 2× by default → watch out for multiplicative layers

Slide 26 — Architect’s decision tree

Real-time need ?
├── YES → long response ?  → streaming SSE
│         short response ? → synchronous
└── NO  → massive volume? → Batches API (−50 % )
                            + small model
                            + prompt caching if common prefix

Always: guard stop_reason + backoff 429 + log of usage

Slide 27 — The 7 certification pitfalls seen today

1. max_tokens mandatory, no default value
2. Truncation = HTTP 200 + stop_reason: "max_tokens"
3. The prefill is not included in the response
4. temperature: 0 ≠ perfect determinism
5. Cache: exact prefix, tools → system → messages
6. stop_reason final streaming → message_delta
7. Unordered batch results → custom_id

Slide 28 — Next steps

• Exercises: robust client · low-level streaming · 80k emails batch pipeline
• Quiz: 10 certification MCQs (threshold 7/10)
• Interactive page: query builder + SSE viewer + token counter (offline)
• Session 2: tool use & agents — the loop tool_use → tool_result in depth

Permanent reflex: revalidate prices, limits and model names on docs.anthropic.com