# Trainer’s Guide — Advanced Level, Session 1
# “Claude API: Deep Dive”

**Program:** Applied AI — Yann Isola
**Audience:** Solutions architects preparing for certification *Claude Certified Architect*
**Duration:** 2 hours (+ 10 minutes recommended break halfway through)
**Prerequisites:** Intermediate Python, notions of HTTP/REST (REST = Representational State Transfer), having already called any API (API = Application Programming Interface).
**Hardware:** Anthropic demo API key, terminal with `curl` and Python 3.10+, SDK (SDK = Software Development Kit) `anthropic` installed, interactive session web page (`webpage/index.html`).

---

## Educational objectives

At the end of the session, each participant knows:

1. **Build** a complete Claude API request and justify each parameter (`model`, `max_tokens`, `messages`, `system`, `temperature`, `stop_sequences`).
2. **Interpret** each value of `stop_reason` and design the corresponding business logic.
3. **Reason** about the context window like a working memory: linear cost, “lost in the middle” effect, placement strategies.
4. **Implement** prompt caching with `cache_control` and calculate the return on investment (ROI = Return On Investment).
5. **Implement** SSE streaming (SSE = Server-Sent Events) and name the lifecycle events of a message.
6. **Choose** between synchronous call, streaming and Batches API according to the use case, with the cost/latency/SLA arguments (SLA = Service Level Agreement, service level commitment).
7. **Design** a robust error handling strategy: 429, exponential backoff, `retry-after` headers.

---

## Timed plan

| Block | Duration | Content |
|------|-------|---------|
| 0. Opening | 5 mins | Certification framework, express roundtable |
| 1. Anatomy of a query | 20 mins | model, max_tokens, messages, system — curl + Python demo |
| 2. Roles & prefilling | 15 mins | user/assistant/system, response prefilling |
| 3. stop_reason | 10 mins | The 4 values, application logic |
| 4. Context window | 15 mins | Working memory, linear cost, lost in the middle |
| **Pause** | 10 mins | |
| 5. Prompt caching | 20 mins | cache_control, saving 90%, additional cost 25%, ROI calculations |
| 6. Token counting | 10 mins | Endpoint count_tokens, tokenizers by family |
| 7. SSE Streaming | 15 mins | Cycle of events, live demo |
| 8. API Batches | 10 mins | Async, −50%, 24h SLA, 100k requests |
| 9. Error handling | 10 mins | 429, exponential backoff, retry-after |
| 10. Closing | 5 mins | Exit tickets, announcement of exercises |

---

## Block 0 — Opening (5 min)

**Catch message:** “Today we're not talking about *prompting*. We're talking about what distinguishes a developer who calls Claude from an architect who designs a system around Claude: the API contract, the token economy, and failure modes. »

Flash question to the group: *“Who has ever received a 429 error in production? What have you done? »* — collect 2-3 answers, come back to them in block 9.

---

## Block 1 — Anatomy of an API request (20 min)

### 1.1 The minimum contract

Three fields are **required**: `model`, `max_tokens`, `messages`.

**Curl demo (to type live):**```bash
curl https://api.anthropic.com/v1/messages \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-sonnet-4-5",
    "max_tokens": 1024,
    "messages": [
      {"role": "user", "content": "Explique le protocole FIX en une phrase."}
    ]
  }'
```⚠ Model name (`claude-sonnet-4-5`) is **volatile**: check official documentation before any course material or deployment.

**Points to hammer home:**

- `anthropic-version`: API version pinning header. Without him → error. It's a contract: behavior will not change under your feet.
- `max_tokens` is a **generation cap**, not a target. The model may stop before. But if it reaches it, the response is **truncated** (see `stop_reason: "max_tokens"` in block 3).
- Billing concerns input tokens **+** output tokens, at different prices (output typically costs ~5× input ⚠ depending on the model).

### 1.2 Python equivalent (official SDK)```python
import anthropic

client = anthropic.Anthropic()  # lit ANTHROPIC_API_KEY dans l'environnement

response = client.messages.create(
    model="claude-sonnet-4-5",          # ⚠ volatil
    max_tokens=1024,
    temperature=0.2,                     # déterminisme relatif pour tâches techniques
    stop_sequences=["FIN_RAPPORT"],      # arrêt personnalisé
    system="Tu es un analyste financier. Réponds en français, style concis.",
    messages=[
        {"role": "user", "content": "Résume les risques d'un stablecoin adossé à des matières premières."}
    ],
)

print(response.content[0].text)
print(response.usage)        # input_tokens / output_tokens → pilotage des coûts
print(response.stop_reason)  # toujours l'inspecter en production
```**Decryption of the answer:**```json
{
  "id": "msg_01XFDUDYJgAACzvnptvVoYEL",
  "type": "message",
  "role": "assistant",
  "content": [{"type": "text", "text": "..."}],
  "model": "claude-sonnet-4-5",
  "stop_reason": "end_turn",
  "usage": {"input_tokens": 58, "output_tokens": 212}
}
```- `content` is an **array of blocks** (not a simple string): blocks `text`, `tool_use`, etc. Architect habit: always iterate on the blocks, never assume `content[0]` unique.
- `usage`: this is your billing counter. In production, we **systematically log** it (cost observability).

### 1.3 Sampling parameters

| Parameter | Role | Architect recommendation |
|---------------|------|----------------------------|
| `temperature` (0–1) | Sampling hazard | 0–0.3 for extraction/classification, 0.7–1 for creativity |
| `top_p` | Nucleus sampling | Do not combine with `temperature` — choose one |
| `top_k` | Restricted to the k most probable tokens | Rarely necessary, advanced cases |
| `stop_sequences` | Custom Stop Chains | Useful for delimiting structured outputs |

**Certification pitfall:** `temperature: 0` does **not** guarantee perfect determinism (residual numerical non-determinism possible). Correct wording: “significantly reduces variability”.

---

## Block 2 — Message roles & prefilling (15 min)

### 2.1 Three channels, three privilege levels

1. **`system`** — **privileged** channel. This is NOT a “disguised first user message”. It is treated with particular priority by the model. Uses: persona, security rules, exit policy (format, language, refusal), stable business context.
2. **`user`** — the caller's turn. Content: question, documents, tool results (`tool_result`).
3. **`assistant`** — model tricks. But also: **prefilling**.

**Structural rule:** the table `messages` must alternate user/assistant and **start with `user`**. Two consecutive messages with the same role → error 400.

### 2.2 Prefilling: putting words into the model’s mouth

The last message may be a `assistant` **partial**: the pattern continues from there.```python
response = client.messages.create(
    model="claude-sonnet-4-5",  # ⚠
    max_tokens=500,
    messages=[
        {"role": "user", "content": "Liste 3 risques de contrepartie en JSON."},
        {"role": "assistant", "content": "{"}   # prefill : force l'ouverture JSON
    ],
)
# La réponse commence directement après "{" → pas de préambule "Voici le JSON :"
```**Architect use case:**
- Force an output format (JSON — JavaScript Object Notation, XML — eXtensible Markup Language).
- Remove preambles (“Of course! Here…”).
- Constrain a choice: prefill `"La réponse est ("` for a multiple choice questionnaire (MCQ = Multiple Choice Questionnaire).

**Warning:** the prefill text is **not** part of the returned response — remember to re-prefix it on the client side (`"{" + response.content[0].text`).

**Suggested live demo:** same question with and without prefill, compare outputs. Immediate effect, very telling.

---

## Block 3 — stop_reason: the 4 signals (10 min)

Each response indicates **why** the model stopped. An architect writes a branch of code for each.

| `stop_reason` | Meaning | Application reaction |
|---------------|---------------|----------------------|
| `end_turn` | Natural end of the round | Nominal case — process response |
| `max_tokens` | Ceiling reached → **truncated answer** | Alert/relaunch with higher ceiling, or continue generation |
| `stop_sequence` | A `stop_sequences` was encountered | Read `response.stop_sequence` to find out which one; parse bounded output |
| `tool_use` | The model requests the execution of a tool | Run the tool, return a `tool_result`, loop back |

**Canonical custody code (to be written to participants):**```python
match response.stop_reason:
    case "end_turn":
        return extract_text(response)
    case "max_tokens":
        logger.warning("Réponse tronquée — usage=%s", response.usage)
        raise TruncatedResponseError(partial=extract_text(response))
    case "stop_sequence":
        return parse_delimited(extract_text(response), response.stop_sequence)
    case "tool_use":
        return handle_tool_loop(response)
    case _:
        raise UnexpectedStopReason(response.stop_reason)
```**Certification pitfall:** `max_tokens` is **not an HTTP error** — the request returns 200. This is a business state to detect yourself. Many production systems silently deliver truncated JSON because no one is testing `stop_reason`.

---

## Block 4 — The context window as working memory (15 min)

### 4.1 Change mental model

The context window (⚠ 200,000 tokens on most current Claude models, some models offer 1M in beta) is not “a limit not to be exceeded”. This is the **working memory** of the model: everything it “knows” for this query is there — system prompt, history, documents, tool definitions, tool results.

**Three architectural consequences:**

1. **Linear cost.** Each entry token is charged on each call. A conversation that accumulates 150k history tokens costs 150k entry tokens **per round**. Without strategy (summarization, truncation, caching), the cost of a conversation increases quadratically with its length (sum of prefix lengths).
2. **“Lost in the middle**” effect.** Models recall information placed at the **beginning** and **end** of the context better than in the **middle**. Strategic placement: critical instructions and question as close as possible to the end; bulky documents in mind; never bury a key instruction in the middle of 80k log tokens.
3. **Latency.** The time until the first token (TTFT = Time To First Token) increases with the size of the input.

### 4.2 Mental exercise (2 min, orally)

“A RAG assistant (RAG = Retrieval-Augmented Generation) injects 40 chunks of 1,000 tokens. Where do you place the user's question? » → Expected response: after the documents, at the end of the prompt, possibly repeated if the documents are very long.

### 4.3 Live cost calculation```python
# Ordre de grandeur — tarifs ⚠ volatils, vérifier la grille officielle
PRIX_INPUT_PAR_MTOK = 3.00    # $ / million de tokens d'entrée ⚠
PRIX_OUTPUT_PAR_MTOK = 15.00  # $ / million de tokens de sortie ⚠

def cout_appel(input_tokens: int, output_tokens: int) -> float:
    return (input_tokens * PRIX_INPUT_PAR_MTOK
            + output_tokens * PRIX_OUTPUT_PAR_MTOK) / 1_000_000

# Conversation de 20 tours, historique moyen 30k tokens, réponses 500 tokens
total = sum(cout_appel(30_000, 500) for _ in range(20))
print(f"{total:.2f} $")   # ≈ 1.95 $ pour UNE conversation
```Multiply by 10,000 users/day → the context economy becomes a subject of **technical direction**, not a detail.

> 📌 **Panorama of border models (July 2026, ⚠ volatile — revalidate official grids):** the market is organized into families, each offering an expensive high-end model and lighter variants. As a benchmark: **OpenAI GPT-5.6** (July 9, 2026, family *Sol / Terra / Luna*, variant **Sol Ultra**, reasoning effort up to `max`, “ultra mode” with sub-agents; **Terra** ≈ GPT-5.5 at half price), **xAI Grok 4.5** (July 8 2026, positioned “Opus-class”, strong in coding/agentic, configurable reasoning effort *low/med/high*, API announced around **2 $/million en entrée et 6 $/million in output** ⚠; Llama; Spark 1.0 = April 2026, Muse Image = July 7, Meta Model API in preview). Takeaway for an architect: **never** hard-code a model name in your cost reasoning — set the price per million tokens, and make the choice of model replaceable (see Session 8, *quantization* and routing; “model-agnostic” principle of Session 10).

---

## Block 5 — Prompt caching (20 min)

### 5.1 The principle

Prompt caching allows you to **reuse the prefix** of a prompt that has already been processed. We set break points `cache_control`; everything above (and matches exactly) is served from cache.

**Savings:** ⚠
- **Cache hit: ~10% of the normal price → 90% savings** on the hidden portion.
- **Cache write: additional cost of ~25%** on the written portion.
- Basic TTL (TTL = Time To Live, lifespan): ~5 minutes ⚠, refreshed on each hit; 1 hour option ⚠ available at higher additional cost.

### 5.2 Syntax```python
response = client.messages.create(
    model="claude-sonnet-4-5",  # ⚠
    max_tokens=1024,
    system=[
        {
            "type": "text",
            "text": GROS_CONTEXTE_METIER,   # ex. 50k tokens de documentation
            "cache_control": {"type": "ephemeral"}   # ← point de rupture
        }
    ],
    messages=[{"role": "user", "content": question_utilisateur}],
)

u = response.usage
print(u.cache_creation_input_tokens)  # tokens écrits au cache (surcoût 25 % ⚠)
print(u.cache_read_input_tokens)      # tokens lus du cache (−90 % ⚠)
print(u.input_tokens)                 # tokens non cachés, plein tarif
```**Rules to know (certification):**
- The cache works on **exact prefix**: the slightest byte modified upstream of the breakpoint invalidates the cache. → Put the **stable content** first (system, tools, documents), the **variable content last** (question).
- Order in the prefix: `tools` → `system` → `messages`. A tool definition change invalidates everything.
- Minimum cacheable: ⚠ 1,024 tokens on most models (2,048 on some). Below, the breakpoint is silently ignored.
- Up to **4 breakpoints** ⚠ per request.

### 5.3 Profitability calculation (to be done)

“System prompt + docs = 60k tokens, question = 300 tokens, 500 requests/hour. »

- Without cache: 500 × 60,300 full price tokens.
- With cache: 1 write (60k × 1.25) + 499 reads (60k × 0.10) + 500 × 300 full price.
- → saving ≈ **88%** on entry. The cache is profitable from **2 requests** in the TTL window (1.25 + 0.10 = 1.35 < 2.00).

**Anti-pattern to quote:** place a timestamp or session identifier at the start of the system prompt → 0% cache hit, 25% overhead on each call. We pay *more* than by deactivating the cache.

---

## Block 6 — Token counting (10 min)

### 6.1 Why count before sending

- Validate that you are in the window **before** paying.
- Size `max_tokens` intelligently.
- Pre-calculate a budget / do internal chargeback.

### 6.2 The dedicated endpoint```python
count = client.messages.count_tokens(
    model="claude-sonnet-4-5",   # ⚠ le comptage dépend du modèle
    system="Tu es un assistant juridique.",
    messages=[{"role": "user", "content": contrat_complet}],
)
print(count.input_tokens)   # gratuit, pas de génération ⚠ (soumis à rate limit dédié)
```**Key points:**
- Each **family of models has its tokenizer**: the same text does not make the same number of tokens on Claude and on a GPT model (GPT = Generative Pre-trained Transformer), nor necessarily between generations of Claude. → Never reuse a tiktoken count (OpenAI tokenizer) to size a Claude call.
- Orders of magnitude for mental estimation: **~3.5–4 characters/token in English**, a few more tokens/word in French (accents, morphology). Code and JSON tokenize more densely than you might think.
- Count includes system + messages + tools: pass **full** request to `count_tokens`.

---

## Block 7 — SSE Streaming (15 min)

### 7.1 Why stream

- **UX** (UX = User eXperience): perceived TTFT of ~1 s instead of waiting 30 s for a complete response.
- **Mandatory in practice** for long generations (HTTP timeouts wait for responses > 10 min).

### 7.2 The SSE event cycle

SSE = Server-Sent Events: unidirectional HTTP flow `text/event-stream`, events `event:` + `data:` separated by empty lines.

Sequence for a simple message:```
message_start          → enveloppe du message (id, model, usage d'entrée)
content_block_start    → ouverture du bloc n°0 (type: text)
content_block_delta    → {"delta": {"type": "text_delta", "text": "Le"}}
content_block_delta    → {"delta": {"type": "text_delta", "text": " protocole"}}
...                       (des dizaines/centaines de deltas)
content_block_stop     → fermeture du bloc n°0
message_delta          → stop_reason + usage de sortie finaux
message_stop           → fin du flux
```(+ `ping` keep-alive events to ignore, + `error` possible during flow.)

**Certification pitfall:** `stop_reason` and the final output token count arrive in **`message_delta`**, not in `message_start`. A client who does not read `message_delta` will never know if the response was truncated.

### 7.3 Implementation```python
# Version haut niveau (SDK)
with client.messages.stream(
    model="claude-sonnet-4-5",  # ⚠
    max_tokens=1024,
    messages=[{"role": "user", "content": "Explique le netting bilatéral."}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
    final = stream.get_final_message()
    print("\n→", final.stop_reason, final.usage)
```

```python
# Version bas niveau (événements bruts) — celle qu'il faut connaître pour la certif
stream = client.messages.create(..., stream=True)
for event in stream:
    match event.type:
        case "message_start":
            msg_id = event.message.id
        case "content_block_delta":
            if event.delta.type == "text_delta":
                buffer += event.delta.text
        case "message_delta":
            stop_reason = event.delta.stop_reason
            output_tokens = event.usage.output_tokens
        case "message_stop":
            break
```**Demo:** Open the session web page streaming viewer (`webpage/index.html`) — each event is displayed with its colored type. Very effective in anchoring the sequence.

---

## Block 8 — API Batches (10 min)

### 8.1 The third execution mode

| Fashion | Latency | Cost | Use cases |
|------|---------|------|-------------|
| Synchronous | seconds | full price | interactive |
| Streaming | first token in ~1 s | full price | interactive, long outings |
| **Batch** | **up to 24 hours (SLA)** ⚠ | **−50%** ⚠ | mass treatment, non-urgent |

- Up to **100,000 requests** ⚠ (or ~256 MB ⚠) per batch.
- Most batches finish in **less than an hour** in practice; 24 hours is the contractual commitment.
- Each request in the batch is independent (no state sharing).
- Results available 29 days ⚠, **not guaranteed in order** → always correlate by `custom_id`.

### 8.2 Code```python
batch = client.messages.batches.create(
    requests=[
        {
            "custom_id": f"doc-{i}",                      # clé de corrélation — OBLIGATOIRE
            "params": {
                "model": "claude-haiku-4-5",              # ⚠ petit modèle pour la masse
                "max_tokens": 512,
                "messages": [{"role": "user", "content": f"Classifie : {doc}"}],
            },
        }
        for i, doc in enumerate(documents)
    ]
)

# Polling de l'état
status = client.messages.batches.retrieve(batch.id)
print(status.processing_status)        # in_progress → ended
print(status.request_counts)           # succeeded / errored / canceled / expired

# Récupération (JSONL — JSON Lines, un objet JSON par ligne)
for result in client.messages.batches.results(batch.id):
    if result.result.type == "succeeded":
        traiter(result.custom_id, result.result.message)
    else:
        rejouer(result.custom_id, result.result)   # errored / expired / canceled
```**Use cases for the group to find:** nightly classification of tickets, enrichment of CRM (CRM = Customer Relationship Management), massive evaluation of prompts, generation of documentary metadata, backtesting of historical prompts.

**Certification pitfall:** batch + prompt caching **combine** — requests sharing a long prefix in the same batch can benefit from caching, accumulating discounts (cache hits are not guaranteed in batch).

---

## Block 9 — Error handling (10 min)

### 9.1 Taxonomy

| Code | Name | Cause | Retry? |
|------|---------|-------|---------|
| 400 | invalid_request_error | Malformed query (roles, JSON…) | ❌ fix code |
| 401 | authentication_error | Invalid key | ❌ |
| 403 | permission_error | Key without access to the resource | ❌ |
| 404 | not_found_error | Model/resource does not exist | ❌ |
| 413 | request_too_large | Request too big | ❌ reduce |
| **429** | **rate_limit_error** | RPM/ITPM/OTPM exceeded | ✅ backoff |
| 500 | api_error | Internal error | ✅ backoff |
| 529 | overloaded_error | Service Overload | ✅ backoff |

(RPM = Requests Per Minute; ITPM/OTPM = Input/Output Tokens Per Minute — the three rate limit counters.)

### 9.2 Exponential backoff with jitter```python
import random, time
import anthropic

def appel_robuste(client, max_retries=5, **kwargs):
    for tentative in range(max_retries):
        try:
            return client.messages.create(**kwargs)
        except anthropic.RateLimitError as e:
            # Priorité au serveur : respecter retry-after s'il est fourni
            retry_after = e.response.headers.get("retry-after")
            if retry_after is not None:
                delai = float(retry_after)
            else:
                delai = min(60, (2 ** tentative)) * random.uniform(0.5, 1.5)  # jitter
            time.sleep(delai)
        except anthropic.APIStatusError as e:
            if e.status_code in (500, 529):
                time.sleep(min(60, 2 ** tentative) * random.uniform(0.5, 1.5))
            else:
                raise    # 4xx ≠ 429 : inutile de réessayer
    raise ExhaustedRetriesError()
```**Three architect points:**
1. **`retry-after` takes precedence over your plan** — the server knows better than you when to try again.
2. **Mandatory jitter** — without hazard, all your workers try again at the same time (“thundering herd”, herd effect).
3. The official SDK already does 2 retries by default — know this behavior before stacking your own layer (risk of multiplicative retries).

Also mention: monitor `anthropic-ratelimit-*-remaining` headers for **proactive** rather than reactive throttling.

---

## Block 10 — Closing (5 min)

- Reminder of the common thread: *parameters → signals (stop_reason) → economy (context, cache, batch) → robustness (errors)*.
- Announce the 3 exercises (query builder, streaming, batch design) and the multiple choice questions.
- Distribute exit tickets.

---

## Exit tickets (5)

To be completed in 3 minutes, picked up at the exit:

1. **stop_reason:** your application receives `stop_reason: "max_tokens"` on a JSON fetch. What happened and what is your code doing? (2 sentences)
2. **Caching:** why does placing a timestamp at the start of the system prompt ruin the caching prompt, and how much extra does it cost (%)?
3. **Streaming:** in which SSE event can we find the final `stop_reason`? (exact name)
4. **Batch:** list two conditions that make the Batches API preferable to synchronous calls, and the associated delivery.
5. **Errors:** we receive a 429 with header `retry-after: 12`. What deadline should you apply and why not use your own backoff formula in this case?

**Express correction:** 1) `max_tokens` ceiling reached, JSON truncated → detect and restart with higher ceiling or continuation. 2) The cache requires an exact prefix; the timestamp changes on each call → 0 hits, but we pay the ~25% writing overhead ⚠. 3) `message_delta`. 4) Massive volume + no latency requirement (24h SLA ⚠) → −50% ⚠. 5) 12 s: the server header takes precedence over any client heuristics.

---

## Trainer appendices

- **Live demo risks:** provide recorded responses (fixtures) in case the network/API key fails. The SSE viewer of the web page works **offline** (simulation).
- **Differentiation:** fast participants → have them implement continuation after `max_tokens` (re-prompt with partial exit in prefill assistant).
- **All values ​​⚠** (rates, TTL, limits, model names) must be revalidated on https://docs.anthropic.com before each session.