# Exercises — Session 10: Governance, ethics & final project

**Program:** Applied AI — Intermediate Level — Instructor: Yann Isola
**Materials:** exercise sheet, session web page (bias simulator, decision tree), final project outline.

---

## Exercise 1 — Bias audit of a prompt (in class, 15 min, pairs)

### Context

Your fictitious company “Nexa Assurances” uses the following prompt to pre-select applications for a customer advisor position. The prompt was written quickly by a manager in a hurry.

### The prompt to audit```
Tu es un assistant de recrutement. Analyse le CV ci-dessous et donne un score de 0 à 10.

Nous cherchons un jeune conseiller dynamique, disponible le soir et le week-end,
avec une excellente présentation. Le candidat idéal a fait ses études dans une
grande école, habite près de nos bureaux de Neuilly, et s'exprime dans un
français parfait, sans accent. Privilégier les profils qui s'intégreront
naturellement à notre équipe soudée de jeunes diplômés.

CV : {cv_text}
Réponds uniquement par : score, puis 2 phrases de justification.
```### Your mission

1. **Detect (7 min).** Identify **at least 6 problematic formulations** in this prompt. For each, indicate:
- the exact term or criterion,
- the **type of bias**: direct (explicitly discriminating criterion) or indirect/proxy (apparently neutral criterion which penalizes a group),
- the group(s) potentially penalized.
2. **Correct (5 min).** Rewrite the prompt in a neutral version: criteria **objective and linked to the position**, structured output (reminder Session 3) with traceable justification by criterion.
3. **Check (3 min).** Paste your corrected version into the **bias simulator** of the web page. Do you get a residual report? Is it justified or is it a false positive from the detector? (Reminder: the tool detects *patterns*, it does not certify anything.)

### Indicative answer (for the trainer)

| Wording | Type | Group penalized |
|---|---|---|
| “young advisor”, “young graduates” | Direct | Senior applicants (age discrimination — illegal) |
| “dynamic” | Indirect | Age/disability stereotype |
| “available in the evenings and weekends” | Indirect | Parents, caregivers (statistical gender impact) — acceptable **only** if it is a real requirement of the position, formulated factually |
| “excellent presentation” | Indirect | Subjective criterion, gateway to appearance bias |
| “big school” | Indirect | Social discrimination; Is the specific diploma necessary for the position? |
| “lives near Neuilly” | Indirect | Socio-economic proxy (postal code) |
| “without accent” | Direct | Discrimination linked to origin — illegal in France |
| “will fit naturally into our team” | Indirect | “Cultural fit” = cloning; penalizes any difference |

**Corrected version expected (example):** criteria = experience in customer relations (years, context), mastery of the tools listed, communication skills assessed on factual elements of the CV; hourly availability mentioned as a factual constraint of the position if real; output JSON (JavaScript Object Notation, structured data format) `{score_par_critère, justification_par_critère, éléments_manquants}`; no reference to age, address, appearance, origin; explicit mention “ignores name, age, address and photo”.

**Debriefing point:** half of the biases in the original prompt are **proxies** — that's the key lesson. The modern bias does not say “no women”; it says “available on weekends” and “near Neuilly”.

---

## Exercise 2 — Privacy impact analysis (in class, 15 min, pairs)

### Context

“MediConseil”, a telemedicine practice, wants to launch an AI assistant which **summarizes patient-doctor exchanges** to pre-fill the medical file. The assistant would send the consultation transcript to an LLM API hosted in the United States.

### Your mission — complete the mini-AIPD

A complete AIPD (Data Protection Impact Assessment) is a legal exercise; Here you do the **engineer version** in 5 questions:

1. **What data goes to the API?** List everything a consultation transcript actually contains (think broadly: identity, health, but what else?).
2.**Classify this data**: direct PII (Personally Identifiable Information) / indirect PII (re-identifying by combination) / sensitive data within the meaning of the GDPR (General Data Protection Regulation) — health is a **special** category, enhanced protection.
3. **Propose an anonymization pipeline**: what technical steps *before* the API call? (PII detection by rules + NER — Named Entity Recognition, recognition of named entities —, pseudonymization by tokens, re-association on return). What remains **impossible** to anonymize without destroying medical usefulness?
4. **Residence & supplier**: what problems does accommodation in the United States pose? What questions should you ask the supplier (EU processing available? retention of prompts? training on your data? contractual commitments?)?
5. **Verdict**: under what conditions can this project be deployed? Who must validate (hint: DPO — Data Protection Officer — obligatory here, and probably the opinion of a health lawyer)?

### Indicative answer (for the trainer)

1. **Data transmitted:** identity of the patient and doctor, health data (symptoms, diagnoses, treatments, history), but also: family situation mentioned in consultation, profession, address, lifestyle habits (alcohol, tobacco), sometimes third parties mentioned (“my daughter has the same thing”).
2. **Classification:** name/date of birth = direct PII; profession + city + rare pathology = re-identifying combination; all medical content = **GDPR special category data** (article 9) — processing in principle prohibited except for regulated exceptions (care), maximum requirements.
3. **Pipeline:** transcription → PII detection (rules for known formats: phone, email, social security number; NER for names/locations) → replacement with stable tokens (`[PATIENT]`, `[MEDECIN]`, `[VILLE]`) → API call → local re-association of tokens in the summary. **Structural limit:** the medical content itself (symptoms, diagnosis) IS the useful data — it cannot be anonymized, only **pseudonymized**. The residual risk remains: this is precisely why the choice of supplier and processing location becomes critical.
4. **Residence:** transfer of health data outside the EU = the most sensitive area of ​​the GDPR ⚠; require an EU processing option ⚠, zero retention or minimum contractual retention, contractual exclusion of data training, and check certifications (health data hosting: in France, HDS certification — Health Data Host ⚠).
5. **Expected verdict:** deployable **only** with: systematic pseudonymization upstream, supplier with EU processing + contractual guarantees (or self-hosted model), formal AIPD validated by the DPO, patient information, and human supervision (the doctor validates each summary — the AI ​​summary never becomes a medical file without proofreading).

**Debrief point:** the correct answer is neither “it's forbidden” nor “let's go” — it's a **list of conditions**. This is exactly what maturity in AI governance is.

---

## Exercise 3 — 🎓 FINAL PROJECT: design a complete AI product

**Format:** by team of 2–3. Ideally prepared in advance (brief distributed one week before) and presented during the session (4 min pitch).“Express workshop” variant: 25 minutes of design session on the canvas, then display gallery.

### The brief

Design **a complete AI product, from the blank sheet to the deployment plan**, on a use case of your choice (ideally taken from your real professional context). You don't code anything: you produce the **design file** that an engineer could execute and that a review committee could approve.

### The outline (one page per section maximum — conciseness is noted)

1. **Problem Statement.** Who hurts where? What is the current situation (baseline)? What is the **measurable** success criterion? Why AI (and not a script, a form, a human)?
2. **Architecture.** Diagram of components: data sources, RAG (Retrieval-Augmented Generation, generation augmented by recovery) yes/no and why, tools/tool ​​calling, single agent or multi-agents, anticipated model(s) with possible routing (economical vs. powerful). Justify each choice by a need, not by fashion.
3. **Prompt strategy.** The main system prompt (written!), context management, expected structured outputs, planned versioning (prompt register).
4. **Evaluation plan.** Your evaluation game: how many cases, where do they come from, what metrics, what threshold for production, evaluation segmented by group if the case lends itself to it (Session 9 + today). How will you evaluate **continuously** after launch?
5. **Deployment plan.** Selected pattern (serverless/container/dedicated — Session 8), latency budget, monthly cost estimate in tokens ⚠ (range + assumptions), observability (what to log), gradual ramp-up strategy (pilot → general).
6. **Risk analysis.** The 5 main risks: technical (hallucination, injection), ethical (bias - on whom?), legal (GDPR, AI Act: **classify your case in a risk category** via the decision tree on the web page), operational, reputational. For each: probability, impact, mitigation.
7. **Governance framework.** Who validates the launch? Who is responsible for each decision in the system? Where is the human in the loop? What escalation path should an incident occur? What documentary artifacts (model card, decision log, incident report)?

### Final project evaluation grid — /100| # | Criterion | Points | What is evaluated |
|---|---|---|---|
| 1 | **Problem & value** | /12 | Real and precise problem (4); measurable success criterion (4); honest “why AI” rationale — including what a non-LLM would do less well (4) |
| 2 | **Architecture** | /15 | Components adapted to the needs, without over-engineering (6); choice RAG/tools/justified agents (5); choice and routing of argued models (4) |
| 3 | **Prompt Strategy** | /12 | Prompt system effectively written and of professional quality (6); defined structured outputs (3); planned versioning/registry (3) |
| 4 | **Evaluation plan** | /15 | Realistic and sourced evaluation game (5); explicit production metrics and threshold (5); continuous post-launch evaluation + segmentation if relevant (5) |
| 5 | **Deployment plan** | /12 | Boss justified (4); latency budget + cost estimate with assumptions ⚠ (4); observability and soft launch (4) |
| 6 | **Risk analysis** | /15 | 5 risks covering technical/ethical/legal (6); correct and well-argued AI Act classification ⚠ (4); concrete and proportionate mitigations (5) |
| 7 | **Governance** | /12 | Clear nominal responsibilities (4); human in the loop positioned in the right place (4); climbing + documentary artifacts (4) |
| 8 | **Pitch quality** | /7 | Time keeping (2); clarity and structure (3); answers to questions (2) |
| | **TOTAL** | **/100** | |

**Interpretation scale:** ≥ 85: file presentable to a real review committee. 70–84: solid, some blind spots. 55–69: the technique holds, the governance or evaluation is decorative — this is the n°1 fault to correct. < 55: go over the framework section by section with the trainer.

**Bonus (up to +5, without exceeding 100):** exceptional transversal consistency — each risk identified in section 6 has a trace in evaluation (4), deployment (5) or governance (7). This is THE mark of a mature file.

**Typical penalty (to be announced before):** −3 if the cost estimate does not state its assumptions; −3 if “bias” is mentioned without saying *who* would be affected; −3 if the AI ​​Act classification is absent or obviously false for an HR/health/credit case.

---

*End of program reminder: keep your three exercises. The bias audit (Ex. 1), the express AIPD (Ex. 2) and the framework (Ex. 3) are three templates that can be directly reused at work — it’s your governance toolbox.*