title: "Applied AI — Session 3: The language of machines"
author: "Yann Isola"
theme: "Applied AI Beginner"
palette: ["#1A2230", "#0F7A6C", "#B4612A", "#E9F6F3", "#F4F7F6"]

Slide 1 — Title

The language of machines

Applied AI — Beginner Level · Session 3

Yann Isola

Today, we open the hood: how does an AI really “read” your messages?

Slide 2 — The shocking question

When you write “Hello!” to an AI…

…is she reading your message?

(Discuss 2 minutes with your neighbor)

Slide 3 — The answer

NO.

She never sees your words.

She sees… numbers.

« Bonjour ! »  →  [ 14085 , 0 ]   →   calculs  →  [ 3992, 618, … ]  →  « Salut ! … »

Illustrative numbers — each model has its own

Slide 4 — On the program

1. The computer understands nothing (and yet…)
2. Tokens: the text in pieces
3. Embeddings: the word map
4. The context window: working memory
5. Hallucinations: when AI invents
6. Temperature: the creativity slider
7. Lab: play with the settings!

Slide 5 — Everything is number

A computer = an ultra-fast calculator

• He only knows numbers
• Text, images, music → everything becomes numbers
• The letter “A”? For the machine: 65 (ASCII code*)

*ASCII = American Standard Code for Information Interchange: an old standard which gives a number to each letter.

Metaphor of the day: AI is a tourist with a number dictionary and an awesome calculator.

Slide 6 — Tokens

Token = a piece of text processed at once

Text	Possible tokens
cat	chat → 1 token
intelligence	intelli + gence → 2 tokens
today	aujourd + ' + hui → 3 tokens
	even the emojis are cut out!

The exact cut varies depending on the model.

Slide 7 — Why cut?

3 reasons

1. Too many words — conjugations, plurals, new words: impossible to number everything
2. LEGO effect — re + faire, dé + mont + er: the bricks recombine
3. Unknown words — “smurfology”? Don't panic: we cut it into known pieces

Slide 8 — Vocabulary

Each token has a fixed number

• The model has a large list: its vocabulary
• Typical size: 30,000 to 200,000 tokens
• “cat” = always the same number (for a given model)
• The model never sees your letters — only the list of numbers

Fun fact: French often costs more tokens than English (the models mostly read English)

Slide 9 — Mini-game: the human tokenizer

Over to you! Cut like an AI:

• ordinateur
• incroyablement
• parapluie
• chatbot
• refaire

In pairs, 5 minutes. There is no ONE right answer — each model has its own breakdown!

Slide 10 — The number problem

Token no. 9846 = “cat”… so what?

A number says nothing of meaning.

How does the machine know that
cat looks like dog
but not in wheelbarrow ?

→ Embeddings!

Slide 11 — Embeddings

Words transformed into coordinates

Like Paris and Lyon on a map of France…
but the map represents MEANING, not geography.

Words close in meaning = neighboring points on the map

     roi •         • reine
   homme •         • femme

  chien • • chat
        • hamster        • pizza
                         • sushi

Slide 12 — The magic of word arithmetic

king − man + woman ≈ queen

The direction “masculine → feminine” is an arrow on the map!

It's up to you to guess:

• Paris − France + Italy ≈
• eat − food + drink ≈

Intuition from old word models (word2vec, 2013). Modern AIs are more complex, but the idea remains: direction = position.

Slide 13 — In reality: not 2 dimensions…

…but hundreds, even thousands!

• Us: unable to imagine a 1000D map
• The machine: it calculates this effortlessly
• To view it, you crunch the map into 2D

Demo: the interactive word map → webpage/index.html

Slide 14 — Pause

Back in 10 minutes

Question to ponder: How many things can you remember at once?

Slide 15 — The context window

The “working memory” of AI

• Everything that the AI keeps “in mind” at once: your conversation, your documents, its responses
• Measured in tokens
• For each message, the AI rereads everything that fits in the window… and nothing else

Slide 16 — The table analogy

You do your homework on a small table: 10 sheets maximum.
An 11th sheet is arriving? An ancient grave.

• On the table = the AI sees it
• Fell off the table = forgotten

Consequence: very long conversation → the AI may forget the beginning!

Slide 17 — Window sizes

Bigger and bigger

• Yesterday: a few thousands of tokens
• Today: up to millions for certain models
• 1 million tokens ≈ several big novels

Tips when the AI loses track:

1. Give a summary of the key points
2. Or start with a clean new conversation

Figures that change very quickly!

Slide 18 — Hallucinations

When AI invents… with confidence

Hallucination = false or invented information, presented as true:

• A book that doesn't exist
• A fabricated quote
• A distorted date
• Precise figures… out of nowhere

Slide 19 — Why does AI invent?

It calculates the next most probable token

• She is trained to produce plausible text, not true text
• “I don’t know” is rarely the most likely outcome…
• Like a student who has not revised but writes with confidence

“AI is a machine for producing plausible, not real. Often the two coincide. Not always.”

Slide 20 — The 4 anti-hallucination reflexes

1. Check elsewhere for important facts (dates, numbers, names, quotes)
2. Be wary of things that are too precise and unverifiable
3. Ask for sources… and check them (AI can also invent sources )
4. Rule of issues: health, money, graded homework → we ALWAYS check

Slide 21 — Temperature

The random cursor

“The sky is…”

Next token	Probability
blue	60%
gray	25%
magnificent	10%
in cheese	0.1%

• Bass: almost always “blue” — wise, predictable
• Average: sometimes “gray”, “magnificent”
• Very high: one day... “in cheese”!

Slide 22 — What temperature for what?

Task	Temperature
translation, summary, calculations	Bass
Emails, writing	Average
Brainstorming, poetry, crazy ideas	High

Consumer chatbots often don't allow you to adjust the temperature... but you can imitate it: "answer in a very classic way" vs. "be bold!"

Slide 23 — Lab: it’s up to you!

In pairs, on the interactive page:

1. Test the same question in cold / medium / hot
2. Find the French word that costs the most tokens
3. Find the most surprising pair of words on the map

Restitution: your favorite discovery in one sentence!

Slide 24 — The 6 sentences to remember

1. The computer calculates with numbers, it does not understand words
2. The text is divided into tokens, each has a number
3. The embeddings = the map of meaning: close in meaning = close in coordinates
4. The context window = working memory, large but limited
5. AI produces plausible, not real → we check!
6. Temperature = the slider between predictable and creative

Slide 25 — Next time…

Session 4

Now that we know how the machine reads…
…we're going to learn how to talk to him to get exactly what we want!

THANKS ! Time for the quiz

Applied AI — Yann Isola