Daily lesson Five minutes. One new idea. Learn a little AI every day, at your own pace. Come back tomorrow, or keep going today.
0 lessons opened Start with Artificial intelligence.
Chapter 1 of 29 What AI is
Saved on this device No account. Clearing this browser's data starts you over.
All lessons 275 lessons · 29 chapters The daily order covers both levels of the course: 139 beginner lessons and 136 deeper ones. In each chapter the beginner lessons come first. Start here shows one level at a time.
FundamentalsYou are here 8 chapters · 84 lessons Deep learningYou are here 3 chapters · 28 lessons Large language modelsYou are here 7 chapters · 70 lessons Building apps and agentsYou are here 5 chapters · 44 lessons Production, evals and safetyYou are here 6 chapters · 49 lessons Prefer the course laid out by part and level? See Start here →
concepts
Machine learning: training on labelled examples adjusts a model, which then turns a new input into a prediction
TRAIN
reduce error
labelled
examples
adjust
model
new
input
saved
model
prediction
then use
concepts
Deep learning: pixels pass through a stack of hidden layers, each building on the last, to reach a prediction
HIDDEN LAYERS 1 TO 3
EACH LAYER BUILDS ON THE LAST
Pixels
Stacked layers
Prediction
raw numbers
edges, shapes,
then objects
“dog”
concepts
Generative AI
Many examples are used to train a generative model, which learns their patterns. A prompt then conditions the model, and it generates a highlighted new sample rather than copying one stored example.
many examples
learn
model
learned
patterns
prompt
generate
new
sample
not a copy
concepts
AI model
Past examples are used to train the model, a structure plus learned numbers saved as a file; the model then runs inference on new input to give a prediction such as 12 mm of rain.
past days
train
model
structure +
learned numbers
infer
12 mm
rain
examples
saved as a file
prediction
concepts Algorithm: a list goes in, exact steps run, the largest number comes out
INPUT
ALGORITHM
OUTPUT
7, 3,
12, 5
Find the largest
5 exact steps
12
Same input, same answer, every time
concepts
Artificial general intelligence (AGI)
One AI system is scored on many kinds of task, such as writing, maths, coding and learning a new skill. A dashed line marks better than half of skilled adults. The highlighted test asks whether the system is above that line on most tasks.
half of skilled adults
One AI
system
write
maths
code
learn
AGI?
most tasks
above line
concepts
Symbolic AI: written facts and an if-then rule are matched, and the rule's then part becomes a new fact
facts
if-then rule
new fact
match
and derive
parent(a, b)
IF … THEN …
grandparent
concepts
Expert systems: an inference engine applies if-then rules from a separate knowledge base to a case's facts and gives advice it can explain
case
facts
inference
engine
advice
+ why
knowledge base
IF … THEN rules
reads rules
concepts
Turing test
A judge exchanges typed messages with two hidden witnesses, A and B. One is a person and one is a machine, and the judge is not told which is which. The highlighted step is the verdict: the interrogator names which witness they think is the human.
Judge
types only
who is who: hidden
Witness A
Witness B
Verdict
which one
is human?
concepts
Model training: a batch is predicted, scored by a loss against its labels, and the weights are updated before the next batch
predict
batch
loss vs
labels
update
weights
next batch
concepts
Training data: examples are split so each one goes into only one of the training, validation or test sets
examples
unseen until the end
Split
one set each
Training set
Validation set
Test set
concepts
Inference: new input goes into a trained model and an answer comes out, while the trained weights are only read on every pass and never updated
new input
trained model
answer
trained weights
set by training
read every pass
never updated
concepts
Parameters: an input passes through the model's parameters, a kernel and a bias, to become an output, while training adjusts those parameter values from example data
training
input
parameters
output
adjusts values
kernel + bias
concepts
Weights: each input is multiplied by a weight to make the output; during training the output is compared with the target through a loss, and the weights are updated
input
× weights
output
loss
update
compare to target
concepts
Features: a car listing's mileage, age and colour become one feature vector of numbers that the model reads
car listing
scale + encode
feature vector
10 numbers
1 weight each
60,000 km
4 years
Blue
[0.3, 0.2,
0,0,1,0,0,0,0,0]
model
concepts
Labels
Three raters label the same photo on their own: two say cat and one says dog. Their answers meet at the highlighted step, which asks whether they agree. Two of three picked cat, so cat becomes the one final label that goes into training with the example.
SAME PHOTO, 3 RATERS
rater 1
rater 2
rater 3
final label
cat
cat
dog
cat
KEY STEP
do they agree?
2 of 3: cat
to training
concepts
Ground truth: a model's prediction is compared with the real answer and counted right or wrong
Model
Ground truth
right
Compare
says: tumour
expert: tumour
counted
same case
concepts
Data labeling: an annotator follows the label guide and attaches the answer "traffic light" to an unlabeled photo
raw example
labeling
labeled example
input only
input + answer
no label
traffic light
Annotator
follows the
label guide
concepts
Feature engineering: a raw row with a price and a colour name passes through encoding rules and becomes one list of numbers
price 450k
colour red
encode
[0.42,
1, 0, 0]
RAW ROW
FEATURE VECTOR
every value becomes a number
concepts
One-hot encoding: the category car becomes 0, 0, 1, 0, with a single 1 in the car position
CATEGORY
ONE POSITION PER CATEGORY
car
0
0
1
0
bike
bus
car
train
one 1, every other slot 0
concepts
Outliers
Nine delivery times pass through the highlighted 1.5 × IQR fence check. The eight values from 28 to 37 minutes land inside the fences, and 95 lands above the 44.25 fence, so it is flagged.
minutes
inside
flagged
28 30 31
32 33 34
35 37 95
28 to 37
95 > 44.25
fences
1.5 × IQR
concepts
Data augmentation: one photo labelled cat goes through a random change and comes out as flipped, cropped and brighter copies that all keep the label cat
cat photo
random change
flipped cat
cropped cat
brighter cat
ONE EXAMPLE
label stays cat
concepts
Missing data: a table with blank cells leads to the key question of why they are missing, which decides whether to drop the rows, fill in a guess or use a model that accepts gaps
blank cells
why missing?
drop rows
fill a guess
keep gaps
DATASET
pick the safe one
concepts
Imbalanced data: 990 negatives and 10 positives are downsampled to 99 and 10, then each kept negative is upweighted by 10
Each small square stands for about 10 examples: 99 dark squares of negatives and one square of positives, then 10 dark squares and the same one positive square after keeping 1 negative in 10.
real data
downsample
upweight
99 to 1
keep 1 in 10
ratio restored
990
10
99
10
neg
pos
neg
pos
×10 loss
on each negative
99 → 990
concepts
Synthetic data: a model is fit on private real data, new values are simulated from that model, and the result is stand-in data that reuses some of its patterns
real data
fit model
simulate
new data
kept private
similar
concepts
Data leakage: every column goes through a leak check that asks whether it is known at prediction time; keeping the hospital name, recorded after diagnosis, makes the score look great, while dropping it gives an honest score
columns
after diagnosis
hospital name kept
hospital name dropped
age
vital signs
hospital name
leak check
known at
prediction?
looks great
honest score
concepts
Feature importance: shuffle one feature's column, measure how far the model's score drops, and rank the features by that drop
held-out rows
shuffle A
rescore
rank
importance
A
B
C
score drop
0.90 − 0.72
= 0.18
A
B
C
concepts
Vectors: a word or image is stored as an ordered list of numbers, such as (3, 1), which you can also picture as an arrow going 3 steps along and 1 step up
word or
image
(3, 1)
arrow
3 along, 1 up
THING
LIST OF NUMBERS
SAME VECTOR
concepts
Matrices: a weight matrix W with 2 rows and 3 columns takes an input of 3 numbers, 2, 1 and 3, and gives an output of 2 numbers, 8 and 7, one output per row
2, 1, 3
matrix W
8, 7
2 rows x 3 cols
3 NUMBERS IN
2 NUMBERS OUT
each output = one row times the input
concepts
Dot product
Illustrative numbers. Vector a equals (3, minus 1, 2) and vector b equals (1, 4, 2) go into the highlighted step, which multiplies matching positions and adds them: 3 minus 4 plus 4. The result is one number, 3, which is above 0, so the vectors point broadly the same way.
vector a
vector b
one number
(3, −1, 2)
(1, 4, 2)
above 0
3
MULTIPLY PAIRS
then add
3 − 4 + 4
concepts
Cosine similarity: a = (2, 1) and b = (4, 2) point the same way; dividing their dot product by both lengths gives a score of 1.00
two vectors
divide by lengths
score
b twice as long
10 ÷ 10
same
direction
b = (4, 2)
a = (2, 1)
a · b
|a| × |b|
1.00
concepts
item "cat"
embedding model
[0.12, -0.48, 0.91]
vector: a list of numbers
similar meaning, nearby
concepts
Linear algebra for machine learning: a batch matrix X, 32 by 4, times weights W plus b, 4 by 3, gives outputs H, 32 by 3, in one product
batch X
32 × 4
layer
one product
outputs H
32 × 3
× W + b
W is 4 × 3
concepts
Probability: 30 of 100 emails are spam, but keeping only the 25 with "free" gives 20 of 25, a probability of 0.80, which a 0.5 cut-off flags as spam
ALL EMAILS
DECIDE
HAS "FREE"
30 of 100 spam
20 of 25 spam
0.80 > 0.5
P = 0.30
P = 0.80
spam
concepts
Statistics for machine learning: a test score of 85% becomes an honest range of 81.5% to 88.5% by adding a margin of error either side, and fewer test examples make the range wider
score 85%
± error
81.5–88.5%
ONE TEST SET
KEY STEP
HONEST RANGE
fewer test examples = wider range
concepts
Softmax: three scores, 1, 2 and 0, go into softmax and come out as three shares, 0.24, 0.67 and 0.09, that add up to 1
1 2 0
softmax
.24 .67 .09
SCORES
SHARES
add up to 1
concepts
Loss function: a prediction of 0.70 and a target of 1.00 enter a squared-error loss, (ŷ − y)², which scores the miss as one number, 0.09, for the optimizer to reduce
prediction
target
loss
loss
function
ŷ = 0.70
y = 1.00
0.09
(ŷ − y)²
0.30² = 0.09
to optimizer
concepts
Stochastic gradient descent: one example is picked at random from the training set, and its gradient alone sets the weight step before the next draw
training set
one example
weight step
random pick
w - gain
× its gradient
next draw
concepts
Bayesian inference: a flat Beta(1, 1) prior plus 3 clicks and 7 no clicks updates to a Beta(4, 8) posterior
PRIOR
DATA
POSTERIOR
3 clicks
7 no clicks
Beta(1, 1)
Beta(4, 8)
concepts
Gradient descent: measure the slope of the loss at the current weight, step against it scaled by the learning rate, and repeat from the new, lower-loss weight
measure slope
step
new weight
g = −4
w = 1.0
w − lr × g
1.0 − 0.1×(−4)
w = 1.4
loss 2.56
repeat
concepts
Supervised learning
Labelled examples, whose highlighted label column holds the answers spam, not spam and not spam, train a model, which then predicts the label spam for a new email that has none.
features
labelled examples
predicted label
label
spam
not spam
not spam
model
learns
new email
spam
concepts
Unsupervised learning
Illustrative points with no labels go into k-means. The key step assigns each point to its nearest centre. Out come three unnamed groups, each with a centre marked by a plus sign.
NO LABELS
K-MEANS
3 GROUPS
assign
nearest centre
No answers go in; points are grouped by distance alone.
concepts
Reinforcement learning: the agent's policy sends an action to the environment, which returns the next state and a reward
agent
policy
environment
game or board
action
next state
reward
repeat for the most total reward
concepts
Self-supervised learning
A raw sentence, the cat sat on the mat, passes through the highlighted hide step, which masks the word mat. The model sees the gap and predicts it, and a dashed line shows the hidden word becoming the label it is checked against.
raw text
hide a word
model
the cat
sat on
the mat
sees gap
guess: mat
the cat
sat on
the [MASK]
hidden word = label
concepts
Few-shot learning: three labelled examples and an unlabelled query go into a pretrained model, which compares and adapts to predict the query label
3 labelled examples
pretrained model
prediction
new input, no label
A1 → maple
A2 → maple
B1 → oak
Q → ?
compare
and adapt
Q → maple
concepts
Semi-supervised learning
A model trained on 50 labelled examples guesses labels for 519 unlabelled ones. The highlighted step keeps only guesses with confidence above 0.7. Those become pseudo-labels, which join the labelled set so the model trains again.
labelled
unlabelled
pseudo-labels
50 examples
519 examples
new answers
train again
guess
KEEP A GUESS IF
confidence
is above 0.7
concepts
Transfer learning: general layers trained on a large source dataset are copied to the target task, where a new head is added and trained on a small dataset
general
layers
copied
layers
new
head
pretrained
freeze or tune
large dataset
small dataset
target outputs
SOURCE
copy
TARGET
concepts
task A task B task C update without erase
current model
concepts
Q-learning: the agent takes one action, adds the reward to the best score at the next state to make a target, and moves that action's score toward the target
take one
action
reward +
best next
move its
score
in one state
the target
toward target
concepts
Multi-armed bandits: pick one arm, see only that arm's reward, update your guesses, then repeat, balancing exploring and exploiting
pick an arm
see only its reward
update guess
repeat: explore or exploit?
ONE ROUND
concepts
Regression
A house's features pass through the highlighted learned function, which places the answer at one point on a number line: an estimate of 412 thousand dollars.
features
estimate
size 1,850
age 12
location B
$300k
$500k
f(x)
learned
$412k
concepts
Linear regression
Scattered data points pass through the highlighted least-squares step, which picks the straight line y′ = b + wx with the smallest squared vertical gaps to the points.
data points
fitted line
weight → mpg
y′ = b + wx
least
squares
concepts
Logistic regression: a score of 0.70 passes through the sigmoid to a probability of 0.67, which is above 0.5, so the class is spam
score z
w·x + b
squash
any z to 0–1
probability
> 0.5: spam
0.70
sigmoid
0.5
0.67
concepts
k-nearest neighbours: a new point finds its 3 closest stored examples, and their majority vote, 2 A against 1 B, gives it the label A
new point
3 nearest
label: A
closest stored examples
A, A, B
majority vote
2 A beat 1 B
concepts
Naive Bayes
The words of a new email, prize and meeting, go into the highlighted multiply step, which multiplies each class's prior by each word's likelihood; spam scores 0.004 and not spam 0.0006, and the top score wins.
new email
prize
meeting
spam
0.004
not spam
0.0006
multiply
prior × words
top score wins
concepts
Ensemble learning: three models each right on 3 of 5 examples, combined by majority vote, are right on 4 of 5
Model A
Model B
Model C
4 of 5
3 of 5 right
3 of 5 right
3 of 5 right
combine
ensemble
right
Majority
vote
concepts
Support vector machines: the SVM line sits in the middle of the widest margin between two classes, and the support vectors on its edges decide where it goes
class 1
class 2
widest margin
support vectors
SVM line
concepts
Gradient boosting: each small tree is fitted to the current errors, half its output is added, and the loop repeats
errors
key step
update
Residuals
−80
−20
+100
Small tree
fits errors
Add 0.5 × tree
to prediction
errors shrink
repeat on new errors
concepts
Genetic algorithms: a population of candidates, keep the fitter ones, mix pairs of them, add a random flip, then repeat for the next generation
population
keep fitter
mix pairs
random flip
repeat, generation after generation
concepts
Hidden Markov models: ice-cream counts 3, 1, 3 are decoded into the hidden weather hot, cold, hot
ice creams
observed
decode
weather
hidden
3
1
3
hot
cold
hot
Viterbi
keep best path
concepts
Monte Carlo methods: throw 10 random dots in a square, count that 8 land inside the quarter circle, and multiply the share by 4 to estimate pi as 3.2
10 random dots
8 land inside
4 × 8/10 = 3.2
SQUARE
QUARTER CIRCLE
ESTIMATE OF π
more dots, closer to π
concepts
Clustering: k-means assigns unlabeled points to their nearest centroid, giving cluster IDs A and B
unlabeled points
k = 2
cluster IDs
repeat until stable
A
B
assign
nearest
concepts
K-means: unlabelled points each join their nearest centre, each centre moves to the mean of its group, and the loop repeats until the centres stop moving, leaving k groups
unlabelled
points
join
nearest
centre
move
centre to
group mean
k groups
repeat
until centres stop
concepts
Recommender systems: a catalogue of millions is cut to hundreds of candidates, each candidate is scored by how likely this person wants it, and about ten of the best appear on screen
catalogue
find
score
show
millions
hundreds
each one
about 10
concepts
Anomaly detection
Illustrative points. Two random cuts leave the odd point on its own, so its path is short: 2 cuts. A short path gives a score near 1, and the point becomes an alert.
RANDOM CUTS
PATH LENGTH
RESULT
1
2
2 cuts
short path
alert
score near 1
Odd points are cut off early; normal ones take many cuts.
concepts
Principal component analysis: points described by two features are projected onto PC1, the direction of most spread, leaving one score per point that keeps about 90% of the spread
2 features
find PC1
1 feature
x1, x2 per point
keeps ~90%
PC1 score
most spread
concepts
Collaborative filtering: the dot product of user 2's vector and film B's vector fills a blank rating with 4.2
known ratings
dot product
filled
one learned vector per user and per film
A B C
U1 U2 U3
4 4 2
4 ? 2
? 3 5
U2 · B
1.8×2.0+0.6×1.0
= 4.2
4.2
U2, B
concepts
Time series forecasting: past months feed a forecast of the next months, the forecast is compared with the real values, and the gaps are averaged into an error score
past
months
forecast
next ones
compare
with real
average
error
guess only from values before the cut
concepts
Overfitting
Training data points pass through the highlighted memorize step, where a curve bends through every point, including noise. The result is correct on seen data but wrong on new data.
training data
seen data
new data
correct
wrong
memorize
concepts
Validation set: three candidate models are scored on the same held-out validation examples, the best score is kept, and only that frozen choice goes on to the test set
candidates
scores
model A
model B
model C
A 0.82
B 0.89
C 0.85
keep
validation
set
held-out examples
test set
frozen
concepts
Test set: train and validation data shape the chosen model, then the sealed test set is opened once for a final score
split once
develop and tune
TRAIN
VALIDATE
TEST
kept sealed
Chosen model
Score
then frozen
reported
Final test
opened once
concepts
Epoch: eight training examples are split into four batches, each batch is one update step, and after the last batch the epoch counter moves from zero to one
8 examples
4 batches
epoch 0→1
the whole set
one update each
every one used
concepts
Batch size: four examples, one averaged gradient, one weight update
BATCH OF 4
Ex 1
Ex 2
Ex 3
Ex 4
Average
g1…g4
1 weight
update
one direction
concepts
Learning rate: one gradient of 2.5 times rate 1e-3 gives a step of 0.0025, times rate 0.01 gives a ten times longer step of 0.025
gradient
scale
rate 1e-3
rate 0.01
same gradient, 10x the rate, 10x the step
2.5
× rate
then subtract
step 0.0025
step 0.025
concepts
Hyperparameters: people set the learning rate and batch size before training, and training then learns the model's weights
hyperparameters
training
parameters
set by people
on the data
learned
You choose
lr 0.01
batch 32
Train
Weights
0.42 −1.3
0.07 2.1
concepts
Underfitting
Training data points follow a curved trend. The highlighted step fits one flat straight line that misses the bend, so the model is wrong on the data it has already seen and on new data.
training data
seen data
new data
wrong
wrong
too simple
concepts
Regularization
A set of weights with one very large weight passes through the highlighted penalty step, which charges for squared weights. The result is a set of smaller weights.
weights
smaller
λ·Σw²
penalty
concepts
Accuracy: 9 correct answers out of 10 gives 90% accuracy
10 answers
9 right, 1 wrong
divide
accuracy
9 ÷ 10
correct
all answers
90%
concepts
Precision and recall: the same 15 true positives are divided by 20 flagged emails for precision and by 25 real spam emails for recall
15 true
positives
÷ 20 flagged
÷ 25 real spam
precision 0.75
recall 0.60
concepts
Confusion matrix: an email predicted spam that is really not spam is counted in the false positive cell of a two-by-two table
one email, e2
said yes
said no
real spam
real not
FP = false alarm
said: spam
really: not spam
TP
FN
TN
FP
2
1
2
1
concepts
Bias-variance tradeoff: the same training samples feed a rigid model that misses consistently and a flexible model whose guesses scatter, and validation error decides between them
training
samples
rigid model
flexible model
compare
val. error
consistent miss
wide scatter
concepts
F1 score: precision 0.9 and recall 0.1 go through the harmonic mean and give an F1 of 0.18, near the weaker score
precision
0.9
recall
0.1
0.18
harmonic mean
2PR ÷ (P + R)
F1
plain average would say 0.50
concepts
ROC curve: model scores pass a threshold, which splits them into real positives caught and false alarms; each threshold gives one point on the curve
scores
threshold
caught
false alarms
MODEL
UP THE CHART
ACROSS THE CHART
slide it: one point per setting
concepts
Neural networks
Three inputs feed one neuron, which weighs and adds them, then passes the total through the highlighted bend step, a ReLU activation, and sends the result to every neuron in the next layer.
x1
x2
x3
inputs
next layer
weigh
+ add
bend
ReLU
concepts
Perceptron: two input features are each multiplied by a weight and added into one score, and a score above zero gives a yes, otherwise a no
feature 1
feature 2
weighted
sum
above 0?
yes or no
each x weight
the decision
concepts
Multilayer perceptron: three input features pass through a hidden layer that adds a nonlinear ReLU step, then an output layer gives the chance of each class
3 features
hidden
+ ReLU
class
chances
INPUT
BENDS THE LINE
OUTPUT
layers stack to fit curvy patterns
concepts
Activation function: a neuron's weighted sum z passes through a non-linear function f, here ReLU, and the result a goes to the next layer
sum z
next layer
activation f
w·x + b
receives a
ReLU: max(0, z)
concepts
ReLU: the inputs minus 3 and 4 pass through a max(0, x) gate; the negative input comes out as 0 and the positive input comes out unchanged as 4
-3
4
max(0, x)
0
4
INPUT
ReLU
negative → 0
positive: same
concepts
Backpropagation: the loss slope is passed back through the chain rule until every weight has a gradient
FORWARD PASS
BACKWARD PASS, RIGHT TO LEFT
Weights
Chain rule
Loss
one gradient
per weight
slope in
× local slope
one number
slope first
concepts
Dropout: during training the values 2, 4, 1, 3 pass through dropout, which zeroes some at random and scales the rest to 4, 0, 2, 0; during evaluation the same values pass through unchanged
2 4 1 3
dropout
4 0 2 0
2 4 1 3
2 4 1 3
TRAINING
random zeros, rest scaled
EVALUATION: all kept
concepts
Residual connection: the input x goes through a layer that makes F(x), a skip path carries x around the layer, and the two are added as F(x) + x
x
layer F(x)
F(x)+x
out
skip path: copy of x
add them
concepts
Layer normalization: the numbers from one example are measured for their mean and spread, re-centred and rescaled, then adjusted by a learned scale and bias
one
example
find mean
+ spread
re-centre
+ rescale
scale
+ bias
no batch needed
LEARNED
concepts
Convolutional neural networks: a small window of the image grid is multiplied and added with one learned filter, giving one score on a feature map; the same filter slides to every spot
3x3 window
same filter
one score
IMAGE GRID
MULTIPLY AND ADD
FEATURE MAP
then the same filter slides to the next spot
concepts
Vision transformer: a photo is cut into small patches, the patches are lined up as a sequence, a transformer reads that sequence and answers sailboat
photo
patches
transformer
sailboat
IMAGE
cut up, in a row
reads the row
concepts
Recurrent neural networks: at each step the same RNN cell combines the old hidden state with the new word, such as falls, and writes an updated hidden state for the next step
old state
word: falls
RNN cell
new state
same rule at every step
for next step
concepts
LSTM: gates keep part of the old memory and add part of the new input, giving a new memory for the next step
old memory
new input
keep + add
new memory
gates choose what stays
concepts
GRU: the previous hidden state and a new candidate both feed the update gate, which blends them into the next hidden state
old state
new candidate
update gate
next state
blends old and new
concepts
Residual block: the input x goes through layers that compute F(x), while a shortcut carries x around them, and the two are added to give F(x) + x
input x
layers F(x)
F(x) + x
shortcut carries x
add the two
concepts
U-Net: an image is zoomed out for context and zoomed back in, a skip path carries fine detail from the zoom-out side across, and the two are combined to draw a pixel mask
image
zoom out
zoom in
combine
pixel mask
skip:
fine
detail
context
concepts
Autoencoders: an encoder squeezes a six-value input into a small two-value code, and a decoder rebuilds a six-value copy of the input from that code
input
encoder
code
decoder
copy
6 values
2 values
6 values
concepts
Graph neural networks: three neighbor nodes send messages, the messages are combined, and the target node updates its hidden state from them
neighbor
neighbor
neighbor
combine
new state
MESSAGES IN
TARGET NODE
concepts
Text-to-image models: a text prompt goes through a text encoder, its meaning guides the highlighted image generator, and the generator produces an image
prompt
text encoder
generator
image
words
reads the text
guided by text
picture
concepts
Text-to-speech models: written text goes into a predictor that makes a spectrogram, and a vocoder turns that spectrogram into audio you can hear
text
predictor
spectrogram
vocoder
audio
concepts
Speech-to-text models: a recording of spoken words goes into the model, which writes out the transcript as written characters
speech audio
STT model
characters
spoken words
listens, then spells
the transcript
concepts
Diffusion models: generation starts from pure noise, a learned step removes a little noise at a time, repeated many times, until a clean picture remains
pure noise
remove a little
noise
clean
picture
START
repeat many steps
RESULT
concepts
GANs: a generator turns noise into a made sample, a discriminator compares it with a real example, and its score is sent back to help the generator improve
noise
generator
real example
discriminator
made
score helps it improve
concepts
Image editing models: a picture you already have and a written instruction go into the editing model, which returns the same picture with only the asked-for change
your
photo
written
instruction
editing
model
changed
photo
rest kept
concepts
Text-to-video models: a text prompt goes into a base video model, later stages refine the clip, and out comes a high-definition video
prompt
base model
refine
HD video
TEXT
makes the first clip
sharper
VIDEO
concepts
Music generation: a text prompt guides a music model, which writes the song as compact music codes that are then turned into an audio clip
text
prompt
music
model
music
codes
audio
clip
compact form
the sound
concepts
Variational autoencoder: an encoder turns an input into a mean and spread, a code z is sampled as mu plus sigma times noise, and a decoder turns z back into a rebuilt input
input
encoder
sample z
decoder
rebuilt input
mu + sigma × noise
mean and spread
concepts
Neural radiance fields: a 3D point and viewing direction go into a neural field, which returns density and color, and volume rendering combines them into one pixel
3D point
+ direction
neural
field
density
+ color
one pixel
ON A CAMERA RAY
volume rendering
concepts
Tokens: a tokenizer splits the text "unbelievable!" into smaller pieces, and each piece becomes one number the model reads
unbelievable!
tokenizer
un
believ
able
!
TEXT
one number each
concepts
Tokenization: a tokenizer cuts the text "unwatched cats" into the pieces un, watch, ed, cat and s, and maps each piece to an ID number the model reads
unwatched
cats
tokenizer
un|watch|ed|cat|s
IDs for model
TEXT
PIECES
cut, then number
concepts
Vocabulary: the word "unwatched" is split into the pieces un, watch and ed, the vocabulary looks up each piece, and out come the numbers 41, 308 and 19 the model reads
unwatched
un watch ed
vocabulary
41 308 19
PIECES
LOOK UP
IDS
concepts
Context window: instructions, messages and tool results all go into one limited context window, and the model's answer must fit in that same window
instructions
messages
tool results
context
window
answer
limited space
counts too
concepts
Max tokens: a prompt goes to a model with a limit of four tokens, the model writes up to four tokens, and then generation stops at the ceiling
prompt
model
max = 4
1 2 3 4
stop
NEW TOKENS
the answer can end sooner, never later
concepts
Knowledge cutoff: a question is checked against the model's cutoff date; older facts can come from training, newer facts need a current source
question
after cutoff?
use training
check a source
NO: older fact
YES: newer fact
concepts
Byte-pair encoding: the letters l o w are checked for the most common neighbouring pair, l and o are merged into one piece, giving lo w, and the step repeats to make low
l o w
top pair: l o
merge → lo
lo w
LETTERS
NEW PIECE
repeat: lo + w → low
concepts
Long context: in a long document each word looks only at nearby words, while a few selected words see everything, so attention cost grows linearly
long
document
nearby
words
few words
see all
linear
cost
local window
global
concepts
Lost in the middle: a model uses a fact at the start or end of a long input better than the same fact in the middle
start
middle
end
model
LONG INPUT
used well
often missed
used well
concepts
Large language model: the model scores every possible next token for the prompt, the likeliest is appended, and the loop repeats
prompt
LLM
the cat sat
next token?
on 62%
in 21%
at 9%
add "on", repeat
concepts
Next-token prediction: the words so far, "the river rose after", go into the model, which picks the next token, "rain", adds it to the text and repeats
the river rose after
model
rain
WORDS SO FAR
NEXT TOKEN
add it, then predict again
concepts
Logits: a model gives one raw score each for cat, dog and fox, and softmax turns those scores into probabilities that add up to one
cat score
dog score
fox score
softmax
chances
add to 1
RAW LOGITS
PROBABILITIES
concepts
Greedy decoding: the text "The cat" gets three scored next words, sat 0.52, slept 0.31 and ran 0.17, and only the top one, sat, is kept to make "The cat sat"
The cat
sat 0.52
slept 0.31
ran 0.17
The cat sat
TEXT SO FAR
next-word chances
keep only the top
concepts
Top-k sampling: from all ranked next-token candidates, keep only the k most likely, rescale their chances, then pick one at random
all next
keep top k
rescale
pick one
guesses
rest dropped
new shares
at random
concepts
Stop sequences: the model writes "Final answer" then the stop string END, a stop check spots END and stops, and only "Final answer" is returned
Final answer
<END>
stop check
Final answer
MODEL WRITES
YOU GET
sees <END>, stops
concepts
Summarization: a long text goes into a summarizer, which returns a short version that keeps the key points
long text
many pages
summarizer
key points
INPUT
shorter version
concepts
Autoregressive models: the words so far, "rain fell", are used to predict the next word, "softly", which is added to the sequence before the next prediction
rain fell
predict next
softly
WORDS SO FAR
NEXT WORD
add it to the sequence, repeat
concepts
Temperature: the model's logits are divided by temperature T before softmax; a low T gives sharper odds and a high T gives more even odds
logits
divide by T
softmax
low T
sharper
high T
more even
KEY STEP
same logits, new odds
concepts
Top-p sampling: the next-word choices are ranked by probability, the top ones are kept until their probabilities add up to at least p, and one of the kept words is picked at random
ranked
choices
keep top
until sum ≥ p
pick one
at random
next words
the nucleus
next token
concepts
Beam search: four scored candidates, The cat sat, The cat ran, The dog sat and The dog ran, go into a step that keeps only the best two, which move on to the next step
The cat sat −0.7
The cat ran −1.4
The dog sat −1.6
The dog ran −1.0
keep best 2
next
drop the rest
expand again
concepts
Test-time compute: one question gets three separate attempts, and the model picks the answer most attempts agree on
question
attempt 1
attempt 2
attempt 3
pick the
answer most
agree on
more tries = more compute at answer time
concepts
Emergent abilities: the same answers scored by exact match look like a sudden jump, while partial credit shows steady improvement
same answers
tokens right
small 3/10
medium 7/10
large 10/10
exact match
0 → 0 → 1
partial credit
0.3 → 0.7 → 1.0
concepts
Transformers
The tokens The, bank and closed all feed the highlighted self-attention step. The query for bank is compared with every token and gives illustrative weights of 0.10, 0.30 and 0.60, the largest going to closed. The weighted mix becomes a context-aware vector for bank.
TOKENS
The
bank
closed
bank
in context
SELF-ATTENTION
weights for “bank”
0.10
0.30
0.60
concepts
Attention: a query is compared with every stored piece of information, the scores become weights, and the weighted values are mixed into one output
query
keys
weigh each
values
mixed
what I need
one output
concepts
Self-attention: the word bank compares itself with other words in the same sentence, such as river, and mixes their information into a new, updated meaning for bank
river
bank
compare
and mix
new bank
same sentence
meaning updated
concepts
Embedding layer: an integer index such as ID 3 picks one row of a learned table, and that row comes out as a dense vector
ID 3
pick row 3
its vector
INTEGER
LEARNED TABLE
DENSE VECTOR
one row per ID
concepts
Decoder-only models: the words so far, "The river rose after", go into a model that can only look left at earlier words; it predicts the next word, "rain", which is added to the text before the next step
The river rose after
look left only
rain
WORDS SO FAR
NEXT
add it, then repeat
concepts
Encoder-only models: words before and words after a position both feed one encoder, which gives every word a representation built from context on both sides
words before
words after
encoder
meaning of each word
reads both sides at once
concepts
Encoder-decoder: an encoder turns the source sentence "The river rose" into notes, and a decoder reads those notes plus its earlier words to write the next word, "rivière"
The river rose
encoder
decoder
earlier words
rivière
SOURCE
makes notes
next word
concepts
Multi-head attention: one input sequence is sent to three attention heads that run side by side, their outputs are joined, and one combined output comes out
sequence
head 1
head 2
head 3
join
output
run side by side
concepts
Positional encoding: the word "bank" and its position 0 are added together, so attention receives a word that also knows where it sits in the sentence
word: bank
position: 0
add
attention
same size, summed
knows the order
concepts
Rotary position embeddings: a query at position m and a key at position n are each turned by an angle tied to their position, and comparing them reflects the gap m minus n
query
at m
turn by
angle m
key
at n
turn by
angle n
compare
gap m-n
concepts
Mixture of experts: a gate looks at each token and sends it to only a few chosen experts, while the other experts are skipped
token
gate
expert 1
expert 2
expert 3
picks a few
only chosen experts run
concepts
Grouped-query attention: three query heads in one group all read from one shared key head and value head
query head 1
query head 2
query head 3
shared key
+ value head
one group
one copy, read by all
concepts
Sparse attention: queries and keys pass through a sparse mask, so attention is computed only for the selected query-key pairs instead of every pair
queries
+ keys
sparse
mask
selected
pairs only
every token
picks the links
not every pair
concepts
Sparsity: most weights are zero, so the encode step keeps only the nonzero values and their positions
3
0
0
5
0
2
0
0
weights
encode
nonzeros
values
3 5 2
positions
0 3 5
5 of 8 weights are zero, so only 3 are stored
concepts
FlashAttention: query, key and value data in big GPU memory are split into blocks, each block is loaded into fast on-chip memory and attended to there, looping block by block, and only the exact output is written back
Q, K, V
big memory
load one
block
attend in
fast mem
exact
output
next block, keep running totals
no full score table
concepts
Linear attention: keys and values are first combined into one fixed-size summary, and each query reads that summary to make the output instead of comparing itself with every key
keys + values
summary
query
output
fixed size
reads summary
concepts
State space models: each new input and the old running summary combine into a new summary, and the output is read from it
new input
new summary
old summary
output
UPDATE EACH STEP
concepts
Mamba: each incoming token decides what to keep or forget, updating a running memory that moves forward one step at a time and produces the output
token
keep/forget
memory
output
token sets the valve
state
memory moves forward one step at a time
concepts
Foundation models: one model is trained on broad data, then adapted by prompting or fine-tuning for many different tasks
broad data
foundation
model
task 1
task 2
task 3
TRAIN ONCE
adapt by prompt or fine-tuning
concepts
Base models: a pretrained base model can continue text directly, or get extra tuning on instructions to become an instruct model
base model
continue text
extra tuning
instruct
model
PRETRAINED
use it as is
on instructions
concepts
Instruct models: a pretrained base model is fine-tuned on examples that pair a request with a good reply, which turns it into an instruct model that follows instructions
base model
fine-tune
instruct
model
request + reply
PRETRAINED
FOLLOWS ASKS
EXAMPLES TEACH
concepts
Open-weight models: a publisher releases the model's weights file, you download it, check its license, and run the model on your own computer
publisher
weights file
check license
run locally
anyone can download
on your computer
concepts
Closed models: your app sends a request through the provider's API and gets an output back, while the model weights stay inside the provider
your app
API
weights
REQUEST
STAYS WITH PROVIDER
output comes back
concepts
Small language models: a phone's limited memory and compute set the budget, so the model is built with fewer than one billion parameters and can run on the device
phone budget
small model
on device
memory, compute
under 1B params
mobile use
concepts
Code models: a programming problem goes into a code model, which writes many candidate programs; a filter checks how they behave and keeps a few to submit
problem
code model
many programs
check behaviour
few picks
concepts
Agentic models: a goal goes to the model, which acts, observes the result and loops back to adjust, repeating until the task is done
goal
model
act
observe
repeat until done
concepts
Frontier models: an advanced model is tested for risks, and if it reaches a set threshold, safeguards are added; if not, it keeps being watched
advanced
model
test for
risks
reached
threshold?
keep
watching
add
safeguards
yes
no
concepts
Reasoning models: a hard prompt goes through a highlighted scratchpad of reasoning tokens, limited by a token budget, before the final answer is shown
hard
prompt
reasoning
tokens
final
answer
token budget
SCRATCHPAD FIRST
concepts
Chat templates: system, user and assistant messages go through the model's chat template, which turns them into one formatted prompt that ends where the model's reply should start
system
user
assistant
template
one prompt
MESSAGES
adds role markers
then reply starts
concepts
Multimodal models: text, an image and audio all go into one multimodal model, which uses them together to write one text answer
text
image
audio
multimodal
answer
many input types
one model
text reply
concepts
Vision-language models: an image and a written question both go into one vision-language model, which answers in text
image
question
vision-
language model
text answer
sees and reads
writes
concepts
text audio image one shared model
mixed output
concepts
observation action latent state
predict next state
concepts
sensors + goal
robot policy
act
concepts
camera instruction VLA policy
action
concepts
Pretraining: a model first learns from lots of text, then the pretrained model is fine-tuned with labelled task data into a task model
lots of
text
pretrain
pretrained
model
labelled
task data
task
model
fine-tune
concepts
Fine-tuning: a pretrained model plus a small dataset go through gentle extra training and come out as a model for one task
pretrained
small data
fine-tune
task model
low learning rate
concepts
Instruction following: a request with three rules, an answer, and a yes-or-no check for each rule, scoring 2 of 3 met
request
answer
one check per rule
Summarize
3 bullets
a: 3 bullets
b: name risks
c: no jargon
4 risks
named
uses EBITDA
a YES
b YES
c NO
2 of 3 met
Each rule is scored on its own; the jargon rule failed.
concepts
Instruction tuning
Summarize, classify and answer tasks, each written as instructions, are pooled in the highlighted mixing step with balanced shares, then used to fine-tune one pretrained model.
task prompts
summarize
classify
answer
tune one
model
mix tasks
balance
shares
concepts
Supervised fine-tuning: a prompt paired with a good answer is used to update a pretrained model, which becomes a tuned model
prompt
good answer
update
tuned model
TRAINING PAIR
weights change
new weights
concepts
Constitutional AI
A written constitution of principles feeds the highlighted step where the model critiques its draft answer against a principle and revises it, producing a revised answer used as fine-tuning data.
constitution
written principles
draft
may break
a rule
revised
answer
SFT data
critique
+ revise
per principle
concepts
Parameter-efficient fine-tuning: an input runs through a large frozen base and a small trained adapter, and their outputs are added
input
frozen base
adapter
trained
add
Wx+BAx
W · no gradients
building with ai
LoRA: a large frozen weight matrix plus a small trainable low-rank product B times A gives an adapted matrix of the same size
frozen W
d × d
B · A
low rank r
W + BA
same size
not trained
trained part
merged
building with ai
QLoRA: pretrained weights are quantized to a frozen 4-bit base, and a small LoRA adapter is added and trained
weights
4-bit base
LoRA
pretrained
frozen
trains
quantize
add
concepts
Distillation
A large teacher gives soft probabilities for cat, dog and fox. The highlighted match step copies that spread into a smaller student.
teacher
student
cat
dog
fox
soft
small
match
cat
dog
fox
copy spread
concepts
RLHF (reinforcement learning from human feedback)
A human ranks answer A above answer B. The highlighted reward model learns to predict those preferences and scores answers. Reinforcement learning then updates the policy to raise that predicted reward.
human ranks
RL update
A > B
raise reward
REWARD MODEL
learns A > B
scores answers
concepts
RLAIF: an AI judge applies a written constitution to a pair of answers and outputs the preference label a person would normally give
Answer pair
Constitution
AI judge
Label
A · B
written rules
applies rules
picks better
A beats B
concepts
DPO: a chosen and rejected answer pair feeds one loss that compares the policy's gap with the reference model's gap, and only the policy updates
NO REWARD MODEL IN BETWEEN
REFERENCE MODEL STAYS FIXED
Answer pair
DPO loss
Policy
chosen
rejected
policy gap
vs reference
shifts toward
chosen
building with ai
Prompt engineering
A prompt made of instructions, examples and a question goes to a model; the highlighted step scores the output against success criteria, and a dashed feedback path returns to revise the prompt wording.
prompt
instructions
examples
question
model
check
score against
criteria
revise wording
building with ai
System prompt
A system message setting role and format and a user request both enter the highlighted step that ranks them by authority, where the higher level wins, before the model writes a reply.
system
role: support
format: 3 steps
user
reset my key
reply
3 steps
rank by
authority
higher wins
building with ai
Few-shot prompting
One prompt holds three labelled example tickets and a new ticket; the highlighted model step continues the pattern and outputs the label urgent.
prompt
down → urgent
typo → normal
invoice → billing
checkout → ?
output
urgent
continue
pattern
same format
building with ai
Prompt chaining: feedback goes to a first model call that splits it into items, your code checks that list, and only then does the next model call use it
feedback
call 1
split
your code
checks
call 2
next job
each answer feeds the next job
building with ai
Chain-of-thought prompting
A word problem goes to the highlighted model step, which writes visible intermediate steps, 3 times 8 equals 24 and 24 minus 5 equals 19, before the final answer 19.
question
3 boxes × 8
5 pens sold
left = ?
generated text
3 × 8 = 24
24 − 5 = 19
answer: 19
write
steps
then answer
building with ai
Structured outputs: meeting notes pass through schema-constrained decoding and come out as a typed object with owner, task and due fields
notes
constrained
decoding
typed object
"Maya ships
fix by Fri"
schema-valid
tokens only
owner: Maya
task: fix
due: Fri
building with ai
JSON mode: a prompt passes a JSON syntax gate and comes out as parseable JSON, with its keys still unchecked
prompt
syntax
gate
{ ... }
asks "JSON"
block "Sure"
parses
keys unchecked
building with ai
Function calling: the model returns a tool call, get_weather for Paris, your code runs it, and the result goes back to the model
model
tool call
your code
runs it
get_weather
"Paris"
result
building with ai
Tool use: the model writes a call, a program outside the model runs the tool, and the result is written back to the model
question
model
program
writes a call
runs the tool
result written back
building with ai
Code interpreters: the model writes Python, a sandbox runs it, and the output comes back as the answer
python
sandbox
output
MODEL WRITES
RUNS HERE
BACK TO MODEL
a sandbox
building with ai
Semantic search
A question about a cord and a passage about a USB-C power adapter share no words, yet both become vectors. The highlighted step scores them with a dot product, so the adapter line comes back as the top match.
question
passage
top match
what cord charges
USB-C power adapter
adapter line
dot product
nearest first
concepts
Embedding models: two pieces of text go into an embedding model, which turns each into a vector of numbers, and vectors that sit close together mean related meaning
a cat
a kitten
embedding
model
two vectors
close together
= related
lists of numbers
concepts
Reranker: search results that were already found go into a reranker together with the query, which scores each one and puts them in a new order, best match first
query
results
reranker
best
next
last
ALREADY FOUND
NEW ORDER
building with ai
Vector databases
A password-reset query vector enters the highlighted index, which finds the two stored vectors nearest to it and returns their ids ranked by score.
query vector
ids + scores
password reset
id 42 · 0.91
id 17 · 0.88
find nearest
stored vectors
building with ai
Hybrid search
A BM25 keyword list and a vector list feed the highlighted fusion step, which adds 1 over 60 plus rank for each list and returns one merged order with the log for E-1042 first.
BM25 list
vector list
one order
exact words
same meaning
1. log E-1042
fuse ranks
1/(60+rank)
building with ai
Reranking
The question, reset a forgotten password, and one passage from the first-stage list go together into the highlighted cross-encoder, which reads them as one input and gives one score. The list is then sorted by those scores, putting the reset password steps first.
question
one passage
new order
reset password?
from first list
1. reset steps
CROSS-ENCODER
read together
one score
building with ai
Approximate nearest neighbor search: a query is matched to cell centres, only the nearest cells are opened, and the closest vectors come back
query
centres
open 2 cells
closest
rank cells
skip the rest
top k
building with ai
HNSW: a query enters a sparse top layer, drops through denser layers and reaches its nearest vectors on layer 0
query
layer 2
layer 1
layer 0
nearest
long hops
shorter hops
every vector
top k
building with ai
Metadata filtering: records tagged with a category such as digestive pass through a filter, and only the matching ones reach the vector search
digestive
heart
digestive
filter
search
= digestive
matches only
building with ai
Retrieval-augmented generation: a question is used to find matching passages in your documents, and the model answers from those passages with sources
question
find
model
answer
documents
refunds?
passages
reads them
+ sources
building with ai
Chunking
A document of seven text lines is cut at two paragraph breaks. The highlighted split step turns it into three chunks of three lines each. The line just before each cut is copied into both neighbouring chunks, so chunk 1 ends with the line chunk 2 starts with, and chunk 2 ends with the line chunk 3 starts with; brackets mark these as overlap.
document
cut
cut
chunk 1
chunk 2
chunk 3
overlap
overlap
split
concepts
Grounding: a question is used to search a source, the model answers from the relevant snippets, and the answer comes back with links to those sources
question
search
source
relevant
snippets
model uses
snippets
answer
+ links
building with ai
Knowledge graphs: Douglas Adams links to St John's College by an educated at edge, which links to the University of Cambridge by a part of edge
FOLLOW THE EDGES
Douglas
Adams
St John's
College
University
of Cambridge
Q42
Q691283
educated at
part of
building with ai
HyDE: a language model drafts a fake answer to the question, and the embedding of that draft is used to find real documents
question
fake answer
real docs
LLM drafts
found by embedding
building with ai
Contextual retrieval: a bare chunk gets a short context naming its document before it is indexed
bare chunk
indexed text
revenue
grew 3%
ACME,
Q2 2023
revenue
grew 3%
add
context
building with ai
GraphRAG: documents become an entity graph, and the graph is split into communities that each get a written summary
documents
entity graph
summary 1
summary 2
summary 3
TEXT
one per community
building with ai
AI agent: given a goal, the model chooses its next step, calls a tool, reads the result and repeats until it can answer
goal
model
answer
tool
picks next
step
call
result
when done
building with ai
Self-reflection: a model writes a draft, reflects on what went wrong, and writes a second try
draft
reflect
try 2
FIRST TRY
notes on errors
revised answer
building with ai
Voice agents: your spoken question goes to a voice model, which answers out loud
you speak
voice model
it speaks
AUDIO IN
LISTEN, THINK
AUDIO OUT
detects when you stop talking
building with ai
ReAct: a thought picks an action, and the tool's observation feeds the next thought
REASON + ACT, IN TURNS
thought
action
observation
plan next
search[…]
tool reply
feeds next thought
building with ai
Agentic workflows: model calls joined by a code gate (pass ends the run, fail goes to call 2) on a path fixed in code
fail
model
call 1
model
call 2
code gate
pass ends
the run
path fixed in code
building with ai
Human in the loop: an agent's proposed action waits for a person to approve it before the tool runs
agent
proposes
tool
runs
person
approves?
reject
yes
concepts
read screen
click type
building with ai
Context engineering: candidate tokens are curated into a small context window
all tokens
few tokens
history
tools
tool results
context
window
curate
keep, trim
building with ai
Agent memory: a new message triggers a search of saved memory, and the match is put back into the prompt
new
message
search
memory
prompt
+ match
saved notes
write
building with ai
Agent state: each step updates a saved record, so a paused run can reload it and resume
agent
step
update
state
save
checkpoint
pause
reload
resume
later
building with ai
Context compaction: a long history of old turns is summarised into one short summary so the conversation fits the context window
old turn
old turn
tool output
summarise
summary
long history
fits the window
building with ai
Agent planning: a planner turns one big goal into a short list of steps that run in order
big goal
planner
step 1
step 2
step 3
REQUEST
done in order
building with ai
Multi-agent systems: a lead agent splits a question across three subagents that work in parallel, then merges their condensed findings
lead agent
subagent 1
subagent 2
subagent 3
lead
splits the job
merges results
work in parallel
building with ai
Sub-agents: a main agent sends parts of a task to three sub-agents, and each sends back only a summary
main agent
sub-agent
sub-agent
sub-agent
summary
TASK
own context each
building with ai
Agent orchestration: a request reaches an orchestration step where a model or code decides whether a billing agent or a refund agent works next
request
orchestrator
billing agent
refund agent
model or code
who runs next?
building with ai
Model Context Protocol: an AI app connects through one standard protocol to an MCP server, which offers tools, resources and prompts
AI app
MCP server
tools
resources
prompts
MCP client
one protocol
building with ai
A2A protocol: a client agent reads an agent card, then hands a task to a remote agent and gets an artifact back
client
agent
agent card
remote
agent
skills
url · auth
task
artifact
building with ai
Sandboxing agents: every command an agent runs passes an operating-system boundary that allows some access and blocks the rest
agent
command
sandbox
allowed
blocked
files +
network
runs
denied
concepts
Evals: the same test cases go through the AI system, a grader checks each answer, and the result is a score
test
cases
AI
system
grade
answers
score
188/200
expected
answers
concepts
Benchmarks: the same test questions go to two models, one scoring rule marks both, and each gets a comparable score
test set
model A
model B
score
score A
score B
SAME TASKS
one rule
building with ai
LLM evaluation: a set of real test cases goes through the app, a grader marks each answer against pass rules, and the result is a score
20 cases
app
grade
17/20
TEST SET
PASS RULES
SCORE
same cases for every version
concepts
LLM-as-a-judge: an answer and a rubric go into a judge model, which gives a score of 4 out of 5
answer
rubric
judge LLM
4 / 5
score
grades by the rules
building with ai
Agent evaluation: one task runs through the agent in three trials, graders check the end state, and the results become a pass rate
task
agent
graders
score
one test
3 trials
check state
pass rate
concepts
Perplexity: the chances a model gave three words (1/2, 1/4, 1/8) become an average surprise of 1.386, and exponentiating that gives a perplexity of 4
chances
1/2 1/4 1/8
avg surprise
1.386
exponentiate
perplexity 4
EACH WORD
MEAN LOSS
concepts
science history law same exam format
score 57 tasks
concepts
graduate question
expert reasoning
A B C
concepts
function prompt
code candidate
tests
concepts
issue repository apply patch
tests
concepts
infer rule
concepts
Benchmark contamination: a leaked test question goes into training, so the model scores high on that question but lower on a new one
test Q
training
same Q
new Q: drop
LEAKED
compare scores
building with ai
Model serving: requests from three apps are grouped by a batcher and sent together to one model running on a server
app 1
app 2
app 3
batcher
model
ON A SERVER
one batch, three answers
building with ai
On-device AI: a prompt goes to a local model on the phone and the answer comes back, with no server call
prompt
local model
answer
ON THE DEVICE
no server call
building with ai
Batch inference: many prompts go into one batch job, and a results file comes back later
prompt 1
prompt 2
prompt 3
batch job
results
LATER
can wait
building with ai
Self-hosting LLMs: a model file is loaded onto your own server, and your app sends prompts to it
weights
your server
your app
MODEL
PROMPTS
127.0.0.1:11434
concepts
Inference optimization: a prompt goes to the model, which reuses a KV cache of earlier work to produce a faster reply
prompt
model
reply
KV cache
8-bit weights
less memory
less work
concepts
KV cache: the newest token's key and value are appended to a cache that keeps the keys and values of earlier tokens
new token
append k,v
KV cache
k1 v1
k2 v2
k3 v3
ONE DECODING STEP
old keys and values are reused
concepts
Quantization: a 32-bit weight of -1.8 is divided by a scale and rounded to the 8-bit code -114
-1.8
round(w/S)
-114
32-BIT
8-BIT
S = 2/127
concepts
Speculative decoding: a fast draft model guesses a few tokens, the target model checks all the guesses at once, keeps the ones it agrees with and fixes the first one it rejects
fast draft
target model
keep matches
fix 1st miss
guesses tokens
checks all at once
concepts
Latency: a stopwatch starts when a request is sent, passes the first token, and stops when the final response arrives
send
first token
final reply
START CLOCK
STOP CLOCK
latency = send to final reply
concepts
Throughput: 30 finished requests divided by a 10 second window gives 3 requests per second
30 done
10 seconds
divide
3 per sec
TIME WINDOW
FINISHED WORK
throughput
building with ai
Token costs: input tokens and output tokens are each multiplied by their own price, and the output rate is often higher
input
output
x in rate
x out rate
bill
per million tokens
building with ai
Rate limits: requests pass through a limit check; under the limit they get an answer, over it they get a 429 and must wait
requests
limit
answer
429 wait
APP
per minute
under
over
concepts
Tokens per second: a server returns tokens, and counting them over a set time gives throughput, measured separately from latency
requests
come in
server
returns
tokens
per second
latency
(separate)
count ÷ time
building with ai
Prompt caching: a new request with the same prompt start is checked against the cache, the saved work is reused, and only the new question is processed
same start
+ new ask
cache hit
reuse start
process new
PROMPT
saved KV states
building with ai
Model routing: a router sends each prompt to either a small model or a large model
prompt
router
small model
large model
easy prompts
hard prompts
building with ai
LLM gateway: an app sends every model call to one gateway, which checks keys and limits and forwards it to a primary model or a fallback model
app
gateway
model A
model B
keys, limits
logs
primary
on error
one URL for the app
concepts
GPUs: one big matrix multiplication is split across many GPU cores that all compute at the same time, then combined into one result
A × B
GPU cores
result
BIG SUM
all at the same time
ONE ANSWER
concepts
CPUs: a queue of tasks goes into a CPU with four strong cores, which works through a few at a time
tasks
CPU
done
IN LINE
a few strong cores
A FEW AT A TIME
concepts
TPUs: input numbers and weights flow into a TPU matrix unit, which multiplies and adds them into a result
inputs
weights
TPU
matrix unit
result
multiply and add
concepts
NPUs: an AI task inside a phone goes to the NPU, which runs it on the device and returns the answer
your voice
NPU
captions
ON THE DEVICE
low power
data stays here
concepts
AI accelerators: a model's multiply-adds are sent to a matrix unit that does many at once, producing an answer faster
model
maths
answer
matrix
unit
AT THE SAME TIME
GPU · TPU · NPU
concepts
FLOPs: about 6 operations per parameter for each training token, times the number of tokens
N params
× 6
× D tokens
MODEL SIZE
PER TOKEN
TOTAL FLOPs
≈ 6ND operations to train
concepts
Chinchilla scaling: at one fixed FLOP budget, loss dips to a minimum between more parameters and more tokens
fixed FLOP budget
lowest loss
loss
more tokens
more params
concepts
batch split
worker 1 worker 2 worker 3 merge gradients
concepts
Mixed precision: full FP32 weights are rounded to FP16 for fast math, the loss is scaled so small gradients survive, then the gradients are unscaled to update the FP32 weights
FP32 weights
FP16 math
scale loss
update FP32
round
unscale
concepts
Gradient checkpointing: the forward pass keeps only the input of a section, drops the middle steps, and rebuilds them when the backward pass needs them
save input
middle steps
output
rebuild middle
not kept
reuse input
backward pass
concepts
Hallucination: a plausible citation fails an evidence check and is flagged as unsupported
model draft
verify
result
Plausible
citation
DOI 10.xxxx/…
Evidence
check
No support
human review
Sounds right is not the same as supported.
concepts
Sycophancy: two users ask the same question with opposite views, the assistant answers both, and a check asks whether the facts stay the same instead of flipping to agree
I think yes
I think no
assistant
same facts?
same question
no flip to agree
building with ai
MLOps: new data trains a model, a check compares it with the live model, and monitoring sends it back for retraining
train
compare
deploy
new vs live model
monitor drift, then retrain
building with ai
LLMOps: a new prompt version passes an eval step before it goes live, and live monitoring feeds back into the next change
prompt v3
evals
live
VERSION
TEST
MONITOR
latency, tokens, errors
building with ai
LLM observability: one request passes through tracing, which records its prompt, token use and latency for each step so you can inspect it later
request
tracing
prompt
tokens
latency
ONE CHAT
kept per step
concepts
Prompt injection: trusted instructions and a web page with a hidden order are joined into one prompt, which the model reads as a single text
your
request
web page
+ hidden order
one
prompt
model
TRUSTED
UNTRUSTED
sees one text
concepts
Jailbreaks: a request the model would refuse is wrapped in a disguise, then sent to the model
request
disguise
model
role-play
fake chats
odd codes
would be refused
safety rules
tries to slip past
concepts
Guardrails: a message passes an input check, the model, then a highlighted output check before the reply is shown
input
model
output
reply
CHECK IN
CHECK OUT
blocked content gets a safe message
concepts
Data privacy: a chat goes to an AI service, and a training setting decides whether new chats may be used to train future models
your chat
training?
on: may train
off: kept out
MESSAGE
YOUR SETTING
rules vary by provider and plan
concepts
Indirect prompt injection: a hidden line inside an email joins the user's request in one prompt, and the model may follow it
summarise
email
one prompt
model
YOU
hidden order
may obey it
concepts
Red teaming: testers send attack prompts to an AI system, log what goes wrong, and retest after fixes
red team
AI system
harm list
ATTACKER
TARGET
FINDINGS
fix, then retest
