Vectors

1 · What it is

A vector is an ordered list of numbers, such as (3, 1). The order is part of the meaning: each position stands for one thing, and the number of positions is how many dimensions the vector has. So (3, 1) and (1, 3) are different vectors. You can also draw the list as an arrow that starts at the origin and ends 3 steps along and 1 step up. Mathematicians keep a third view too: a vector is anything you can add to another vector and stretch by a number.

Machine learning stores its data as vectors. Each example a model sees becomes a feature vector, one decimal number per feature. A small greyscale photo of 28 by 28 pixels is 784 brightness values from 0 to 255, which a network can flatten into one long row. Text goes in as embedding vectors, lists of learned numbers in which similar items get similar values. These lists are long. Google’s Gemini embedding models return 3,072 numbers by default and recommend 768 or 1,536 as smaller sizes, and OpenAI’s text-embedding-3-small returns 1,536.

Only a few moves are needed. Adding works position by position, and on the grid it means placing one arrow after the other. Scaling multiplies every number by the same amount, which stretches the arrow without turning it. Length, also called the norm, is the square root of the sum of the squared numbers. The distance between two vectors is the length of their difference. In an embedding, a small distance means two items are closely related, which is why search and recommendation compare vectors. The dot product and cosine similarity are two other ways to compare them.

In code, a vector is usually a one-dimensional array. NumPy’s norm function returns the ordinary length by default, and PyTorch holds a model’s inputs, outputs and parameters as tensors, which work like NumPy arrays but can run on GPUs. A table of vectors stacked together is a matrix, and the study of both is linear algebra.