Menu
Concepts

Neural networks

4 min readbeginnerUpdated 28 Sept 2026
1 · In one line

A neural network is a model built from layers of simple units that each weigh their inputs, add them up and bend the result, so that together they can learn curved, complex patterns.

1 · What it is

A neural network is a machine learning model made of layers. The first layer holds the input features, the last layer holds the prediction, and one or more hidden layers sit in between. Each unit in a hidden layer is called a neuron.

A neuron does very little on its own. It multiplies each of its inputs by a weight, adds the results and a bias, and passes the total through an activation function. That last step is what matters: without it, stacking layers would still give a straight-line model. With it, each layer can bend the data a little, and many layers together can follow very complicated patterns.

Nobody sets the weights by hand. Training shows the network many examples, measures how wrong each prediction is, and adjusts every weight and bias to make the next prediction a little better. The standard method for doing this, back-propagation, was popularised for neural networks by a 1986 paper, after earlier versions had been published and reinvented several times. Its key result was that hidden neurons learn useful features of the task by themselves. The name comes from a loose analogy with brain cells, but the working parts are ordinary arithmetic.

2 · Why it exists

A single weighted sum can only draw straight lines.

Curved patternsMany real datasets cannot be split into their groups by any straight line, however the weights are chosen.
Guessing combinationsA simple model can handle curves only if someone first invents the right combinations of inputs, which takes trial and error.
Stacking alone is not enoughPiling up more weighted sums does not help by itself, because a sum of sums is still a straight-line model.
3 · How it works

Follow one number through one neuron.

INSIDE ONE NEURON Input 1 value 2.0 × 0.5 Input 2 value 1.0 × −1.5 Input 3 value 3.0 × 0.4 WEIGHTS: LEARNED IN TRAINING Add up + bias 0.3 1.0 − 1.5 + 1.2 + 0.3 = 1.0 ACTIVATION Bend the result ReLU: below 0 → 0 Output 1.0 sent to every neuron in the next layer WHERE IT SITS INPUT HIDDEN LAYERS OUTPUT One neuron in a hidden layer. A network repeats the same two steps in every neuron, layer after layer.
  1. 1 · inputsThe neuron receives numbers, either the raw features or the outputs of the layer before it.
  2. 2 · weighIt multiplies each input by its own weight, a number learned during training.
  3. 3 · addIt adds those products together, plus one extra learned number called the bias.
  4. 4 · bendIt passes the total through an activation function, such as ReLU, which turns negative values into zero.
  5. 5 · passIn a dense, fully connected layer like this example, the result goes to every neuron in the next layer; the final layer produces the prediction.

Without the bend in step 4, any number of layers would collapse into one straight-line model.

4 · Where it's used
WhoWhat they askWhat it works with
Photo app“Is there a face in this picture?”Pixel values of the image
Speech system“Which words are in this recording?”The audio signal as numbers
Text tool“What is the next likely word in this sentence?”The words that came before
Pricing team“What will this house sell for?”Size, rooms, location and other features
5 · What it solves, and what it doesn't
solves
  • It finds curved, non-linear patterns without anyone designing combinations of inputs by hand.
  • The same building block, repeated in layers, can model very complicated relationships.
  • Standard software libraries handle the training maths, so the structure is easy to build on.
doesn't solve
  • It is not always better than a simpler model with well-chosen input combinations.
  • Its learned weights are hard to interpret, unlike those of a linear model.
  • It can memorise its training data and then fail on new data.
  • Deep networks can be slow or unstable to train when the signal passed back through the layers shrinks or grows too much.
6 · Go deeper
Deep learningActivation functionBackpropagationGradient descentParameters

Sources used

This explainer is written in original language. The links below support its factual claims.

  1. docsMachine Learning Glossary, Google for Developers · read 27 Sept 2026
  2. docsNeural networks (Machine Learning Crash Course), Google for Developers · read 27 Sept 2026
  3. docsNeural networks: Nodes and hidden layers (Machine Learning Crash Course), Google for Developers · read 27 Sept 2026
  4. docsNeural networks: Activation functions (Machine Learning Crash Course), Google for Developers · read 27 Sept 2026
  5. docsNeural networks: Training using backpropagation (Machine Learning Crash Course), Google for Developers · read 27 Sept 2026
  6. paperDeep learning, Nature (LeCun, Bengio and Hinton, 2015) · read 27 Sept 2026
  7. paperLearning representations by back-propagating errors, Nature (Rumelhart, Hinton and Williams, 1986) · read 27 Sept 2026
  8. docstorch.nn.Linear, PyTorch · read 27 Sept 2026
  9. docstorch.nn.ReLU, PyTorch · read 27 Sept 2026
  10. officialFathers of the Deep Learning Revolution Receive ACM A.M. Turing Award, ACM · read 27 Sept 2026
  11. paperAutomatic differentiation in machine learning: a survey, Baydin, Pearlmutter, Radul, and Siskind (arXiv; JMLR) · read 27 Sept 2026
Suggest a correction