Concepts
Weights
1 · In one line
Trainable weights are model parameters that can be learned from data.
1 · What it is
In a dense layer, the input is multiplied by a weight matrix and then a bias is added. During training, an optimizer can update the trainable values.
Keras separates trainable weights from non-trainable weights. Non-trainable weights are not taken into account during backpropagation.
A TensorFlow checkpoint stores the values of variables in a module and its submodules. PyTorch’s state dictionary maps each layer to its parameter tensor. Keras describes a set of weight values as the state of the model.
A dense layer multiplies its input by a weight before adding a bias.
Scale inputsA dense layer matrix-multiplies its input by a weight parameter before adding bias.
Learn valuesTensorFlow.js backs every weight with a variable object.
Save stateTensorFlow calls checkpointed variable values the weights of a module.
Follow one input through a weighted sum and one training update.
- 1 · multiplyMultiply each input by its corresponding weight.
- 2 · combineAdd the weighted inputs and the bias to form the layer output.
- 3 · compareDuring training, compare the prediction with the target through a loss.
- 4 · updateApply an optimizer update to the trainable weights.
Frameworks may use weights for both trainable and non-trainable layer state.
| Who | What they ask | What it works with |
|---|---|---|
| Training engineer | “Which stored values receive optimizer updates?” | The trainable weights collection |
| Inference engineer | “Which values must be loaded before predictions run?” | The model checkpoint or state dictionary |
| Model debugger | “Why did this layer's output change?” | Inputs, weights and bias |
| Fine-tuning team | “Which layers should stay fixed?” | Trainable flags on layer weights |
solves
- TensorFlow calls variable values inside a module its weights.
- Keras categorizes some layer weights as non-trainable.
- TensorFlow calls checkpointed variable values the weights of a module.
doesn't solve
- Not every weight is trainable.
6 · Go deeper
Sources used
This explainer is written in original language. The links below support its factual claims.
- docsIntroduction to modules, layers, and models, TensorFlow · read 28 Sept 2026
- docsTraining models, TensorFlow · read 28 Sept 2026
- docsMaking new layers and models via subclassing, Keras · read 28 Sept 2026
- docsSaving and Loading Models, PyTorch · read 28 Sept 2026
- docsSave, serialize, and export models, TensorFlow · read 28 Sept 2026