Distillation

Knowledge distillation trains a smaller student model to imitate a larger teacher's outputs or internal representations, often reducing deployment cost.

Also known as Model compression.