Concepts

Pipeline parallelism

Reference entryTraining and efficiency

Reference entryPipeline parallelism

Pipeline parallelism places consecutive model stages on different devices and overlaps their work using multiple batches moving through the pipeline.