The Stack is an open dataset of source code from the BigCode project, built for training language models that write code.
In plain words
Code models learn from large amounts of real code. The Stack collects source files from repositories and keeps only code under permissive licences, in many programming languages. It also comes with a data governance plan, and later versions removed code whose authors asked to opt out. The paper found that removing near-duplicate files made trained models noticeably better.
A quick example
Imagine a lab training a small model to complete Python functions. Instead of scraping code itself, it can use the Python part of The Stack, knowing the files were chosen for permissive licences, then measure its model on a code benchmark such as HumanEval.
Where you'll meet it
- Pretraining open code models.
- Research on licensing and consent in training data.
Related
Sources
- PaperThe Stack: 3 TB of permissively licensed source code, Kocetkov et al. (arXiv) · read 29 Sept 2026
- Docsbigcode/the-stack dataset card, BigCode (Hugging Face) · read 29 Sept 2026
Checked by WhatIsAI against the sources above · 29 Sept 2026. Spotted an error? Report it.