ONNX Runtime is an open-source engine that runs machine learning models saved in the ONNX format quickly on many kinds of hardware and operating systems.
In plain words
Models are trained in many different tools, such as PyTorch or scikit-learn. If you export a model to the shared ONNX format, ONNX Runtime can load and run it. Before running, it rewrites the model's graph to make it faster, then hands pieces to plug-ins called execution providers that target a specific chip, such as a CPU, GPU or NPU. This lets you train in one language and ship the model in another.
A quick example
Imagine a team that trains an image model in Python with PyTorch but needs it inside a C# desktop app. They export the model to an ONNX file, then load that file with ONNX Runtime from C#. On a machine with a supported GPU, they can switch on the matching execution provider to speed it up.
Where you'll meet it
- Running models inside apps written in C#, C++ or Java.
- Speeding up inference on CPUs, GPUs and NPUs.
Related
Sources
- DocsWelcome to ONNX Runtime, ONNX Runtime documentation (Microsoft) · read 29 Sept 2026
- DocsONNX Runtime Execution Providers, ONNX Runtime documentation (Microsoft) · read 29 Sept 2026
- Docsmicrosoft/onnxruntime, Microsoft (GitHub) · read 29 Sept 2026
Checked by WhatIsAI against the sources above · 29 Sept 2026. Spotted an error? Report it.