Docs

Volcengine Kubernetes Engine

mGPU scheduling

mGPU overview

Volcengine Kubernetes Engine (VKE) provides the multi-container GPU (mGPU) solution to enable containers to share a GPU.

Introduction to mGPU

mGPU is a container-sharing GPU solution based on kernel virtualization and isolation of GPUs and a self-developed scheduling framework. While ensuring performance and fault isolation, mGPU enables multiple containers to share a single GPU, flexibly schedules and strictly isolates computing power and GPU memory, reduces GPU usage costs, and enables you to efficiently use GPU hardware resources for training and inference.

Benefits

Strict isolation

mGPU achieves strict isolation of GPU memory and computing power, supports fine-grained configuration of computing power amount and proportion, and maximizes GPU resource utilization.

Compatibility and openness

mGPU is compatible with mainstream architectures such as Volta、Turing、Ampere 、Hopper、 Ada Lovelace、 Blackwell, and is adapted to the standard open source Kubernetes.

Multi-dimensional monitoring

mGPU provides built-in monitoring clients and supports multi-dimensional metrics to enable clear visual monitoring.

High performance

mGPU can dynamically adjust timeslices, improving the throughput of business processes and delivering greater business performance gains.

Solution architecture

The following figure shows the architecture of the mGPU solution.

In the mGPU solution, the underlying GPU hardware and drivers are managed by the mGPU kernel driver module to ensure strict isolation of memory, computing power, and faults. mGPU Container Runtime provides a container-runtime configuration API. VKE provides configuration entry points based on Device Plugin and Scheduler Plugin capabilities.

Application scenario

mGPU enables multiple containers to share a GPU. It is suitable for scenarios where the computing power of a GPU is excessive for a single business application. GPU sharing and computing power isolation among applications can improve GPU utilization, reduce costs, and ensure business security isolation.

Last updated: 2026.08.17 17:37:37