Docs

GPU Compute Service

MiniMax series models

GPU - Deploying MiniMax-M2.7 model

This topic describes how to deploy the MiniMax-M2.7 model on a GPU instance, using an ecs.hpcpni3ln.45xlarge instance as an example.

Background

MiniMax‑M2.7 is an open‑source large language model developed by MiniMax. It delivers a 196K context window, boasting approximately 229B total model parameters and 10B active parameters during inference. As a self-evolving model developed with deep involvement from MiniMax, it can build complex agent frameworks and handle highly sophisticated productivity tasks.

Software requirements

NVIDIA driver:

GPU driver: A program used to drive NVIDIA GPUs. In the example provided in this topic, version 535.161.08 is used.

Runtime environment:

SGLang framework: a high-performance service framework specifically built for Large Language Models (LLMs) and vision-language models. With the collaborative design of the frontend structured generation language and the backend runtime system, the SGLang framework supports efficient interaction with models and ensures controllability of the generation process. In the example provided in this topic, v0.5.10 is used.

Procedure

Step 1: Prepare the environment

Create a GPU Accelerated instance.

Follow the Purchasing instances by following the wizard guide to create an instance that meets the following conditions:

Configuration item	Recommended configuration
Basic information
Instance type	ecs.hpcpni3ln.45xlarge tip The instance type ecs.hpcpni3ln.45xlarge is currently in Beta. If you need a trial, please submit a ticket or contact your account manager to apply.
Image	Ubuntu 22.04 with GPU Driver 535.161.08 and doca tip If you choose an image based on another distribution release, you need to install the GPU driver after creating the instance.
Number of instances	1
Storage	Specification: TSSD TL0 volume Capacity: no less than 1,024 GiB tip The MiniMax-M2.7 model used in this topic has 229 billion parameters. We recommend configuring a volume capacity of no less than 1,024 GiB.
Network configuration
EIP	Select "Assign Elastic IP (EIP)".

Install the required software and tools.

Execute the following commands to install and configure Docker.

sudo apt update
sudo apt install ca-certificates curl gnupg lsb-release
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://mirrors.ivolces.com/docker/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://mirrors.ivolces.com/docker/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
sudo apt install docker-ce docker-ce-cli containerd.io docker-compose-plugin -y

Execute the following commands to install the NVIDIA Container Toolkit.

curl -s https://mirrors.ivolces.com/nvidia_all/ubuntu2204/x86_64/3bf863cc.pub | sudo apt-key add -
cat <<EOF >/etc/apt/sources.list.d/nvidia.list
deb http://mirrors.ivolces.com/nvidia_all/ubuntu2204/x86_64/ /
EOF
apt update
apt install nvidia-container-toolkit -y

Execute the following commands to configure Docker so that containers can use the instance's GPU resources.

sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

(Optional) Initialize the local volume.

If the instance type you choose has local volume provided, we recommend that you execute the following commands to mount the local volume to the directory /data00 where the model is stored. By doing this, you can improve the model loading speed.

mkfs -t ext4 /dev/nvme0n1
mount -t ext4 /dev/nvme0n1 /data00

tip

Replace /dev/nvme0n1 with the name of the actual local volume.

Replace ext4 with the file system you want to use.

Step 2: Deploy and run the model

Download the model file.

Execute the following command to install and configure onion.

curl -fsSL https://mirrors.ivolces.com/extra-tools/ubuntu/GPG-KEY-system | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/volc-extra-tools.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/trusted.gpg.d/volc-extra-tools.gpg] http://mirrors.ivolces.com/extra-tools/ubuntu jammy main" > /etc/apt/sources.list.d/volctools.list && apt update && apt install -y onion-ai-data

Execute the following command to enter the directory where the model is stored.

cd /data00

Execute the following commands to download the MiniMax-M2.7 model.

oniond download model MiniMax-M2.7

tip

You can also execute the oniond list model command to view the list of available models.

Execute the following commands on the GPU instance based on the region of the instance to download the Docker image.

China (Beijing)
China (Shanghai)

docker pull ai-containers-cn-beijing.cr.volces.com/deeplearning/sglang:v0.5.10.byted.202604131511

Execute the following commands on the instance based on the region of the instance to start the Docker container.

warning

Please replace /data00 with the actual path where the model is stored.

China (Beijing)
China (Shanghai)

docker run --gpus all -itd --net=host --shm-size=100g --ulimit memlock=-1 --ulimit stack=67108864 --privileged --ipc=host --security-opt seccomp=unconfined --cap-add=ALL -v /data00/:/data00/ -v /var/run/nvidia-topologyd/:/var/run/nvidia-topologyd/ --entrypoint /bin/bash ai-containers-cn-beijing.cr.volces.com/deeplearning/sglang:v0.5.10.byted.202604131511

Execute the following command to check from the Docker logs whether the container and model have started successfully.

View the container ID.

docker ps

View the running log of the corresponding container.

docker logs -f <CONTAINER_ID>

tip

Replace <CONTAINER_ID> with the actual container ID.

Start the model service.

Execute the following command to enter the Docker container:

docker exec -it <CONTAINER_ID> bash

tip

Replace <CONTAINER_ID> with the actual container ID.

Please select the appropriate command to start the MiniMax‑M2.7 model inference service based on the actual number of GPUs in your instance (4 or 8).

tip

You can check the number of GPUs in the "GPU" column of the "Instance types and metrics" table that corresponds to the GPU instance type family you are using. For example: The GPU count for one HPC-optimized hpcpni3ln instance is 8.

4-GPU environment
8-GPU environment

GLOO_SOCKET_IFNAME=eth0 NCCL_MIN_NCHANNELS=24 NCCL_IB_QPS_PER_CONNECTION=8 \
python -m sglang.launch_server \
--model-path /data00/MiniMax-M2.7 \
--tp-size 4 \
--tool-call-parser minimax-m2 \
--reasoning-parser minimax-append-think \
--trust-remote-code \
--host 0.0.0.0 \
--port 8000 \
--mem-fraction-static 0.85

Parameter descriptions:

In the example from the previous step, the commands are used to start and deploy the MiniMax-M2.7 model. To deploy and start other models, replace the relevant environment variables and parameter values in the commands as described in the following table.

Parameter	Parameter type	Description	Sample value
python -m sglang.launch_server	Startup command	Start the SGLang model server as a Python module. This item is not a parameter, but a command entry.	-
--model-path	Path/model ID	Specifies the LLM model to load Hugging Face model ID. Local model path.	/data00/MiniMax-M2.7
--tp-size	Integer	Configures Tensor Parallelism.	4 or 8
--ep-size	Integer	Configures Expert Parallelism.	8
--host	IP address	Specifies the host address associated with the service.	0.0.0.0
--port	Integer	Specifies the port number that the service listens on for client connections.	8000
--mem-fraction-static	Ratio	A memory ratio used for static allocation (model weights and KV cache memory pool)	0.8

Step 3: Call the model

Execute the following command to call the model and verify that the deployed model can perform inference properly.

curl -X POST http://127.0.0.1:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
"model": "MiniMax-M2.7",
"messages": [
{
"role": "user",
"content": "Please introduce the Forbidden City in Beijing."
}
],
"max_tokens": 500,
"temperature": 0.7
}'

Last updated: 2026.06.25 10:27:48