This topic, with 1 ecs.hpcpni3ln.45xlarge instance as an example, introduces how to deploy the DeepSeek-V4.1-Flash model on a GPU instance.

Background information

DeepSeek-V4.1-Flash is the most lightweight model in DeepSeek's new model architecture series. The model provides native multimodal visual understanding capabilities and can process image and text inputs at the same time.

The model adopts a new asymmetric Causal-Encoder-Decoder architecture, with 8 billion activated parameters for input and 16 billion activated parameters for output. The model inference cost is significantly lower than that of known models of the same size. The model also uses a new pre-training method and undergoes larger-scale reinforcement learning during post-training. In benchmark tests, it outperforms flagship models such as DeepSeek-V4-Pro overall.

Software requirements

NVIDIA driver:

GPU driver: A program used to drive NVIDIA GPUs. In the example provided in this topic, version 535.261.03 is used.

Runtime environment:

SGLang framework: a high-performance service framework specifically built for Large Language Models (LLMs) and vision-language models. With the collaborative design of the frontend structured generation language and the backend runtime system, the SGLang framework supports efficient interaction with models and ensures controllability of the generation process. In the example provided in this topic, the SGLang community preview version is used.

Procedure

Step 1: Prepare the environment

Create a GPU Accelerated instance.

Follow the Purchasing instances by following the wizard guide to create an instance that meets the following conditions:

Parameter	Recommended configuration
Basic information
Instance type	ecs.hpcpni3ln.45xlarge tip The DeepSeek V4 model is currently supported only by instance types in the ecs.hpcpni3 family. The instance type ecs.hpcpni3ln.45xlarge is currently in Beta. If you need a trial, please submit a ticket or contact your account manager to apply.
Image	Ubuntu 22.04. Select the "Automatically install GPU driver in the background" option to install the GPU driver 535.261.03. tip If you choose an image of another version, you need to install the GPU driver after creating the instance.
Number of instances	1
Storage	Specification: TSSD TL0 volume Capacity: no less than 1,024 GiB tip The DeepSeek-V4.1-Flash model used in this topic has 552 billion parameters. We recommend that you configure a volume with a capacity of at least 1024 GiB.
Network configuration
EIP	Select the "Assign Elastic IP (EIP)" checkbox.

Install the required software and tools.

Log in to the target instance.

Execute the following command on the GPU instance to install and configure Docker.

sudo apt update
sudo apt install ca-certificates curl gnupg lsb-release
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://mirrors.ivolces.com/docker/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://mirrors.ivolces.com/docker/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
sudo apt install docker-ce docker-ce-cli containerd.io docker-compose-plugin -y

Execute the following commands to install the NVIDIA Container Toolkit.

curl -s https://mirrors.ivolces.com/nvidia_all/ubuntu2204/x86_64/3bf863cc.pub | sudo apt-key add -
cat <<EOF >/etc/apt/sources.list.d/nvidia.list
deb http://mirrors.ivolces.com/nvidia_all/ubuntu2204/x86_64/ /
EOF
apt update
apt install nvidia-container-toolkit -y

Execute the following commands to configure Docker so that containers can use the instance's GPU resources.

sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

(Optional) Initialize the local volume.

If the instance type you choose has local volumes provided, we recommend that you execute the following commands to mount the local volume to the directory "/data00" where the model is stored. By doing this, you can improve the model loading speed.

mkfs -t ext4 /dev/nvme0n1
mount -t ext4 /dev/nvme0n1 /data00

Step 2: Deploy and run the model

Log in to the GPU instance. Learn more.

Download the model file.

Execute the following command to install and configure onion.

curl -fsSL https://mirrors.ivolces.com/extra-tools/ubuntu/GPG-KEY-system | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/volc-extra-tools.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/trusted.gpg.d/volc-extra-tools.gpg] http://mirrors.ivolces.com/extra-tools/ubuntu jammy main" > /etc/apt/sources.list.d/volctools.list && apt update && apt install -y onion-ai-data

Execute the following command to download the DeepSeek-V4.1-Flash model and store it in the target directory.

oniond download model DeepSeek-V4.1-Flash --dir /data00

Execute the following commands on the GPU instance based on the region of the instance to download the Docker image.

China (Beijing)
China (Shanghai)

docker pull ai-containers-cn-beijing.cr.volces.com/deeplearning/sglang:dev-dsv41

Based on the instance region, execute the corresponding command on the GPU instance to start the Docker container and the DeepSeek-V4.1-Flash model service.

China (Beijing)
China (Shanghai)

docker run --gpus all -itd --net=host --shm-size=100g --ulimit memlock=-1 --ulimit stack=67108864 --privileged --ipc=host --security-opt seccomp=unconfined --cap-add=ALL -v /data00/:/data00/ -v /var/run/nvidia-topologyd/:/var/run/nvidia-topologyd/ --entrypoint sglang ai-containers-cn-beijing.cr.volces.com/deeplearning/sglang:dev-dsv41 serve \
--trust-remote-code \
--model-path /data00/DeepSeek-V4.1-Flash \
--tp 8 \
--ep-size 8 \
--attention-backend dsv4 \
--moe-runner-backend flashinfer_mxfp4 \
--enable-decoder-swa-bounded-replay \
--reasoning-parser auto \
--tool-call-parser auto \
--image-processor-backend pil \
--cuda-graph-max-bs-decode 32 \
--host 0.0.0.0 \
--port 30000

Parameter descriptions:

The preceding commands are used to start and deploy the DeepSeek-V4.1-Flash model. To deploy and start other models, replace the relevant environment variables and parameter values in the commands according to the parameter descriptions in the following table.

Parameter	Parameter type	Description	Sample value
--model-path	Path/model ID	Specifies the LLM model to load Hugging Face model ID Local model path	/data00/DeepSeek-V4.1-Flash
--tp	Integer	Configures the tensor parallelism size. In this example, TP8 is configured on a single instance.	8
--host	IP address	Specifies the host address associated with the service.	0.0.0.0
--port	Integer	Specifies the service listening port, which is used for client connection.	30000

Execute the following command to check from the Docker logs whether the container and model have started successfully.

View the container ID.

docker ps

View the running log of the corresponding container.

docker logs -f <CONTAINER_ID>

Step 3: Call the model

Log in to the instance and enter the Docker container.

Execute the following command to call the model and verify that the deployed model can perform inference properly.

curl -X POST http://localhost:30000/v1/chat/completions -H "Content-Type: application/json" -d '{
"model": "deepseek-v4.1-flash",
"messages": [
{
"role": "user",
"content": ""
}
],
"max_tokens": 500,
"temperature": 0.6
}'

Parameter descriptions:

This topic only describes the parameters used for verification.

Parameter	Description	Sample value
model	Indicates the actual model used.	deepseek-v4.1-flash
messages	The list of messages in the conversation. role: the role that initiates the message. content: the content of the message.	-
max_tokens	Specifies the maximum number of tokens in the completion generated by the model for a single request. Valid values: An integer from 1 to 8192. The default value is 4096.	500
temperature	The sampling temperature. Value: Between 0 and 2. The higher the temperature (for example, 1), the more random the output is. A lower value (for example, 0.2) makes the output more focused and deterministic.	0.6

GPU Compute Service