Docs

Code Pipeline

Qwen

Quickly deploy a QwQ large model inference service

To help enterprise users quickly experience the QwQ-32B large model in a cloud environment, this document introduces a rapid deployment solution for QwQ-32B to run inference services through the vLLM large model framework. The solution integrates Volcano Engine Elastic Compute Service (ECS), Volcengine Kubernetes Engine (VKE), and Code Pipeline (CP).

Background information

QwQ-32B is a newly open-sourced inference model in the community. It has demonstrated outstanding performance in a series of authoritative benchmark tests, including the AIME24 evaluation set for mathematical reasoning, LiveCodeBench for programming ability, LiveBench, the IFEval instruction-following test, the BFCL evaluation, and other authoritative benchmark tests. It has significantly improved the intelligence of the language model through large-scale reinforcement learning technology, specifically in its explicit chain-of-thought capability. During reasoning, it displays a complete path of thought, which enhances the interpretability of the results.

Notes

The following content mainly introduces practices that have been tested and verified. To achieve the expected results and comply with the limit, please follow the approach described in this document or operate on the resources recommended in this document. If you require a replacement solution, you can contact the relevant account manager for advice.

Prerequisites

When creating a container cluster in Volcengine Kubernetes Engine, pay attention to the following parameter configuration. For detailed operating instructions, refer to Create cluster.

Volcengine Kubernetes Engine

Container network model: select VPC-CNI.

Compute specifications: It is recommended to deploy different models on different machine types to maximize the cost-performance ratio. The following are recommended configurations for commonly used distillation model machine types for your reference.

Machine model	GPU quantity	GPU video memory	CPU	Memory
ecs.gni2.28xlarge	4	4 * 24G	112	448G
ecs.gni3cl.11xlarge	2	2 * 48G	44	240G
ecs.pni2.3xlarge	1	1 * 80G	14	245G
ecs.pni3l.5xlarge	1	1 * 96G	22	244G

Component configuration: Install the csi-tos and nvidia-device-plugin components.

API gateway

The API gateway has been created. The private network setting must be the same as that of the created VKE cluster. Select gateway node specification 1c2g 2. The protocol is HTTP1.1. For detailed instructions on creating an API Gateway instance, refer to Create instance.

Procedure

This article describes how to use the Volcano Engine Code Pipeline Product to quickly deploy DeepSeek-R1-Distill in an already created Volcengine Kubernetes Engine.

Step 1: Create a deployment cluster

Integrate the created VKE cluster into the Code Pipeline platform.

In the left navigation bar, select Resource management.

On the Resource management page, switch to the deploy resource tab.

On the deploy resource tab, click create deploy resource.

In the Create a deploy resource dialog box, configure the deploy resource information as required. Pay special attention to the following parameter configurations. For descriptions of other parameters, refer to Access a VKE cluster.

Configuration Item	Descriptions
Connection Type	Select Volcengine Kubernetes Engine VKE.
Region	Select the region where the created Volcengine Kubernetes Engine cluster is located.
Sharing scope	Select all workspaces.

Step 2: Create an AI application

In the Code Pipeline AI application module, deploy large model applications.

Select AI application in the left navigation bar.

On the AI application page, click create application.

Select Create based on AI model > QwQ-32B, click Deploy.

Fill in the relevant configuration information for the application as instructed. After configuration is complete, click Create. The application will begin creating and deploying.

Model configuration

Configuration Item
Descriptions
Deployment method
Select vLLM.
Advanced configurations
Supports deploying distributed inference models across multiple hosts. This example is not enabled.

Deploy clusters

Configuration Item
Descriptions
deploy resource
Select the created Volcengine Kubernetes Engine clusters.

Inference service specifications

The specifications of the cloud servers used for model deployment vary, and the specifications of the corresponding configurable services also differ. The following are the recommended service specifications for different device models. This document uses ecs.gni3cl.11xlarge as an example.

Configuration Item	Descriptions
Number of instances	select1.
Volcengine Container Instance (VCI)	This example does not select the feature.
Resource configuration type	The specifications of the cloud servers used for model deployment vary, and the corresponding configurable service specifications also differ accordingly. It is recommended to configure the corresponding service specification parameters according to the table below.

Machine model	Number of GPUs	CPU requests/limits	Memory requests/limits
ecs.gni2.28xlarge	4	14	160G
ecs.gni3cl.11xlarge	2	14	160G
ecs.pni2.3xlarge	1	8	160G
ecs.pni3l.5xlarge	1	14	160G

Step 3: Create an API Gateway to access the inference service

Volcano Engine API Gateway (APIG) is a cloud-native, highly scalable, and highly available managed cloud gateway service. Building on traditional traffic gateways, it integrates comprehensive service discovery and governance capabilities. This enables seamless connectivity between internal and external networks in a microservices architecture and ensures secure communication.

Select AI application in the left navigation bar.

On the AI application page, select the target AI application, click the application card, and enter the basic information tab for the current application.

On the Basic Information > Access settings tab, select API Gateway.

Click add API gateway to add an API gateway that meets the requirements of the prerequisites.

After completing the above Actions, you can view the model's public domain name on the Access settings page.

Actions result

By using the local curl command to call the large model API, you can successfully converse with the large model.

curl -X POST http://example.com/v1/chat/completions -H "Content-Type: application/json" -d '{​
    "model": "/model",​
    "messages": [​
        {​
            "role": "user",​
            "content": "your question"​
        }​
    ],​
    "temperature": 0.7​
}'​

Last updated: 2025.09.10 16:17:53

Configuration Item	Descriptions
Deployment method	Select vLLM.
Advanced configurations	Supports deploying distributed inference models across multiple hosts. This example is not enabled.

Configuration Item	Descriptions
deploy resource	Select the created Volcengine Kubernetes Engine clusters.