Volcengine Kubernetes Engine
Volcengine Kubernetes Engine
Volcengine Kubernetes Engine
Document
Volcengine Kubernetes Engine
What's New
History feature release notes
Kubernetes version release notes
Containerd runtime release notes
Add-on release notes
Monitoring add-ons
Scheduling add-ons
DNS add-ons
Security add-ons
GPU add-ons
Sandbox add-on
Security announcements
Product Announcement
Historical announcements
Overview
Getting started
User guide
Clusters
Creating clusters
Connecting to clusters
Upgrading clusters
Node pool
Creating node pools
Scaling out/in node pools
Nodes
Authorization management
Image
Namespaces
Networking
Cluster network
Container Network
Service
LoadBalancer
CLB
Ingress
APIG Ingress
ALB Ingress
Customize ALB Ingress forwarding rules
Certificate and security
CLB Ingress
Nginx Ingress
Gateway API
Workloads
Creating workloads
Configure workloads
Pods
Configuration management
Image acceleration
Storage
FSX client
EBS Volumes
TOS Volumes
NAS volumes
FSX access methods
NFS access methods
EFS volumes
vePFS volumes
FSX access methods
vePFS Client access methods
NFS access methods
CloudFS volumes
Local volumes
Autoscale
Node autoscaler
Workload autoscaler
Horizontal Pod Autoscaler (HPA)
Cron Horizontal Pod Autoscaler (CronHPA)
Intelligent Horizontal Pod Autoscaler (IHPA)
Horizontal Pod Autoscaler based on event-driven (KEDA)
Add-on management
Network add-ons
Storage add-ons
Monitoring add-ons
Scheduling add-ons
DNS add-ons
GPU add-ons
Core add-ons
Sandbox add-on
Upgrading add-ons
Heterogeneous computing
NVIDIA GPU
mGPU scheduling
RDMA high-performance computing
Scheduling management
Topology-aware scheduling
QoS-aware scheduling
Observability
Log management
Basic observation
Setup & Configuration
Cluster observation
Storage observation
Other observation
Using metrics
Full stack observation
Alert management
Diagnosis and inspection
Node check and self-healing
Application center
Backup center
Stability
Large-scale cluster management
Best practices
Authorization
Clusters
Nodes and node pools
Networking
Storage
Scheduling
Observability
Solution
Security and compliance
API reference
Cluster management
NodePool management
Node management
Virtual nodes management
Add-on management
Tag management
Access policy management
Instance images management
Cloud native observability
Inspection and diagnosis
Task management
Volcengine Container Instance
Appendix
SDK reference
Development guide
Terraform
FAQs
General FAQs
Cluster FAQ
cluster management
cluster network
cluster upgrade
Node and Node pool FAQs
Authorization FAQs
Workload FAQs
Service and Ingress FAQs
Storage FAQs
Heterogeneous computing FAQs
Observability FAQs
Quota FAQs
Add-on FAQs
Cloud-native AI
ServingKit
LLM applications
ComfyUI application
Deploying the Al inference application through Helm
Deepseek practices
Qwen practices
Kimi practice
Seed practices
Best practices
DeepSeek model deployment
GLM model deployment
MiniMax model deployment
Qwen model deployment
Kimi model deployment
Volcengine Container Instance
What's new
Historical feature release notes
Product Announcement
Overview
Billing
User guide
Integrating with VCI
Virtual node
VKE cluster uses virtual nodes
Self-managed/third-party cluster uses virtual node
Creating an instance
General purpose instance
GPU-accelerated computing instance
VCI Agent Sandbox
Startup acceleration
E2B-based management
Upgrade guide
Networking
Monitoring
Scheduling
Container configuration
VCI Image Cache
Best practices
Security and compliance
FAQs
- Documentation
Volcengine Kubernetes EngineCloud-native AIServingKitLLM applicationsLLM FAQ
Copy page
Download PDFLLM applications
LLM FAQ
Copy pageDownload PDFLLM FAQ
This topic provides frequently asked questions and solutions related to deploying LLM applications using ServingKit.
*How do I download a custom model to a node?
*When deploying an AI application with ServingKit using a custom model, you need to download the custom model to the GPU node where the application resides. Follow the steps below to download the model.
*Log in to the GPU node where the application resides via ECS Terminal. For detailed operations, see ECS Terminal.
*Run the following command to install the Oniond (model downloader) tool on the node.
*
*Run the following command to enter the specified mount directory on the node's local disk.
*
*The specified mount directory on the node's local disk is the directory specified under the Data volume parameter in the Advanced settings section when creating the managed heterogeneous computing cluster for your ServingKit application. By default, the mount directory is /data01.
*
*Run the following command to download the custom model to the specified mount directory on the node's local disk, that is, the /data01 directory.
*
*After the model is downloaded, the expected response is as follows.
*
*
How do I download a custom model to a node?
When deploying an AI application with ServingKit using a custom model, you need to download the custom model to the GPU node where the application resides. Follow the steps below to download the model.
- Log in to the GPU node where the application resides via ECS Terminal. For detailed operations, see ECS Terminal.
- Run the following command to install the Oniond (model downloader) tool on the node.
- curl -fsSL https://mirrors.ivolces.com/extra-tools/debian/GPG-KEY-system | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/volc-extra-tools.gpgecho "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/trusted.gpg.d/volc-extra-tools.gpg] http://mirrors.ivolces.com/extra-tools/debian buster main" > /etc/apt/sources.list.d/volctools.list && apt update && apt install -y onion-ai-data
- Run the following command to enter the specified mount directory on the node's local disk.
- cd /data01
- The specified mount directory on the node's local disk is the directory specified under the Data volume parameter in the Advanced settings section when creating the managed heterogeneous computing cluster for your ServingKit application. By default, the mount directory is /data01.
-
- Run the following command to download the custom model to the specified mount directory on the node's local disk, that is, the /data01 directory.
- oniond download model <your_model_name> # Replace <your_model_name> with the name of the model that you want to download. The following uses the GLM-5.2 model as an example. The complete command is as follows:# oniond download model GLM-5.2-FP8
- After the model is downloaded, the expected response is as follows.
- Download Results:gid |stat|avg speed |path/URI======+====+===========+=======================================================600657|OK | 136MiB/s|GLM-5.2-FP8/model-00077-of-00141.safetensors524cb4|OK | 149MiB/s|GLM-5.2-FP8/model-00079-of-00141.safetensors852a47|OK | 150MiB/s|GLM-5.2-FP8/model-00080-of-00141.safetensors85febd|OK | 151MiB/s|GLM-5.2-FP8/model-00081-of-00141.safetensorsd52b39|OK | 150MiB/s|GLM-5.2-FP8/model-00082-of-00141.safetensorsf22383|OK | 150MiB/s|GLM-5.2-FP8/model-00083-of-00141.safetensors......b90f80|OK | 164MiB/s|GLM-5.2-FP8/model-00130-of-00141.safetensors47bff4|OK | 164MiB/s|GLM-5.2-FP8/model-00131-of-00141.safetensorsb9049f|OK | 163MiB/s|GLM-5.2-FP8/model-00132-of-00141.safetensors134e45|OK | 166MiB/s|GLM-5.2-FP8/model-00133-of-00141.safetensors373a9f|OK | 164MiB/s|GLM-5.2-FP8/model-00134-of-00141.safetensorsc0a415|OK | 165MiB/s|GLM-5.2-FP8/model-00136-of-00141.safetensors73cb8f|OK | 165MiB/s|GLM-5.2-FP8/model-00135-of-00141.safetensorseaccd1|OK | 165MiB/s|GLM-5.2-FP8/model-00137-of-00141.safetensors481532|OK | 66MiB/s|GLM-5.2-FP8/model.safetensors.index.json82a118|OK | 87MiB/s|GLM-5.2-FP8/tokenizer.json2be1f2|OK | n/a|GLM-5.2-FP8/tokenizer_config.jsone7d2ae|OK | 171MiB/s|GLM-5.2-FP8/model-00138-of-00141.safetensors6dde28|OK | 205MiB/s|GLM-5.2-FP8/model-00139-of-00141.safetensors820aaf|OK | 226MiB/s|GLM-5.2-FP8/model-00141-of-00141.safetensorsea5d87|OK | 244MiB/s|GLM-5.2-FP8/model-00140-of-00141.safetensorsStatus Legend:(OK):download completed.Download completed successfully (elapsed 396 s). Repo directory: /data01/GLM-5.2-FP8
Last updated: 2026.08.03 21:49:41
All rights reserved © Beijing Volcano Engine Technology Co., Ltd2024
