Docs

Volcengine Managed Service for Prometheus

Integration Guide

Integration overview

Volcengine Managed Service for Prometheus (VMP) features an excellent open source ecosystem affinity, supports cloud-native monitoring, and supports various SDKs and exporters provided by the open source community for monitoring various environments, middleware, and custom applications. This topic describes how to integrate your business with VMP by using different methods.

Feature modules and business processes

When your business is deployed on Volcano Engine, VMP provides comprehensive and user-friendly features to help you quickly integrate your business. If you use a self-built Internet data center (IDC) or an environment provided by a third-party cloud provider, VMP also offers features such as collector deployment scripts and network connectivity to help you build a multi-cloud platform for centralized metric observability with VMP at its core. The following section lists the common business deployment environments and integration steps.

Businesses can be deployed in the following environments:

Volcano Engine

Self-built IDC or third-party cloud provider

Businesses are integrated in the following steps:

Deploy a collection agent.

(Optional) Expose metrics.

Configure collection.

Query and use metrics.

The following figure shows the main feature modules and business processes of VMP. For business integration scenarios, the core features of VMP include:

VMP workspace: The primary storage unit for metrics. You need to specify a specific workspace to send or query metrics. For more details, refer to Selecting a workspace and Workspace type.

prometheus-agent: A collector that gathers infrastructure and business metrics of Volcengine Kubernetes Engine (VKE) clusters or self-built Kubernetes clusters. The collected metrics can be sent to VMP workspaces.

Integration center: The cloud service integration center provided by VMP for quick integration of Volcano Engine cloud services. For details, see Integration center.

Select an integration method based on the deployment environment

VKE integration

Deploy a collection agent: Deploy the prometheus-agent in a cluster and bind it to a workspace. For more details, refer to VKE integration.

Expose metrics. Enable the monitoring feature for different resources in the cluster and expose the corresponding metrics based on your business requirements.

Observability Object	References
Cluster observability	For more details about collecting the metrics of Kubernetes resources such as nodes, workloads, and containers, refer to Container Service Observation. For more details about collecting the metrics of control plane agents such as APIServer, refer to Control plane service observation. For more details about collecting AI-related metrics such as GPU and RDMA, refer to AI resource observation and Monitoring AI training jobs in VKE. For more details about collecting other advanced VKE metrics, refer to the following topics: DNS service observability Ingress service observation CNI network observation Image acceleration observability CR observability CSI storage observation Check self-healing observation
Business observability	For businesses deployed in the cluster, you can use the SDK provided by the Prometheus community to define business metrics during application development, and then monitor the internal status (including the number of requests and the number of queues) of the applications by using VMP. Supported monitoring scenarios include: Business count metrics: such as the number of requests and the number of queues Core logic latency: such as the time taken for external service requests Refer to the following documentation: Custom business integration Go application integration In addition, the Prometheus open source community supports SDKs for various development languages to allow you to select the appropriate SDK for business metric tracking and development. For more details, refer to the official documentation of Github.
Other observability	Kafka integration
Exporter observability	If the self-built middleware already exposes metrics in Prometheus format, you only need to configure service discovery. For many conventional applications, middleware, hardware, and systems, the Prometheus open source community offers multiple types of exporters for metric collection. For more details, refer to Exporters and integrations. For configuration methods, refer to the following topics: NGINX exporter integration Redis exporter integration MySQL exporter integration PostgreSQL exporter integration MongoDB exporter integration Elasticsearch exporter integration RabbitMQ exporter integration You can also create a custom exporter based on the client library provided by Prometheus. The Prometheus community officially supports languages including Go, Java, Python, and Ruby. For more details, refer to the official documentation of Github.

Configure custom collection: You can configure custom collection in the cluster. For more details, refer to Service discovery and How do I create CRD resources such as ServiceMonitor in a cluster.

ServiceMonitor: Configuring Service discovery through ServiceMonitor

PodMonitor: Configuring Service discovery through PodMonitor

Query metrics.

Elastic Compute Service (ECS)

Elastic Compute Service (ECS) also serves as a common business deployment environment. You can deploy your business in the ECS environment. VMP allows hosted collectors to collect basic resource data and business data on the target ECS instance.

ECS can be integrated with VMP in two different ways, of which the differences are described in the following table.

Integration methods	Integration description	Solution characteristics	References
Cloud Monitor integration	The Cloud Monitor integration tasks in the integration center are used to collect basic metrics of ECS from Cloud Monitor.	The configuration is easy. When you create a Cloud Monitor integration task, select ECS as the cloud service. However, the metric latency is relatively high because Cloud Monitor serves as an intermediary.	Cloud Monitor integration
ECS integration	The ECS integration tasks in the integration center are used to directly collect metrics of ECS. Basic metrics: cloud-monitor-agent or open source node-exporter can be used to obtain basic ECS metrics. Business metrics: Custom business metrics deployed in ECS can be obtained.	Compared with Cloud Monitor integrating tasks, the collection link is shorter with lower latency. It is suitable for scenarios that are sensitive to metric latency. ECS metrics can be directly collected and sent to VMP workspaces. However, you need to manually enable the metrics interface in ECS, whose configuration is complex. It is compatible with open source node-exporter, and you can deploy node-exporter on ECS to collect metrics. In addition to basic metrics, it also supports the collection of business metrics deployed in ECS.	ECS integration Monitoring ECS instances Monitoring applications in ECS instances

Other cloud services

If you need to use VMP to monitor other cloud services of Volcano Engine, you can use one of the following methods of integrating a cloud service with VMP:

Cloud Monitor integration: For cloud services that are integrated with Cloud Monitor, you can integrate Cloud Monitor with VMP to deliver a one-stop, end-to-end collection and storage, and comprehensive centralized monitoring management experience. For more details, refer to Cloud Monitor integration.

Cloud services already integrated with VMP: Some cloud services of Volcano Engine are already integrated with VMP and you only need to bind a workspace. For more details, refer to Other cloud service integration.

Self-built IDC/Third-party clouds

When your business is deployed in a self-built IDC or a third-party cloud, you can also use VMP to monitor basic resources and businesses. Procedure:

Establish network connections: Connect the networks of self-built IDCs or third-party clouds to the network of Volcano Engine over the public network or dedicated connection to ensure that business environments can access VMP workspaces as expected. For more details, refer to Accessing a workspace over the public network by using a proxy and Accessing a workspace across clouds by using the dedicated connection and PrivateLink.

warning

Due to the long network path and poor reliability of public network access, it is recommended to only use this method during initial validation and to avoid using the method in an online production environment.

Deploy a collection agent: Use the deployment script to deploy the VM Agent collector in a custom cluster. For more details, refer to Monitoring custom Kubernetes clusters.

tip

You can perform the following steps to deploy the VM Agent collector:

Obtain the script. For more details, refer to Obtaining the script.

Use the script. For more details, refer to Using the script.

Deploy the VM Agent collector. For more details, refer to Deploying VM Agent.

Perform O&M. For more details, refer to the following topics:

Managing VM Agent

Agent scale-out

Alerting and O&M suggestions.

(Optional) Expose metrics. Expose the basic and business metrics of Kubernetes clusters based on your business requirements.

Observability Project	References
Cluster basic observability	When you deploy the VM Agent collector by using the script, node-exporter (node metrics) and kube-state-metrics ( operational status metrics of Kubernetes resources) are also installed by default. kubelet/kubelet-cadvisor (container runtime statistics metrics) is also installed by default. Therefore, after you deploy the prometheus-agent by using the script, it will automatically collect container basic metrics of the custom Kubernetes cluster. For other metrics, you need to deploy the exporter based on the actual cluster environment and configure collection. For more details, refer to Service discovery. For example, if you need to monitor the GPU in your custom cluster, refer to Monitoring GPU in custom Kubernetes clusters.
Business observability & middleware observability	The configuration method is consistent with that of VKE cluster observability. For more details, refer to the preceding content.

Configure collection.

Third-party cloud service

If your business is deployed on a third-party cloud service, you can also use VMP to monitor the cloud service and business. For an example, refer to Monitoring Tencent Cloud resources.

Query metrics

Query methods

VMP supports the query syntax of open source Prometheus. You can query service metrics by using PromQL statements. You can query metrics by using one of the following methods:

Query metrics by writing code. For more details, refer to Using the Go language to write and query data in workspaces.

Query metrics by using the Explore feature of VMP. For more details, refer to Querying metrics.

Query metrics by using Grafana.

tip

You can deploy custom Grafana in VKE clusters and ECS instances. For more details, see Deploying Grafana in a VKE cluster and accessing workspaces, and Delivering instance monitoring data to VMP.

Metric list

The following table shows the metrics of common Volcano Engine services.

Product name	Metric list
Volcengine Kubernetes Engine (VKE)	Basic metrics: API server metrics For more details of metrics, refer to the following topics: official documentation of node-exporter official documentation of kube-state-metrics official documentation of cAdvisor official documentation of DCGM
Elastic Compute Service (ECS)	For more details, refer to ECS metrics.
Cloud Monitor	For more details, refer to Cloud Monitor metric queries. tip If Cloud Monitor is integrated with VMP, when the metrics from Cloud Monitor are written into a VMP workspace, the metrics are appended with the suffix SubNamespace_Namespace. The metric names are in the following format: MetricName_SubNamespace_Namespace. Example: CPUUser_Instance_VCM_ECS.
Microservices Engine	For more details, refer to Microservices Engine metrics.
API Gateway	Refer to API Gateway metrics.

Monitoring dashboard

VMP provides monitoring dashboards for common services, including:

VKE cluster resource dashboard

Preset cloud service dashboard. For more details, refer to Preset dashboard.

Use Grafana. You can deploy Grafana and construct monitoring dashboards. You can deploy Grafana in VKE clusters or self-built Kubernetes clusters.

Deployment: Deploying Grafana in a VKE cluster and accessing workspaces.

Advanced usage

Cross-workspace query: Creating an aggregation query workspace.

Cross-region query: Achieving cross-region access to workspaces by using CEN and PrivateLink.

Common FAQs

How do I troubleshoot if I cannot query the expected metric?

Refer to Why am I unable to view expected metrics?

How do I expose metrics?

Prometheus mainly pulls metric endpoints on a scheduled basis.

If your business references the Prometheus SDK, you can develop by yourself to expose a metrics port.

If the service does not expose metrics, you can use Exporter to call the service interface or data to obtain original metrics and then convert the metrics into Prometheus format to expose a metrics port.

How do I collect metrics?

In most cases, VMP uses the prometheus-agent to collect metrics and send the metrics to the specified workspace.

How do I know which metrics of which services to collect?

Native Prometheus supports the configuration of various service discovery jobs. For more details, refer to the official documentation of Prometheus.

In the Kubernetes cluster, prometheus-operator is often introduced to use Custom Resource Definition (CRD) to define the collection configuration. For more details, refer to the official documentation of Prometheus.

Commonly used CRD resources include:

ServiceMonitor

PodMonitor

In addition, you can also quickly configure collection by using service/pod annotations. For more details, refer to Service discovery.

Last updated: 2026.10.10 10:16:50