Docs

Torch Log Service

Machine learning task log collection

Collect machine learning task logs

This document provides an overview of the features and collection methods related to machine learning task logs.

Features overview

Volcano Engine Machine Learning Platform is an enterprise-grade, fully self-developed platform for machine learning application developers. It offers a comprehensive set of modeling tools, including development environment, custom training job, and online service, as well as high-performance model inference services supporting multiple frameworks. The platform supports the entire lifecycle workflow, from data hosting and code development to model training and model deployment.

The logging capabilities of the machine learning platform provide users with robust log processing features and simplify log query and search interfaces, better meeting users' needs for log management and queries.

The related configuration is as follows:

Configuration	Description
Log type	Machine learning logs.
Log storage location	Before binding a project and log topic in the machine learning platform, you must manually create the corresponding project and log topic in TLS. For details on managing log resources, please refer to Manage machine learning task log resources. Project: Custom Log topic: Custom
Log storage period	The default is 30 days.
Log shards	Two shards are preset. Automatic splitting of log topic shards is enabled by default. When write traffic is high, shards are split to meet data write requirements.
Index settings	Indexes are configured automatically.
Log fields	For a detailed list of log fields, please refer to Machine learning task log fields.

Collect machine learning task logs

For usage restrictions and instructions on enabling machine learning task logs in TLS, please refer to Log feature upgrade for details.

Machine learning task log fields

Online task fields

Field	Description
availability_zone	Availability zone.
flavor	Instance specification.
mlp_deployment	Online service deployment ID.
mlp_resource_group	Resource group ID.
mlp_resource_queue	Resource queue ID.
mlp_service	Online service ID.
mlp_service_instance	Online service instance ID.
mlp_task_source_type	Task source type.
role	Role.
role_index	Role index.
content	Original log content.

Development environment fields

Field	Description
tag_account_id	Account ID.
cluster_id	Cluster ID.
container_name	Container name.
mlp_devinstance	Development machine ID.
pod_name	Pod name.
devinstance_worker_id	Development machine worker node ID.
content	Original log content.

Custom training job fields

Field	Description
availability_zone	Availability zone.
flavor	Instance specification.
mlp_customtask	Custom task ID.
mlp_customtask_instance	Custom task instance.
mlp_pipeline	Workflow (Pipeline) ID.
mlp_resource_group	Resource group ID.
mlp_resource_queue	Resource queue ID.
mlp_task_source_type	Task source type.
role	Role.
role_index	Role index.
content	Original log content.

Custom training job PySpy fields

Field	Description
aggregated_by	Aggregation method.
dump_id	Sampling dump ID.
hash	Hash value.
instance	Instance.
is_dataloader	Whether it is a data loader.
owns_gil	Whether it holds the GIL.
pid	Process ID.
rank	Rank in distributed training.
thread_name	Thread name.
vpc_ір	VPC IP address.
content	Original log content.

Last updated: 2026.02.06 10:12:25