Docs

Torch Log Service

Data collection

Collection overview

VolcanoEngine Torch Log Service(TLS) provides various log collection methods, including LogCollector and SDK. These collection methods are suitable for collecting log data from hosts, containers, mobile devices, cloud products, and other data sources. This topic introduces collection methods and solutions that are supported by TLS.

Features overview

TLS supports collection methos like using log collection tool LogCollector and multi-language SDKs. TLS is compatible with multiple open-source protocol frameworks such as Kafka and OpenTelemetry. It also supports collection of log data from host applications, K8s clusters, mobile clients, VolcanoEngine cloud products, and other cloud providers.

To select log collection method, you can go to TLS console homepage or the log collection page under the specified log topic. Through simple configuration, log data can quickly access VolcanoEngine TLS. After writing logs into a log topic and setting up indexes, you can preview collected log data in real time on the log query page. You can also use the query and analysis syntax provided by TLS for further search and analysis to realize data visualization and other advanced features.

Collection methods

Collection methods	Description
LogCollector	You can use LogCollector, the log collection tool provided by TLS, to collect application logs from hosts or containers. It supports multiple collection modes including single-line and full-text, and offers various plugins for data masking. For the process to quickly integrate with TLS via LogCollector, see Overview.
Kafka protocol	TLS supports uploading log data to the server via Kafka protocol. You can collect log data using the Kafka Producer SDK and upload it to TLS via Kafka protocol. For detailed information, see Upload logs using the Kafka protocol.
OpenTelemetry protocol	Based on the native OpenTelemetry protocol and standards, trace storage supports access, storage, query, analysis, and visualization of various trace data. For detailed information, see Trace Feature Overview.
API	TLS provides OpenAPI interfaces for uploading logs. This allows you to send structured logs to a specified log topic. For detailed information, see PutLogs.
SDK	TLS provides multilingual SDKs. Via SDK, you can call OpenAPI interfaces PutLogs to upload logs to the specified log topic. For detailed information, see Overview.

Collection solutions

Data source type	Log source	Description	Related documentation
OS system	Linux	Collect Linux operating system logs using LogCollector. Supports Volcano Engine cloud servers, self-hosted IDC, and servers from other cloud service providers.	Install LogCollector (host machine)
		K8s cluster	Supports log collection from self-built K8s clusters, VolcEngine container service, or other cloud providers' K8s clusters. Supported data types: Text logs Standard output and standard error	Container collection overview
		iOS	Collect log data via iOS SDK provided by TLS.	iOS SDK overview
		Android	Collect log data via Android SDK provided by TLS.	Android SDK overview
Application	Application log output	Collect output logs from various systems and applications via LogCollector.	Install LogCollector (host machine)
		Web page	Collect web application log data via WebTracking.	Use WebTracking to collect logs
		Trace	Based on the native OpenTelemetry protocol and standards, trace storage supports access, storage, query, analysis, and visualization of various trace data.	Trace Feature Overview
		Logstash	You only need to simply modify the Logstash configuration file and then you can upload log data to TLS by using the built-in Logstash Kafka output plugin (logstash_output_kafka).	Upload logs using the Kafka protocol
Language	Logback Appender	Supports importing logs to TLS via simple configuration of XML files. It has features like asynchronous sending, high performance, and retry on failure. Therefore, it is suitable for high-concurrency scenarios.	TLS Logback Appender
Data import	Torch Object Storage TOS	Supports structuring the data that is stored in TOS and saving the data in TLS. Currently, only historical data is supported to import.	Import logs from TOS
		Alibaba Cloud Log Service SLS	Supports importing data from Logstore in Alibaba Cloud simple log service(SLS). It also supports importing SLS log collection configurations.	Import logs from SLS Import SLS collection configuration
		Elasticsearch cluster	Import data from VolcanoEngine or self-managed Elasticsearch clusters, or Elasticsearch clusters from other cloud service providers. Currently, only historical data is supported to import.	Import data from Elastacsearch
Volcano Engine cloud products	VolcEngine cloud products such as CDN and TOS	TLS supports collecting log data from various cloud products such as container services and object storage. For example, container logs and access logs can be collected.	Cloud product log collection

Billing description

When writing logs to TLS, fees generate in terms of log storage, write traffic, and the number of shards. If you write log data via API or SDK, additional fees will be charged based on service request count. For detailed billing rules, see Billing item.

Cloud service logs include a certain amount of free quota. For specific billing strategies, refer to the product documentation of each cloud service.

Limits

When importing log data into TLS, note the following limits: For basic resource quotas, read/write traffic limits, and query analysis concurrency restrictions of TLS, see Limits.

LogCollector limits

Limits

Trace Feature Overview

PutLogs

Last updated: 2025.12.03 18:42:00