Torch Log Service
Torch Log Service
Torch Log Service
Document
Product News
Product Overview
Product Billing
User Guide
TLS Agent
AgentLoop
Resource management
Host group
Data collection
LogCollector collection
Introduction to LogCollector
Installation and upgrade
Collect host logs
Collect container logs
LogCollector plugins
Manage LogCollector
general reference
Data import
Cloud product log collection
TOS log collection
DBW log collection
VPN Log collection
VKE log collection
CLB log collection
CloudTrail log collection
CFW log collection
NTA log collection
ALB log collection
APMPlus log collection
veFaaS log collection
VPC log collection
veImageX log collection
TrafficRoute DNS log collection
DDoS log collection
WAF log collection
DCDN log collection
Multi-cloud security platform log collection
APIG log collection
Machine learning task log collection
CR log collection
Volcano Ark application lab log collection
FileNAS log collection
LLM Application Firewall Service log collection
vePFS log collection
Cloud security center log collection
EFS log collection
Data storage
Query and analysis
Configure log indexes
Query log
Analyze logs
Query and analyze logs using third-party tools
Associate external data source for log query
SQL function
SQL syntax
Visualization
Query and analysis charts
Summaries of query and analysis charts
Create dashboard
Use dashboard
Add dashboard filters and variables
Use other systems to analyze logs
Log application
Log applications for product observability
Log applications for large models observability
Coding Agent log app
ModelProvider log app
No-coding Agent log app
Log applications for basic service monitoring
Data processing
Data transformation
Syntax and functions
Global operation function
expression function
Data processing best practices
general reference
Data write processor
Data consumer processor
Data consumption and shipping
Ship logs to TOS
Ship logs to Kafka
Ship logs to Splunk
Alarm
Alarm policy
Alarm notification content
Alarm notification content template
alarm notification channel
Tracing service
Monitor Log Service
API reference
Project management API
Topic management API
Shard management API
Index management API
Log management API
Host group management API
Collection configuration management API
Data import API
Data processor API
Data transformation API
Data backflow API
SQL analysis API
Alarm management API
Consumer group management API
Kafka protocol consumption API
Tag management API
Trace storage API
Account management API
SDK reference
Go SDK
Java SDK
Python SDK
iOS SDK
C++ SDK
CLI reference
Best practices
Data collection
Query and analysis
permission management
Consumption and delivery
Visualization
Alarm
FAQs
Billing and cost FAQ
Data collection FAQ
Data storage FAQ
Index configuration FAQ
Query and analysis FAQ
Scheduled SQL tasks FAQ
Alarm FAQ
- Documentation
Torch Log ServiceGetting StartedCreate resources
Copy page
Download PDFGetting Started
Create resources
Copy pageDownload PDFCreate resources
Basic log resource management mainly includes projects and topics. One log project can contain multiple topics. This topic introduces the steps to creating basic log resources in torch log service (TLS) console.
Prerequisites
- A Volcano Engine account is created and real name verification is completed. For detailed steps, see Account registration and Real name authentication.
- The ECS instance on Volcanoengine that collects logs is obtained, and cloud assistant agent is installed and running properly. For instructions on installing cloud assistant, see Install cloud assistant client.
- The Volcanoengine ECS instance is in the same region as the TLS service. In this example, the North China 2 (Beijing) region is selected.
Create a host group
- Log in to Torch log service console.
- In the top navigation bar, select the North China 2 (Beijing) region where TLS is located.
- On the left navigation bar, select Resource management > Host group.
- On the Host group management page, click Create host group.
- In the Create host group dialog box, complete the following settings, and then click Create.
- ParametersExample valueDescriptionHost group nameHost-group-aName of the host group.Host group typeIP addressThe type of host group.Volcanoengine ECSEnableEnable and add the Volcanoengine ECS instances to be collected to the current host group.
- Only supports adding ECS instances with Cloud Assistant installed and both Cloud Assistant and the instance are in Running status.
- The system automatically installs LogCollector for the selected ECS instances.
- ECS instances that have started installing LogCollector cannot be stopped or uninstalled. Proceed with caution.
IP address192.168.0.100Server IP address.Once ECS is enabled and LogCollector is successfully installed, the system automatically adds the ECS IP address.LogCollector service logEnableAfter you enable this feature, the LogCollector client records runtime status and collection progress information in real time during operation.LogCollector auto upgrade/After creating the host group, go to the LogCollector management tab on the host group details page to enable this feature.IAM Projectdefault (default project)Select the IAM project to which the host group belongs.You can click Create project to create a new IAM project.
Create a project
- After the host group is created successfully, in the left navigation bar, select Resource management > Project.
- On the Project page, click Create project.
- In the Create project dialog, complete the following configurations, and then click Create.
- ParametersExample valueDescriptionLog project nameproject-aThe name of the log project.Only supports Chinese, lowercase letters, numbers, underscores (_), and hyphens (-).DescriptiontestBrief description of the log project.IAM Projectdefault (default project)Select the IAM project to which the project belongs.You can click Create project to create a new IAM project.Tags/The tags for the log project. In this example, no tags are set.Tags are used to identify and classify resources. You can categorize projects and topics by tags to facilitate searching and resource aggregation.
Create a topic
- After the project is created, click Continue: Create topic in the prompt dialog box.
- In the Create topic dialog box, complete the following configurations.
- Enter the basic configuration.
- ParametersExample valueDescriptionTopic nametopic-aName of the log topic.Record external IPEnableThis parameter determines whether to enable or disable the feature to record external IP addresses. By default, it is enabled. If enabled, the following metadata fields are automatically added to the log entries:
- __tag____client_ip__: the public IP address of the device where the log originates.
- __tag____receive_time__: The timestamp when the log reaches the server; it is in 10-digit Unixtime format.
DescriptiontestA brief description of the log topic.Tags/The tags for the log topic. In this example, no tags are set.
- Configure log storage time.
Warning
Logs that exceed specified storage time will be automatically deleted.
- ParametersExample valueDescriptionStandard storage30 daysLog retention period in the TLS standard storage layer.The unit is days, and the valid range is 1 to 3650. Specify 3650 days for permanent storage.
- When infrequent access storage is enabled, the standard storage retention period ranges from 7 to 3650 days.
- When infrequency access storage and archive storage are enabled, the standard storage period ranges from 7 to 3650 days.
- When archive storage is enabled but infrequency access storage is disabled, the standard storage duration ranges from 30 to 3650 days.
Infrequent access storageDo not enableLog retention period in the infrequent access layer of TLS.The unit is days, and the valid range is 1 to 3650. Specify 3650 days for permanent storage.Archive storageDo not enableLog retention period in the archive layer of TLS.The unit is days, and the valid range is 60 to 3650. If it is set to 3650, the storage time becomes permanent storage.
- Fill in shard configuration.
- ParametersExample valueDescriptionNumber of shards2The number of log shards.Each shard provides write capacity of 5 MiB/s and 500 times/s, and read capacity of 20 MiB/s and 100 times/s.After creation, modifying the number of shards is not supported. However, the overall read and write capacity of the log topic can be increased through automatic split feature.Automatic shardingEnableThis parameter determines whether to enable or disable automatic sharding.
- (default) Enabled: When the amount of written data exceeds the existing shard service capacity continuously for 5 minutes, TLS automatically splits shards based on data volume to meet business needs. However, the number of shards after splitting can not exceed the maximum number of splits. New shard created within the last 15 minutes will not split automatically.
- Disabled: automatic sharding is not enabled.
Maximum number of splits100Maximum number of shard splits, i.e., the maximum number of all read-write state shards after shard splitting.- The number of maximum shards is required to set only when Automatic sharding is enabled.
- If the maximum number of splits is less than or equal to the number of log shards, auto-splitting is not available.
- Fill in advanced configuration.
- ParametersExample valueDescriptionWebTrackingEnableEnabling WebTracking allows you to quickly collect frontend tracking data through WebTracking.NoticeAfter enabling WebTracking for a log topic, data written via the WebTracks API does not require authentication. This grants anonymous write access to the public network, but it may cause dirty data. For more details, see Use WebTracking to collect logs.
- Click create.
Next step
Last updated: 2025.12.02 16:56:21
All rights reserved © Beijing Volcano Engine Technology Co., Ltd2024
