Torch Log Service
Torch Log Service
Torch Log Service
Document
Product News
Product Overview
Product Billing
User Guide
TLS Agent
AgentLoop
Resource management
Host group
Data collection
LogCollector collection
Introduction to LogCollector
Installation and upgrade
Collect host logs
Collect container logs
LogCollector plugins
Manage LogCollector
general reference
Data import
Cloud product log collection
TOS log collection
DBW log collection
VPN Log collection
VKE log collection
CLB log collection
CloudTrail log collection
CFW log collection
NTA log collection
ALB log collection
APMPlus log collection
veFaaS log collection
VPC log collection
veImageX log collection
TrafficRoute DNS log collection
DDoS log collection
WAF log collection
DCDN log collection
Multi-cloud security platform log collection
APIG log collection
Machine learning task log collection
CR log collection
Volcano Ark application lab log collection
FileNAS log collection
LLM Application Firewall Service log collection
vePFS log collection
Cloud security center log collection
EFS log collection
Data storage
Query and analysis
Configure log indexes
Query log
Analyze logs
Query and analyze logs using third-party tools
Associate external data source for log query
SQL function
SQL syntax
Visualization
Query and analysis charts
Summaries of query and analysis charts
Create dashboard
Use dashboard
Add dashboard filters and variables
Use other systems to analyze logs
Log application
Log applications for product observability
Log applications for large models observability
Coding Agent log app
ModelProvider log app
No-coding Agent log app
Log applications for basic service monitoring
Data processing
Data transformation
Syntax and functions
Global operation function
expression function
Data processing best practices
general reference
Data write processor
Data consumer processor
Data consumption and shipping
Ship logs to TOS
Ship logs to Kafka
Ship logs to Splunk
Alarm
Alarm policy
Alarm notification content
Alarm notification content template
alarm notification channel
Tracing service
Monitor Log Service
API reference
Project management API
Topic management API
Shard management API
Index management API
Log management API
Host group management API
Collection configuration management API
Data import API
Data processor API
Data transformation API
Data backflow API
SQL analysis API
Alarm management API
Consumer group management API
Kafka protocol consumption API
Tag management API
Trace storage API
Account management API
SDK reference
Go SDK
Java SDK
Python SDK
iOS SDK
C++ SDK
CLI reference
Best practices
Data collection
Query and analysis
permission management
Consumption and delivery
Visualization
Alarm
FAQs
Billing and cost FAQ
Data collection FAQ
Data storage FAQ
Index configuration FAQ
Query and analysis FAQ
Scheduled SQL tasks FAQ
Alarm FAQ
- Documentation
Torch Log ServiceUser GuideData consumption and shippingShip logs to SplunkShip logs to Splunk using Add-On plugin
Copy page
Download PDFShip logs to Splunk
Ship logs to Splunk using Add-On plugin
Copy pageDownload PDFShip logs to Splunk using Add-On plugin
This topic describes the basic method for shipping TLS topic data to Splunk using the plugin.
Prerequisites
- We recommend using IAM users for access authorization. Before using an IAM user, ensure that the Volcanoengine primary account has created an IAM user and granted consumption-related permissions. For more details, see Authorizable operations.
- Access Key for the current login account has been obtained. For more information, see Create Access Key.
- A log project and log topic are created. For detailed instructions, see Create resources.
- The TLS Volcengine Add-On package has been obtained. For more information, see the Volcengine TLS Add-On download page.
- Check environment dependencies:
- System: Only Linux is supported.
- Python: 3.9.x and later versions.
- Splunk: 10.0.x and later versions.
Install Splunk Enterprise
Prepare the Linux environment and install the Splunk Enterprise edition. For details, see the Splunk website.
Install the TLS plugin.
- Log in to Splunk Enterprise Web.
Description
Visit http://<host name or ip address>:8000 in your browser, enter your account and password, and log in to Splunk Enterprise Web.
- Select Apps > Manage Apps to enter the application management page.
- Click Install App From File to go to the app installation page.
-
- In the pop-up window, select and upload the Volcengine TLS Add-On plugin package downloaded in advance.
-
- After successful installation, you can view the Volcengine Torch Log Service Add-on for Splunk on the application homepage.
-
Configure access credentials for TLS.
- Click the app name to open the application page.
- Click Configuration to access the AccessKeys and log level settings.
- In AccessKeys, click Add and follow the instructions on the interface to add your Volcanoengine account information.
-
- ParametersDescriptionParametersDescriptionNameCustom identifier name.AccessKeyThe access key ID (AK) of your Volcanoengine account.SecretKeyThe secret access key (SK) of your Volcanoengine account.SecurityTokenTemporary credentials. Provide this if you obtained the access credentials through role assumption. See IAM Overview for details.
- Once added, you can view relevant information on the list page.
-
Create Input task
- Click Inputs to go to the Input task tab.
- Click Create New Input to configure Input task information.
-
- Configure the following information in the Input task pop-up window.
- Basic information:
- ParametersDescriptionParametersDescriptionNameEnter the task name, which will be used as the TLS consumer name.GroupTLS consumption group nameProjectTLS project ID.TopicsTLS topic ID. Use “,” to separate multiple IDs.RegionThe TLS region to accessEndpointTLS service access address.AccessKeyVolcano access credential, which refers to the credential configured in the above steps.IndexSplunk Indexer used for data ingestion.IntervalIf the task exits abnormally, it will retry periodically based on this interval.
-
- (Optional) Configure Advanced Settings:
- ParametersDescriptionParametersDescriptionConsumeFromThe starting position for consumption takes effect only when no record exisit in the consumer group.
- begin (default): consumption starts from the earliest position.
- end: consumption starts from the latest position.
- Unix timestamp in seconds: Consumption starts from the specified time.
HeartbeatIntervalThe interval for heartbeat execution. The unit is seconds. The default value is 20 seconds.CheckpointIntevalThe interval for submitting consumption progress. It is measured in seconds and defaults to 5 seconds.FetchIntervalThe interval for message retrieval. It is measured in milliseconds and defaults to 200 ms.CompressionThe compression method used for message retrieval.- lz4 (default)
- zlib
- none: No compression is used.
OrderedConsumeEnable ordered consumption.- Disabled (default).
- Enabled: Prioritizes consuming data from the parent shard during shard splitting before consuming from the newly created child shards.
-
- View Input task
- After the task is created, only the configuration information is displayed on the list page. You can click the collapse button to view all configurations.
- Click the Active button to pause or resume the task. After the task runs successfully, you can query the data on the Search tab.
-
Query data synchronized by the Input task.
- Click Search to enter the query tab.
- Run the following command to query the corresponding log topic for data.
- For example, to query the Input task named volc_tls_test, you need to replace it with the value set in your actual environment.
- * source="volcengine_tls_input://volc_tls_test"
-
Description
If no data is found, you can troubleshoot as follows:
- Check whether any data has been written in the Topic of TLS.
- Check if the log timestamps in TLS are too old and you can try increasing the query time.
Last updated: 2026.05.06 14:28:30
All rights reserved © Beijing Volcano Engine Technology Co., Ltd2024
