Torch Log Service
Torch Log Service
Torch Log Service
Document
Product News
Product Overview
Product Billing
User Guide
TLS Agent
AgentLoop
Resource management
Host group
Data collection
LogCollector collection
Introduction to LogCollector
Installation and upgrade
Collect host logs
Collect container logs
LogCollector plugins
Manage LogCollector
general reference
Data import
Cloud product log collection
TOS log collection
DBW log collection
VPN Log collection
VKE log collection
CLB log collection
CloudTrail log collection
CFW log collection
NTA log collection
ALB log collection
APMPlus log collection
veFaaS log collection
VPC log collection
veImageX log collection
TrafficRoute DNS log collection
DDoS log collection
WAF log collection
DCDN log collection
Multi-cloud security platform log collection
APIG log collection
Machine learning task log collection
CR log collection
Volcano Ark application lab log collection
FileNAS log collection
LLM Application Firewall Service log collection
vePFS log collection
Cloud security center log collection
EFS log collection
Data storage
Query and analysis
Configure log indexes
Query log
Analyze logs
Query and analyze logs using third-party tools
Associate external data source for log query
SQL function
SQL syntax
Visualization
Query and analysis charts
Summaries of query and analysis charts
Create dashboard
Use dashboard
Add dashboard filters and variables
Use other systems to analyze logs
Log application
Log applications for product observability
Log applications for large models observability
Coding Agent log app
ModelProvider log app
No-coding Agent log app
Log applications for basic service monitoring
Data processing
Data transformation
Syntax and functions
Global operation function
expression function
Data processing best practices
general reference
Data write processor
Data consumer processor
Data consumption and shipping
Ship logs to TOS
Ship logs to Kafka
Ship logs to Splunk
Alarm
Alarm policy
Alarm notification content
Alarm notification content template
alarm notification channel
Tracing service
Monitor Log Service
API reference
Project management API
Topic management API
Shard management API
Index management API
Log management API
Host group management API
Collection configuration management API
Data import API
Data processor API
Data transformation API
Data backflow API
SQL analysis API
Alarm management API
Consumer group management API
Kafka protocol consumption API
Tag management API
Trace storage API
Account management API
SDK reference
Go SDK
Java SDK
Python SDK
iOS SDK
C++ SDK
CLI reference
Best practices
Data collection
Query and analysis
permission management
Consumption and delivery
Visualization
Alarm
FAQs
Billing and cost FAQ
Data collection FAQ
Data storage FAQ
Index configuration FAQ
Query and analysis FAQ
Scheduled SQL tasks FAQ
Alarm FAQ
- Documentation
Torch Log ServiceUser GuideData collectionUpload logs using Logback Appender
Copy page
Download PDFData collection
Upload logs using Logback Appender
Copy pageDownload PDFUpload logs using Logback Appender
This topic explains how to upload logs to TLS using Logback Appender.
Background information
Logback is an open-source logging component designed by Log4j founder. With Logback, you can achieve the following features using just a configuration file, without modifying your application's code.
- Control the target destination for log delivery.
- Destinations include the console, files, GUI components, socket servers, NT event recorders, UNIX Syslog Daemons, and more.
- Control the output format for each log entry.
- Define the log level for each message to control the log generation process with greater precision.
Volcengine TLS Logback Appender is an appender based on the Logback framework. It is used to upload logs to the TLS service. The uploaded log format is as follows:
level: ERROR
location: com.volcengine.service.tls.logback.example.LogbackAppenderExample.main(LogbackAppenderExample.java:18)
message: this is an error log
throwable: java.lang.RuntimeException: xxx
thread: main
time: 2023-10-24T12:00+0000
log: 2023-10-24 12:00:00,000 ERROR [main] com.volcengine.service.tls.logback.example.LogbackAppenderExample: this is an error log
__source__: 192.168.1.100
__path__: sys.log
Field | Description |
Field | Description |
level | Log level. |
location | Code location of the log statement. |
message | Log content. |
throwable | Log exception information. This field is only shown if exception information is recorded. |
thread | Thread name. |
time | Log print time. You can use the timeFormat or timeZone parameter to configure the format and time zone for the time field. |
log | Custom log format. This field is shown only when encoder is set. |
__source__ | Log source. You can specify the log source in the configuration file. |
__path__ | Log data source path. You can specify the log data source path in the configuration file. |
Features
- Ship logs without disk: Logs are sent directly to the server over network.
- Zero code modification: Report logs to TLS by simply configuring XML files.
- High performance: It is built on TLS Java SDK Producer; supports asynchronous sending, high performance, and retry on failure; suits for high-concurrency scenarios.
Instructions
1 Add dependencies in your Maven project
Refer to the following example to add dependencies in your Maven project.
<dependency>
<groupId>com.volcengine</groupId>
<artifactId>ve-tls-logback-appender</artifactId>
<version>1.0.2</version>
</dependency>
2 Modify the configuration file
Refer to the example below to define the TLS Logback Appender in the logback.xml file. That is, binds the com.volcengine.service.tls.logback.LogbackAppender type in the file.
The sample code below registers two Logback Appenders: one outputs logs to the console, and the other sends logs to TLS. For detailed code, see logback-example.xml.
The sample code includes detailed annotations indicating required and optional parameters for the TLS Logback Appender. For custom configuration of TLS Java Producer, see LOG Java Producer.
<configuration>
<!--To prevent data loss in memory when the process exits, add this configuration.-->
<shutdownHook class="ch.qos.logback.core.hook.DelayingShutdownHook"/>
<!-- Configure the appender whose log output destination is TLS -->
<appender name="TLSLogbackAppender" class="com.volcengine.service.tls.logback.LogbackAppender">
<!-- Required configuration: TLS log service basic configuration -->
<endpoint></endpoint>
<region></region>
<accessKeyId></accessKeyId>
<accessKeySecret></accessKeySecret>
<!-- Required configuration: TLS project/topic configuration -->
<projectID></projectID>
<topicID></topicID>
<!-- Optional configuration: log data source configuration -->
<source></source>
<filename></filename>
<!-- Optional configuration: TLS Java Producer custom configuration -->
<totalSizeInBytes>104857600</totalSizeInBytes>
<maxThreadCount>50</maxThreadCount>
<maxBlockMs>60000</maxBlockMs>
<maxBatchSizeBytes>524288</maxBatchSizeBytes>
<maxBatchCount>4096</maxBatchCount>
<lingerMs>2000</lingerMs>
<retryCount>2</retryCount>
<maxReservedAttempts>3</maxReservedAttempts>
<!-- Optional configuration: custom log format via encoder pattern -->
<encoder>
<pattern>%d %-5level [%thread] %logger{0}: %msg</pattern>
</encoder>
<!-- Optional configuration: set time format -->
<timeFormat>yyyy-MM-dd'T'HH:mmZ</timeFormat>
<!-- Optional configuration: set time zone -->
<timeZone>Asia/Shanghai</timeZone>
<!-- Optional configuration: configure LevelFilter -->
<filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>INFO</level>
<onMatch>ACCEPT</onMatch>
<onMismatch>DENY</onMismatch>
</filter>
<!-- Optional configuration: configure mdcFields -->
<mdcFields>THREAD_ID,MDC_KEY</mdcFields>
</appender>
<!-- Configure log output destination as appender from Stdout -->
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger - %msg %X{THREAD_ID} %n</pattern>
</encoder>
</appender>
<!-- Can be used to obtain status from StatusManager -->
<statusListener class="ch.qos.logback.core.status.OnConsoleStatusListener"/>
<!-- Solve problems of repeatedly transmission under debug mode -->
<logger name="org.apache.http.impl.conn.Wire" level="WARN" />
<root>
<level value="DEBUG"/>
<appender-ref ref="TLSLogbackAppender"/>
<appender-ref ref="STDOUT"/>
</root>
</configuration>
Notice
- W recommend that you add the DelayingShutdownHook tag to prevent loss of a small amount of TLS Logback Appender data cached in memory when the process exits.
- Exceptions generated by TLS Logback Appender during operation are captured and stored in Logback’s BasicStatusManager class. You can view error information by configuring OnConsoleStatusListener or using other methods. For more information, see Logback-access.
3 Report logs using existing Logback log output code
The sample code in LogbackAppenderExample displays basic Logback log output configuration. After you create an instance based on Logger object, you can output logs to destinations configured in your logback.xml file by using use the trace, debug, info, warn, or error methods.
Notice
If you modify the logback.xml configuration file, you do not need to modify the existing log output code in the system.
package com.volcengine.service.tls.logback.example;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.slf4j.MDC;
public class LogbackAppenderExample {
private static final Logger LOGGER = LoggerFactory.getLogger(LogbackAppenderExample.class);
public static void main(String[] args) {
MDC.put("MDC_KEY","MDC_VALUE");
MDC.put("THREAD_ID", String.valueOf(Thread.currentThread().getId()));
LOGGER.trace("This is a trace log.");
LOGGER.debug("This is a debug log.");
LOGGER.info("This is a info log.");
LOGGER.warn("This is a warn log.");
LOGGER.error("This is an error log.");
}
}
Last updated: 2025.12.18 10:36:27
All rights reserved © Beijing Volcano Engine Technology Co., Ltd2024
