This section introduces the TLS LLM observability log application for large models, available in the TLS application marketplace.

Overview

TLS LLM is a Python and Node.js LLM tracing SDK built around OpenTelemetry. It supports manual instrumentation, decorators (@trace), and automatic instrumentation for OpenAI clients, and is used to record GenAI semantic information such as LLM invocation input/output and tool calls.

After you integrate the TLS LLM observability log application, a dedicated TLS LLM Trace log topic is automatically created in TLS, generating visual charts and displaying session, trace, token, and tool invocation data to achieve tracing and observability. It can be used in the following scenarios:

Observability	Description
Performance analysis	Model invocation latency distribution Tool execution time ranking Token consumption trend
Error tracking	View the complete invocation chain Locate the error stage Analyze the cause of the error
Cost optimization	Identify high-consumption conversations Optimize prompts to reduce tokens Select a more cost-effective model

Prerequisites

Category	Description
Software limits	The server for deploying TLS LLM must have NPM and Node.js 22 or later installed.
Region restriction	The server for deploying TLS LLM must be in the same region as TLS to ensure network connectivity.
Authentication method	Obtain the user's Access Key ID, Secret Access Key, or APIKey according to the authentication method. AK/SK: AccessKey ID and AccessKey Secret for Volcano Engine accounts. Obtain them on the Volcano Engine console Key Management page. We recommend using IAM user keys. Ensure this user has the required permissions. For details, see IAM overview. Supports automatic creation of TLS resources (project/app/topic) and auto-generates log topic IDs. API Key: A key credential for requesting Volcano TLS. It is used only for reporting Log, Metric, and Trace data. Enforces strict permissions and does not support auto creation of TLS resources. It is suitable for multi-data center environments and strict permission management scenarios. Notice API Key is in closed beta. If needed, contact your account manager for access to the allowlist.

1. Integrate the TLS LLM log application

Log in to TLS console.

Select the region where log service is located from the top navigation bar.

In the left navigation bar, select AgentLoop > AI observability.

On the AI observability page, click Integrate AI observability.

On the Integrate AI observability page, create an application.

Click the TLS LLM icon and select the TLS LLM application.

Configure the application integration parameters, and then click Create application.

Parameters	Description
Region	The region where the accessed log application data is stored.
Log project	Select the project where the log application data is stored. Notice After the log application is successfully created, a log topic with the TLSLLM tag will be automatically created in the project to store trace data. Details of this log topic are as follows: Log topic name: application name_trace Number of shards: 2 Log storage period: Standard storage for 90 days Index: Key-value index configured automatically.
Application name	Enter the name of the TLS LLM log application. Length must be between 3 and 64 characters. Only supports Chinese, lowercase letters, numbers, underscores (_), and hyphens (-). Can not start or end with a hyphen (-) or underscore (_). Cannot start with tls_, tos_, or certain reserved prefixes.
Description	Description of the log application. Length limit is 0–128 characters.

After creating the application, proceed to the next step to obtain the usage guide and integrate the log application.

2. Integrate TLS LLM application data

Obtain the usage guide for integrating TLS LLM application data.

Go to the Integration configuration page.

Obtain when accessing the log application
Obtain after accessing the log application
When creating a TLS LLM log application in AgentLoop, follow the on-screen instructions to obtain the corresponding usage guide.
Refer to 1. Integrate the TLS LLM log application to access the TLS LLM log application.
On the Access AI Observability page, in Access configuration, configure the following information to obtain the usage guide.
Log in to TLS console.
Select the region where log service is located from the top navigation bar.
In the left navigation bar, select AgentLoop > AI Monitoring.
On the AI Monitoring page, access the usage guide page as follows.
Click the access configuration icon in the Operation column of the target TLS application.
Click the name of the target TLS application, then click Access management.

On the Integration configuration page, select the authentication method, configure the following information, and obtain the usage guide.

AK/SK access
API Key access
For Access method, select AK SK access.
For Connection method, select Public network or Volcano Engine internal network as appropriate.
In Usage guide, copy the auto-generated usage guide.
For Access method, select API Key access.
For Connection method, select Public network or Volcano Engine internal network as appropriate.
For API Key, select the API Key that has been created.
In Usage guide, copy the auto-generated usage guide.

Usage guide example (using AK/SK integration method):

Public network
Volcano Engine internal network

### Python：TLS LLM Observer（AK/SK）

Install the base SDK：

```bash
pip install tls-llm-observer
```

If automatic tracing for OpenAI or OpenAI-compatible clients is required:

```bash
pip install "tls-llm-observer[openai]"
```

Configure TLS OTLP/HTTP report:

```bash
export TLS_TRACE_ENABLED=true
export TLS_TRACE_CAPTURE_CONTENT=true
export OTEL_SERVICE_NAME=tls-llm-python

export TLS_TRACE_OTEL_ENDPOINT=https://The system autopopulates the current endpoint:4318/v1/traces

export TLS_TRACE_OTEL_TRACE_TOPIC_ID=The system autopopulates the trace-topic-id
export TLS_TRACE_OTEL_REGION=The system autopopulates the current region
export TLS_TRACE_OTEL_AK=<Requires manual replace of the actual AccessKeyID>
export TLS_TRACE_OTEL_SK=<Requires manual replace of the actual SecretAccessKey>
```

Inject AK/SK through deployment platform Secret. Do not embed credentials in source code or Dockerfile.
OpenAI integration sample:

```python
from tls_llm_observer import TraceClient
from tls_llm_observer.openai import OpenAI

trace_client = TraceClient()
client = OpenAI(
trace_client=trace_client,
api_key="<model-api-key>",
base_url="<openai-compatible-endpoint>",
)

try:
response = client.chat.completions.create(
model="<model-name>",
messages=[{"role": "user", "content": "Hello"}],
)
finally:
trace_client.flush()
trace_client.shutdown()
```

### TypeScript / Node.js：TLS LLM Observer（AK/SK）

Applicable to Node.js 18 and later. Install TLS LLM Observer. If tracing for OpenAI or OpenAI-compatible client is required, install `openai`：

```bash
npm install @volcengine/tls-llm-observer openai
```

Configure TLS OTLP/HTTP report:

```bash
export TLS_TRACE_ENABLED=true
export TLS_TRACE_CAPTURE_CONTENT=true
export OTEL_SERVICE_NAME=tls-llm-node

# TLS OTLP/HTTP public access endpoint
export TLS_TRACE_OTEL_ENDPOINT=https:// The system autopopulates the current endpoint:4318/v1/traces

# TLS Trace Topic and AK/SK authentication
export TLS_TRACE_OTEL_TRACE_TOPIC_ID=The system autopopulates trace-topic-id
export TLS_TRACE_OTEL_REGION=The system autopopulates the current region
export TLS_TRACE_OTEL_AK=<Requires manual replace of the actual AccessKeyID>
export TLS_TRACE_OTEL_SK=<Requires manual replace of the actual SecretAccessKey>
```

Inject Inject AK/SK using the Secret capability of the deployment platform. Do not write secrets into source code, images, or code repositories.
SDK will auto-generate the following according to the above configurations:

- `x-tls-otel-tracetopic`

- `x-tls-otel-region`

- `x-tls-otel-ak`

- `x-tls-otel-sk`

OpenAI integration sample:

```typescript
import OpenAI from 'openai';
import { TraceClient } from '@volcengine/tls-llm-observer';
import { instrumentOpenAI } from '@volcengine/tls-llm-observer/openai';

const traceClient = new TraceClient();

try {
const openai = instrumentOpenAI(
new OpenAI({
apiKey: '<model-api-key>',
baseURL: '<openai-compatible-endpoint>',
}),
{ traceClient },
);

const response = await openai.chat.completions.create({
model: '<model-name>',
messages: [
{
role: 'user',
content: 'Hello',
},
],
});

console.log(response.choices[0]?.message.content);
} finally {
const result = await traceClient.shutdown();

if (!result.completed) {
console.error('TLS Trace report failed', result);
}
}
```

(AK/SK method only) Modify the authentication information in the usage guide.

Enter the obtained Access Key ID and Secret Access Key into the AK and SK fields in the usage guide, respectively.

Integrate TLS LLM observability logs into the log application.

Log in to the server where TLS LLM is deployed.

Use the information obtained from the usage guide to install the TLS plugin.

After the plugin installation is complete, restart the TLS LLM service.

Return to the TLS console to view the observability chart data.

Monitoring chart details

The monitoring chart details for the TLS LLM application in TLS are as follows:

Type	Chart
Homepage	Request overview: Session count, conversation count, model request count, tool invocation count, request trends Cost overview: Total Tokens, Input Tokens, Output Tokens, Cache Read Tokens, Reasoning Tokens, Token consumption trend, model invocation trend Tool overview Number of successful invocations, number of failed invocations, tool invocation duration P90, tool invocation Top 10, tool invocation failure Top 10, tool invocation trend Model performance overview Model invocation distribution Top 10, model invocation duration P90, provider invocation distribution, provider invocation duration P90
Trace analysis	Homepage Total trace count, total token consumption, average tokens, P90 duration Trace call chain analysis Display Input, Output, Trace ID, Total Tokens, Input Tokens, Output Tokens, duration (s), start time, end time, and more for the specified call chain.
Session analysis	Homepage Session count, total token consumption, model request count, average session tokens Session analysis Display First Input, Last Output, Session ID, Total Tokens, total duration (ms), number of call chains, duration (s), model, start time, end time, and more for the specified session.
Token analysis	Homepage Total Tokens, Input Tokens, Output Tokens, Cache Read Tokens, Reasoning Tokens, Input Tokens (excluding Cache), Top 10 session token consumption, Top 10 model token consumption, token category proportion
Topic list	Display the log types, projects, log topics, description information, and more generated by the current TLS application.

Manage my applications

After the log application is successfully connected, you can manage connected log applications as follows:

AgentLoop

Application marketplace

Log in to TLS console.

Select the region where log service is located from the top navigation bar.

In the left navigation bar, select AgentLoop > AI Monitoring.

On the AI Observation page, in the Actions column for the corresponding log application, the following management actions are supported:

Edit application information: Click the edit icon to modify the application name and description.

Obtain usage guide: Click the access configuration icon to open the access configuration popup, where you can obtain example information for the usage guide.

Delete my application: Click the delete icon to remove the log application.

application_name_trace (log topic) log fields

Field	Type	Description
Attributes	json	Custom key-value pairs mounted on the Span, recording business-related information.
Duration	long	Duration of the Span (End - Start), measuring the time consumed by the operation.
End	long	Span end time.
Events	text	Events occurring during the Span's lifecycle.
Host	text	The host running the service.
Kind	text	Type of the Span.
Links	text	References to other Spans (used for cross-Trace associations, such as asynchronous task scenarios).
Name	text	Name of the Span, usually a description of the operation.
OTLPName	text	Original name reported via the OTLP protocol (OTLP = OpenTelemetry Line Protocol).
OTLPVersion	text	Version number of the OTLP protocol used.
ParentSpanID	text	ID of the parent Span, indicating which operation triggered the current operation. This field is empty for the root node.
Resource	json	A set of attributes describing the service itself (more comprehensive than ServiceName), such as service name, version, runtime environment, and so on.
ServiceName	text	Name of the service that generated this Span.
SpanID	text	Unique ID of the current Span.
Start	long	Span start time.
StatusCode	text	Execution result of the Span.
StatusDescription	text	Supplementary description of the Span's execution result.
TracelD	text	TraceID for the entire call chain. All Spans for the same request share the same TraceID.
TraceState	text	Custom trace context metadata.

Attributes

Field	Type	Description
error.type	text	Error category for this call.
gen_ai.input.messages	text	Complete list of input messages sent to the model.
gen_ai.output.messages	text	Content of the model's output message, that is, the full reply text returned by the LLM.
gen_ai.provider.name	text	Name of the model provider.
gen_ai.request.model	text	Name of the model specified in the request.
gen_ai.response.finish_reasons	text	Reason for generation completion, indicating why the model stopped outputting.
gen_ai.response.first_output_time_unix_nano	long	Time when the first token was output, Unix nanosecond timestamp.
gen_ai.response.model	text	Name of the model actually used for the response.
gen_ai.tool.name	text	Name of the tool involved in this invocation.
gen_ai.tool.type	text	Type of tool when invoking an external tool.
gen_ai.usage.cache_read.input_tokens	long	Number of input tokens that hit the KV Cache.
gen_ai.usage.input_tokens	long	Number of input tokens consumed by this request.
gen_ai.usage.output_tokens	long	Number of output tokens consumed by this request.
gen_ai.usage.reasoning.output_tokens	long	Number of output tokens consumed by Reasoning.
gen_ai.usage.total_tokens	long	Total number of tokens.
otel.status_code	text	OpenTelemetry status code, used to quickly determine whether this Span completed normally.
session.id	text	Session ID of the current session.
tls.openai.request.model_parameters	text	Model parameters passed in the request, recording the parameter configuration carried when invoking the LLM.
tls.span.type	text	Custom Span type defined by TLS.
user.id	text	User ID.

Torch Log Service