Docs

Torch Log Service

Framework log app

LLM application observability (LangChain/Eino)

This section introduces the LLM application observability log application provided in the TLS application marketplace.

Notice

It is recommended to use the latest version of the log application. For details, refer to Eino observability, LangGraph observability.

Overview

By integrating LLM application observability data, services such as LLM application cost and performance analysis, call chain analysis, and session analysis are provided.

Monitoring chart details

The monitoring chart details for TLS LLM application observability are as follows:

Cost and performance analysis

Type	Chart
Overview	Model call count Token usage Call chain count Avg LLM call per request Avg LLM Token per mins Model call errors/1m Request count trend TTFT - model first token latency - P99 percentile/1m Operation type distribution
LLM	Model call count, average model call latency, model call error count Average model call latency/1m, model latency percentile (P99)/1m, model call errors/1m TTFT - model first token latency - P99 percentile/1m, TTFT - model first token latency - average/1m Top 5 average model call latency, Top 5 model call count, Top 5 model call errors
Embedding	Embedding count, average embedding latency, embedding error count Embedding count/1m, embedding latency/1m, embedding errors/1m Top 5 embedding latency, Top 5 embedding count, Top 5 embedding errors
Retrieval&Rerank	Retrieval - call count, retrieval - average call latency, retrieval - error count Retrieval - call count/1m, retrieval - call latency/1m, retrieval - error count/1m Rerank - call count, rerank - average call latency, rerank - error count Rerank - call count/1m, rerank - call latency/1m, rerank - error count/1m
Tool	Call count, average call latency, error count Call count/1m, call latency/1m, error count/1m Top 5 call count, Top 5 call latency, Top 5 errors
Task	Call count, average call latency, error count Call count/1m, call latency/1m, error count/1m Top 5 call count, Top 5 call latency, Top 5 errors
Token	Token usage, Avg tokens per LLM call Tokens consumed/1m, Avg tokens per LLM call/1m Top 5 models by token usage, Top 5 sessions by token usage

Call chain analysis

Displays the Trace ID, status, total duration (ms), start time, and end time for the specified call chain.

Session analysis

Displays the Input / Session ID, tokens, total duration (ms), call chain, user ID, and start time for the specified session.

Integrate log applications

Select the region where log service is located from the top navigation bar.

In the left navigation bar, click Common functions > Log application.

On the App store tab, select the log application card to be integrated. For example, on the LLM application observability card, click Integrate application.

Configure the integration application parameters, and then click Create LLM observability.

Parameters	Description
LLM application name	Name of the integrated LLM log application.
Description	Description information for the log application.
Region	The region of the log application to be integrated.
Log project	Select the project where data will be stored.

After creating the application, proceed to the next step to integrate LLM application data.

Integrate LLM application data

After completing the creation of LLM application observability, LLM application data must be integrated to use the performance and cost analysis dashboards, call chain analysis, session analysis, and other capabilities.

Integrate using Python applications

Download and install the Python agent.

Download the probe installer from the PyPI repository.

pip3 install volcengine-tls-bootstrap

Configure environment variables.

Manually add the following environment variables to the Python application:

export TLS_APP_NAME="LLM app name" # Application name
export TLS_EXPORTER_OTLP_ENDPOINT="tls-cn-shanghai.volces.com:4317" #Application access point. You need to specify the access methods for both public and private network environments.
export TLS_EXPORTER_OTLP_HEADERS="x-tls-appid=586411c2-442e-4753-8e37-xxx,x-tls-otel-ak={your-ak},x-tls-otel-sk={your-sk},x-tls-otel-region=cn-shanghai"

Notice

x-tls-appid: Automatically populated by the frontend based on the name created by the user.

x-tls-otel-region: Automatically populated by the frontend based on the current Region.

(Optional) For Docker environments, refer to the following Dockerfile example to modify the Dockerfile:

ENV TLS_APP_NAME="LLM app name" # Application name
ENV TLS_EXPORTER_OTLP_HEADERS="tls-cn-shanghai.volces.com:4317" #Application access point. You need to specify the access methods for both public and private network environments.
ENV TLS_EXPORTER_OTLP_HEADERS="x-tls-appid=586411c2-442e-4753-8e37-xxx,x-tls-otel-ak={my-ak},x-tls-otel-sk={my-sk},x-tls-otel-region=cn-shanghai"

Install the Python probe using volc-tls-bootstrap.

To speed up installation, it is recommended to configure the image repository first using the following command:

pip config set global.index-url https://mirrors.ivolces.com/pypi/simple/ && pip config set install.trusted-host mirrors.ivolces.com

Install the probe:

volcengine-tls-bootstrap -a install

Start the application.

Start the application using the TLS Python probe

volcengine-tls-instrument python app.py

Start the application without using the TLS Python probe (optional)

To start the application without using the Python probe, import the Python probe in the main entry file, and then start the application. For example, import the Python probe in the main.py/app.py file.

from volcengine.opentelemetry.instrumentation.auto_instrumentation import sitecustomize

Notice

If the application is started using Unicorn, you need to replace it with Gunicorn.

If gevent coroutines are used, you need to set the environment variable GEVENT_ENABLE=true.

Integrate using Eino applications

Configure environment variables.

Manually add the following environment variables to the Eino application:

export TLS_APP_NAME="LLM app name" # application name
export TLS_ENDPOINT="tls-cn-shanghai.volces.com:4317" #The endpoint of integrated app. Requires description of integration methods of public network and private network
export TLS_EXPORTER_OTLP_HEADERS="x-tls-appid=586411c2-442e-4753-8e37-xxx,x-tls-otel-ak={your-ak},x-tls-otel-sk={your-sk},x-tls-otel-region=cn-shanghai"

Notice

x-tls-appid: Automatically populated by the frontend based on the name created by the user.

x-tls-otel-region: Automatically populated by the frontend based on the current Region.

(Optional) For Docker environments, refer to the following Dockerfile example to modify the Dockerfile:

ENV TLS_APP_NAME="LLM app name" # application name
ENV TLS_ENDPOINT="tls-cn-shanghai.volces.com:4317" #The endpoint of integrated app. Requires description of integration methods of public network and private network
ENV TLS_EXPORTER_OTLP_HEADERS="x-tls-appid=586411c2-442e-4753-8e37-xxx,x-tls-otel-ak={my-ak},x-tls-otel-sk={my-sk},x-tls-otel-region=cn-shanghai"

Configure TLS Callback.

package main

import (
"context"
"log"

"github.com/cloudwego/eino-ext/callbacks/tls"
"github.com/cloudwego/eino/callbacks"
)

func main() {
ctx := context.Background()

cbh, shutdown, err := tls.NewTLSHandler(ctx,&tls.TLSConfig{
TLSEndpoint: os.GetEnv["TLS_ENDPOINT"], //Or directly use the value of the environment variable TLS_ENDPOINT
AppName: os.GetEnv["TLS_APP_NAME"], //Or directly use the value of the environment variable TLS_APP_NAME
TLSOTLPHeadersStr: os.GetEnv["TLS_EXPORTER_OTLP_HEADERS"], //Or directly use the value of the environment variable TLS_EXPORTER_OTLP_HEADERS
})
if err != nil {
log.Fatal(err)
return
}
if shutdown != nil {
defer shutdown(ctx)
}

// Set tls as a global callback
callbacks.AppendGlobalHandlers(cbh)

// your code here

return
}

Manage my applications

After the log application is successfully connected, you can manage connected log applications as follows:

Select the region where log service is located from the top navigation bar.

In the left navigation bar, click Common functions > Log application.

Go to the My applications tab. In the Actions column for the corresponding log application, the following management operations are supported:

Actions	Description
Access management	Click Access management to enter the Access management tab, where you can edit the description information for the log application access.
Cancel access	Click Cancel access, confirm the information in the cancel access dialog, and then click Cancel access again.
View access configuration	Click View access configuration to enter the access configuration page and view the relevant configuration for accessing LLM application data.

Last updated: 2026.09.15 18:26:13