Description

Dedicated processor for large model text generation (Doubao/DeepSeek)

Key features

Plain text scenario optimization: Automatically constructs a message structure that complies with model specifications based on user input text data
Input simplification mechanism: Natively supports str type input and automatically encapsulates it as {role: user, content: text} format
Out-of-the-box support for NLP scenarios such as translation, summarization, and question answering
Dual prompt system:
- system_content: System-level behavior guidance (such as translation style control)
- prompt: User-level instruction template (supports {query} placeholder replacement)

Input and output specifications

Input format: Plain text data
Output format:
- Default mode: str type generation result
- Diagnostic mode: Set the environment variable LAS_LLM_FINISH_REASON_CHECK=true to return the complete generation result and the model's finish reason:
  - llm_result: str type, generation result
  - finish_reason: str type, the reason for model result termination, possible values: stop, length, content_filter

Daft invocation

Operator parameters

Input

Input column name	Description
raw_text	Contains the text data to be processed. Type is str

Output

(By default) When the environment variable LAS_LLM_FINISH_REASON_CHECK=false, the return field type is str.
When the environment variable LAS_LLM_FINISH_REASON_CHECK=true, the return field type is struct and contains the following fields:

llm_result: Model output result
finish_reason: Model output finish reason

Parameters

If a parameter does not have a default value, it is required

Parameter name	Type	Default value	Description
model	str		Model name. Supported models: Doubao model and DeepSeek model. Example doubao-1.5-lite-32k
version	str or None		Model version. Enter the version information corresponding to the model. Example 250115
inference_type	str	batch	Inference type. Supports online inference and batch inference. The default value is batch, which uses batch inference. - online: Uses the online inference module provided by ModelArk platform for inference - batch: Uses the batch inference module provided by ModelArk platform for inference
max_tokens	int or None		Maximum length of model response (in tokens). The total input and output length is limited by the model context.
max_completion_tokens	int or None		The maximum number of tokens generated by the model, including reasoning_content and content, but excluding the input messages. When this limit is exceeded, the model stops outputting reasoning_content and answers, and returns the finish_reason field with the value "length".
stop	list or None		The model stops generating when it encounters a string specified in the stop field; this string itself will not be output. Supports up to 4 strings. For example, ["你好", "天气"]
frequency_penalty	float	0	Frequency penalty coefficient. If the value is positive, new tokens are penalized based on their frequency in the text, reducing the likelihood of the model repeating tokens verbatim. Value range: [-2.0, 2.0], default is 0.
presence_penalty	float	0	Presence penalty coefficient. If the value is positive, new tokens are penalized based on whether they have appeared in the text so far, increasing the likelihood that the model discusses new topics. Value range: [-2.0, 2.0]. Default value: 0
temperature	float	1	Sampling temperature. Controls the degree to which the probability distribution of candidate words is smoothed when generating text. - When set to 0, the model considers only the token with the highest log probability. - Higher values (such as 0.8) make the output more random, while lower values (such as 0.2) make the output more focused and deterministic. It is generally recommended to adjust only temperature or top_p, not both. Value range: [0, 2]. Default value: 1
top_p	float	0.7	Nucleus sampling probability threshold. The model considers token results within the top_p probability mass. When set to 0, the model considers only the token with the highest log probability. 0.1 means only the top 10% of tokens by probability mass are considered; the higher the value, the greater the randomness of the output, and the lower the value, the more deterministic the output. It is generally recommended to adjust only temperature or top_p, not both. Default value: 0.7
logit_bias	dict or None		Adjusts the probability of specified tokens appearing in the model's output, making the generated content better match specific preferences. The logit_bias field accepts a map value, where each key is a token ID from the vocabulary (obtained using the tokenization interface), and each value is the bias value for that token, with a range of [-100, 100]. -1 decreases the likelihood of selection, 1 increases the likelihood of selection; -100 completely prohibits the selection of that token, and 100 results in only that token being selectable. The actual effect of this parameter may vary depending on the model.
tools	list or None		List of tools to be called, which can be included in the model's returned information. You must configure this structure.
llm_config	dict or None		Custom LLM configuration. In addition to the parameters above, other parameters will be forwarded directly to the model. The parameters above will override the values in llm_config.
request_timeout	int	1200	Timeout duration. The timeout duration for a single request (in seconds).
max_concurrency	int	100	Maximum concurrency. The maximum number of concurrent requests per process.
system_content	str or None		System prompt content. System prompt content, provided to the model as input with the system role.
prompt	str or None		User prompt. User prompt used to guide the model's behavior. When this field is configured, it will be combined with the input text and provided to the model as input with the user role. Additionally, this field can be set to {query}, in which case the input text will replace this field.

Examples

The following code demonstrates how to use daft to access the ModelArk text generation model for batch inference. Note that the result of each large model inference may vary.

from __future__ import annotations

import os

import daft
from daft import col
from daft.las.functions.ark_llm.ark_llm_text_generate import ArkLLMTextGenerate
from daft.las.functions.udf import las_udf

if os.getenv("DAFT_RUNNER", "native") == "ray":
    import logging

    import ray

    def configure_logging():
        logging.basicConfig(
            level=logging.INFO,
            format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
            datefmt="%Y-%m-%d %H:%M:%S.%s".format(),
        )
        logging.getLogger("tracing.span").setLevel(logging.WARNING)
        logging.getLogger("daft_io.stats").setLevel(logging.WARNING)
        logging.getLogger("DaftStatisticsManager").setLevel(logging.WARNING)
        logging.getLogger("DaftFlotillaScheduler").setLevel(logging.WARNING)
        logging.getLogger("DaftFlotillaDispatcher").setLevel(logging.WARNING)

    ray.init(dashboard_host="0.0.0.0", runtime_env={"worker_process_setup_hook": configure_logging})
    daft.set_runner_ray()

daft.set_execution_config(min_cpu_per_task=0)

if __name__ == "__main__":
    # Environment variable LAS_API_KEY must be configured: LAS_API_KEY is obtained by creating it on the LAS service page
    queries = {
        "query": [
            "",
            "",
        ]
    }

    ds = daft.from_pydict(queries)
    ds = ds.with_column(
        "llm_result",
        las_udf(
            ArkLLMTextGenerate,
            construct_args={
                "model": "deepseek-v3",
                "inference_type": "online",
            },
        )(col("query")),
    )
    ds.show()

    # Output (the result of each large model inference may vary)
    # ╭────────────────────┬─────────────────────────────────────────────────────────╮
    # │ query              ┆ llm_result                                              │
    # │ ---                ┆ ---                                                     │
    # │ Utf8               ┆ Utf8                                                    │
    # ╞════════════════════╪═════════════════════════════════════════════════════════╡
    # │ 中国的首都在哪里   ┆ 中国的首都是**北京**。 Beijing is the political, cultural, and international exchange center of China. │
    # ├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
    # │ 十字花科植物有哪些 ┆ 十字花科（Brassicaceae或Cruciferae）…                   │
    # ╰────────────────────┴─────────────────────────────────────────────────────────╯

Last updated: 2026.05.12 19:06:30

Lake AI Service

Description #

Key features #

Input and output specifications #

Operator parameters #

Input #

Output #

Parameters #

Examples #