Description

Dedicated processor for large model text generation (Doubao/DeepSeek)

Key features

Plain text scenario optimization: Automatically constructs a message structure compliant with model specifications based on user input text data
Input simplification mechanism: Natively supports str type input and automatically wraps it in the format {role: user, content: text}
Multi-task support: Ready to use for NLP scenarios such as translation, summarization, and question answering
Dual prompt system:
- system_content: System-level behavior guidance (such as translation style control)
- prompt: User-level instruction template (supports {query} placeholder replacement)

Input and output specifications

Input format: Plain text data
Output format:
- Default mode: str type generation result
- Diagnostic mode: Set the environment variable LAS_LLM_FINISH_REASON_CHECK=true to return the complete generation result and the model's finish reason:
  - llm_result: str type, generation result
  - finish_reason: str type, model finish reason, possible values: stop, length, content_filter

Daft invocation

Operator parameters

Input

Input column name	Description
raw_text	Contains text data to be processed. Type is str

Output

(By default) When the environment variable LAS_LLM_FINISH_REASON_CHECK=false, the returned field type is str.
When the environment variable LAS_LLM_FINISH_REASON_CHECK=true, the returned field type is struct and contains the following fields:

llm_result: Model output result
finish_reason: Model output finish reason

Parameters

If a parameter does not have a default value, it is required

Parameter name	Type	Default value	Description
model	str		Model name. Supported models: Doubao model and DeepSeek model. Example doubao-1.5-lite-32k
version	str or None		Model version. Enter the version information corresponding to the model. Example 250115
inference_type	str	batch	Inference type. Supports online inference and batch inference. The default value is batch, which uses batch inference. - online: Uses the online inference module provided by the ModelArk platform for inference - batch: Uses the batch inference module provided by the ModelArk platform for inference
max_tokens	int or None		Maximum length of model response (in tokens). The total length of input and output is limited by the model context.
max_completion_tokens	int or None		The maximum number of tokens generated by the model, including reasoning_content and content, but excluding the input messages. When this limit is exceeded, the model stops outputting reasoning_content and answers, and returns the finish_reason field as length.
stop	list or None		The model stops generating when it encounters a string specified in the stop field; this string itself will not be output. Supports up to 4 strings. For example, ["你好", "天气"]
frequency_penalty	float	0	Frequency penalty coefficient. If the value is positive, the model penalizes new tokens based on their frequency in the text, reducing the likelihood of repetitive output. Value range is [-2.0, 2.0], default is 0.
presence_penalty	float	0	Presence penalty coefficient. If the value is positive, the model penalizes new tokens based on whether they have appeared in the text so far, increasing the likelihood that the model will discuss new topics. Value range is [-2.0, 2.0]. Default value is 0.
temperature	float	1	Sampling temperature. Controls the degree to which the probability distribution of candidate words is smoothed when generating text. - When set to 0, the model considers only the token with the highest log probability. - Higher values (such as 0.8) make the output more random, while lower values (such as 0.2) make the output more focused and deterministic. It is generally recommended to adjust only temperature or top_p, not both. Value range is [0, 2]. Default value is 1.
top_p	float	0.7	Nucleus sampling probability threshold. The model considers token results within the top_p probability mass. When set to 0, the model considers only the token with the highest log probability. 0.1 means only the top 10% of tokens by probability mass are considered; the higher the value, the greater the randomness, and the lower the value, the more deterministic the output. It is generally recommended to adjust only temperature or top_p, not both. Default value is 0.7.
logit_bias	dict or None		Adjusts the probability of specified tokens appearing in the model's output, making the generated content more closely align with specific preferences. The logit_bias field accepts a map value, where each key is a token ID from the vocabulary (obtained using the tokenization interface), and each value is the bias value for that token, with a range of [-100, 100]. -1 decreases the likelihood of selection, 1 increases the likelihood of selection; -100 completely prohibits selection of the token, and 100 results in only that token being selectable. The actual effect of this parameter may vary depending on the model.
tools	list or None		List of tools to be called, which can be included in the model's returned information. To have the model return the tools to be called, configure this field.
llm_config	dict or None		Custom LLM configuration. Except for the parameters above, other parameters will be passed through to the model. The parameters above will override the values in llm_config.
request_timeout	int	1200	Timeout for a single request (in seconds).
max_concurrency	int	100	Maximum number of concurrent requests per process.
system_content	str or None		System prompt content. System prompt content, provided to the model as input with the system role.
prompt	str or None		User prompt. User prompt used to guide the model's behavior. When this field is configured, it will be concatenated with the input text and provided to the model as input with the user role. At the same time, this field can also be set to {query}, in which case the input text will replace this field.

Examples

The following code demonstrates how to use daft to access the ModelArk text generation model for batch inference. Note that each large model inference result may be different.

from __future__ import annotations

import os

import daft
from daft import col
from daft.las.functions.ark_llm.ark_llm_text_generate import ArkLLMTextGenerate
from daft.las.functions.udf import las_udf

if os.getenv("DAFT_RUNNER", "native") == "ray":
    import logging

    import ray

    def configure_logging():
        logging.basicConfig(
            level=logging.INFO,
            format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
            datefmt="%Y-%m-%d %H:%M:%S.%s".format(),
        )
        logging.getLogger("tracing.span").setLevel(logging.WARNING)
        logging.getLogger("daft_io.stats").setLevel(logging.WARNING)
        logging.getLogger("DaftStatisticsManager").setLevel(logging.WARNING)
        logging.getLogger("DaftFlotillaScheduler").setLevel(logging.WARNING)
        logging.getLogger("DaftFlotillaDispatcher").setLevel(logging.WARNING)

    ray.init(dashboard_host="0.0.0.0", runtime_env={"worker_process_setup_hook": configure_logging})
    daft.set_runner_ray()

daft.set_execution_config(min_cpu_per_task=0)

if __name__ == "__main__":
    # LAS_API_KEY must be configured as an environment variable. You can obtain LAS_API_KEY by creating it on the LAS service page.
    queries = {
        "query": [
            "",
            "",
        ]
    }

    ds = daft.from_pydict(queries)
    ds = ds.with_column(
        "llm_result",
        las_udf(
            ArkLLMTextGenerate,
            construct_args={
                "model": "doubao-1.5-pro-32k",
                "inference_type": "online",
            },
        )(col("query")),
    )
    ds.show()

    #  Output (the result of each large model inference may vary)
    # ╭────────────────────┬───────────────────────────────────────╮
    # │ query              ┆ llm_result                            │
    # │ ---                ┆ ---                                   │
    # │ Utf8               ┆ Utf8                                  │
    # ╞════════════════════╪═══════════════════════════════════════╡
    # │ 中国的首都在哪里      ┆ 中国的首都是北京。                        │
    # │                    ┆                                       │
    # │                    ┆ 北京是中国的政治中心、文化中心、国际…       │
    # ├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
    # │ 十字花科植物有哪些    ┆ 十字花科植物种类繁多，常见的有：           │
    # │                    ┆ 1. **蔬菜类**                          │
    # │                    ┆  …                                    │
    # ╰────────────────────┴───────────────────────────────────────╯

Last updated: 2026.05.12 19:06:29

Lake AI Service

Description #

Key features #

Input and output specifications #

Operator parameters #

Input #

Output #

Parameters #

Examples #