Description #

Seed-X multilingual text translation model – cross-language text translation key features

Key features #

• Intelligent multilingual translation
• Supports text conversion between multiple languages. The source_language and target_language parameters allow customization of the source and target languages.
• Based on the Seed-X-Instruct-7B/Seed-X-PPO-7B models, provides high-quality translation results.
• For supported languages, see: https://huggingface.co/ByteDance-Seed/Seed-X-Instruct-7B
• For general scenarios, it is recommended to use the Seed-X-PPO-7B model, which delivers better translation quality.
• Flexible configuration and optimization
• Supports multiple computation precision options (such as bfloat16), suitable for different performance requirements.
• Integrates tensor parallelism and prefix caching technologies to significantly improve inference efficiency.
• Supports automatic or manual device allocation, perfectly compatible with single-GPU and multi-GPU environments.
• Resource usage
• It is recommended to use a GPU with more than 24 GB of video memory.

Scenario optimization #

• Widely applicable to cross-language content conversion, multilingual document processing, internationalized application development, and many other scenarios.
• Built-in batch processing mechanism efficiently supports large-scale parallel translation of text data.
• Supports precise control of the maximum number of generated tokens to meet diverse business requirements.

Operator parameters #

Input #

Output #

The processed array, with each element being the translation result of each text. For texts that fail to process, an empty string is returned.

Parameters #

If a parameter does not have a default value, it is required.

Examples #

The following code demonstrates how to use daft to run the operator and translate multilingual text based on the Doubao model.

from __future__ import annotations

import logging
import os

import ray

import daft
from daft import col
from daft.las.functions.text.multilingual_text_translate import MultilingualTextTranslate
from daft.las.functions.udf import las_udf

if __name__ == "__main__":
    os.environ["DAFT_RUNNER"] = "ray"

    def configure_logging():
        logging.basicConfig(
            level=logging.INFO,
            format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
            datefmt="%Y-%m-%d %H:%M:%S.%s".format(),
        )
        logging.getLogger("tracing.span").setLevel(logging.WARNING)
        logging.getLogger("daft_io.stats").setLevel(logging.WARNING)
        logging.getLogger("DaftStatisticsManager").setLevel(logging.WARNING)
        logging.getLogger("DaftFlotillaScheduler").setLevel(logging.WARNING)
        logging.getLogger("DaftFlotillaDispatcher").setLevel(logging.WARNING)

    ray.init(dashboard_host="0.0.0.0", runtime_env={"worker_process_setup_hook": configure_logging})
    daft.set_runner_ray()
    daft.set_execution_config(actor_udf_ready_timeout=600)
    daft.set_execution_config(min_cpu_per_task=0)

    samples = {
        "text": [
            "",
        ]
    }

    model_path = os.getenv("MODEL_PATH", "/opt/las/models")
    model_name = "Seed-X-PPO-7B"
    max_model_len = 32768
    max_num_seqs = 128
    tensor_parallel_size = 1
    dtype = "bfloat16"
    enable_prefix_caching = True
    gpu_memory_utilization = 0.9
    use_cot = False
    source_language = "Chinese"
    target_language = "English"
    max_tokens = 1024
    batch_size = 4
    seed = 42

    ds = daft.from_pydict(samples)
    ds = ds.with_column(
        "translate_text",
        las_udf(
            MultilingualTextTranslate,
            construct_args={
                "model_path": model_path,
                "model_name": model_name,
                "dtype": dtype,
                "max_model_len": max_model_len,
                "max_num_seqs": max_num_seqs,
                "tensor_parallel_size": tensor_parallel_size,
                "enable_prefix_caching": enable_prefix_caching,
                "gpu_memory_utilization": gpu_memory_utilization,
                "use_cot": use_cot,
                "source_language": source_language,
                "target_language": target_language,
                "max_tokens": max_tokens,
                "batch_size": batch_size,
                "seed": seed,
            },
            num_gpus=1,
            batch_size=1,
            concurrency=1,
        )(col("text")),
    )

    ds.show()

    # ╭─────────────────────────────────────────────────────────────┬────────────────────────────────╮
    # │ text                                                        ┆ translate_text                 │
    # │ ---                                                         ┆ ---                            │
    # │ Utf8                                                        ┆ Utf8                           │
    # ╞═════════════════════════════════════════════════════════════╪════════════════════════════════╡
    # │ This is a high-quality academic paper on the development of artificial intelligence technology, with detailed and informative content…           ┆ This is a high-quality academ… │
    # ╰─────────────────────────────────────────────────────────────┴────────────────────────────────╯