Operator ID: daft.las.functions.video.video_blackborder_crop.VideoBlackBorderCrop

Description #

Video black border detection and cropping

Key features #

• Multi-algorithm support: Supports three black border detection algorithms, suitable for different scenarios:
• Row and column ratio method (threshold_ratio): Suitable for scenarios where there are watermarks or scattered noise within the black border.
• Edge detection method (edge_detection): Suitable for scenarios with gradient black borders or slight noise.
• Histogram analysis method (histogram): Suitable for scenarios with uneven pixel distribution in black borders.
• Automatic boundary calculation: Automatically calculates the boundaries of the valid frame area.
• Lossless cropping: Video cropping is implemented based on FFmpeg, preserving the original encoding characteristics.
• Audio-video synchronization: Retains the original audio and ensures precise alignment between audio and video.

Applicable scenarios #

• Video preprocessing
• Remove video black borders
• Video standardization

Format support #

• MP4 (.mp4)
• AVI (.avi)
• MOV (.mov)
• MKV (.mkv)
• Other common video formats

Operator parameters #

Input #

Output #

An array containing the paths of the cropped results (string type). Returns the output path on success; returns None on failure.

Parameters #

If a parameter does not have a default value, it is required.

Examples #

The following code demonstrates how to use Daft (for distributed environments) to run the operator for video black border detection and cropping.

from __future__ import annotations

import os

import daft
from daft import col
from daft.las.functions.udf import las_udf
from daft.las.functions.video import VideoBlackBorderCrop

if __name__ == "__main__":
    # The content will be saved to the specified TOS path. Therefore, you need to set environment variables to ensure you have permission to write to TOS, including: ACCESS_KEY, SECRET_KEY, TOS_ENDPOINT, TOS_REGION, TOS_TEST_DIR
    TOS_DIR = os.getenv("TOS_TEST_DIR", "tos_bucket")
    output_tos_dir = f"tos://{TOS_DIR}/video/video_blackborder_crop"
    if os.getenv("DAFT_RUNNER", "native") == "ray":
        import logging

        import ray

        def configure_logging():
            logging.basicConfig(
                level=logging.INFO,
                format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
                datefmt="%Y-%m-%d %H:%M:%S.%s".format(),
            )
            logging.getLogger("tracing.span").setLevel(logging.WARNING)
            logging.getLogger("daft_io.stats").setLevel(logging.WARNING)
            logging.getLogger("DaftStatisticsManager").setLevel(logging.WARNING)
            logging.getLogger("DaftFlotillaScheduler").setLevel(logging.WARNING)
            logging.getLogger("DaftFlotillaDispatcher").setLevel(logging.WARNING)

        ray.init(dashboard_host="0.0.0.0", runtime_env={"worker_process_setup_hook": configure_logging})
        daft.set_runner_ray()

    daft.set_execution_config(actor_udf_ready_timeout=600)
    daft.set_execution_config(min_cpu_per_task=0)

    tos_dir_url = os.getenv("TOS_DIR_URL", "las-cn-beijing-public-online.tos-cn-beijing.volces.com")
    samples = {
        "input_path": [f"https://{tos_dir_url}/public/shared_video_dataset/sample.mp4"],
        "output_path": [f"{output_tos_dir}/test_blackborder_video_cropped_histogram.mp4"],
    }
    ds = daft.from_pydict(samples)

    constructor_kwargs = {
        "detection_method": "histogram",
        "black_threshold": 50,
        "sample_frames": 10,
        "is_keep_audio": True,
        "core_region_ratio": 0.5,
        "continuous_black_rows": 3,
        "continuous_black_cols": 3,
        "dark_region_brightness": 50,
    }

    ds = ds.with_column(
        "crop_result",
        las_udf(VideoBlackBorderCrop, construct_args=constructor_kwargs)(col("input_path"), col("output_path")),
    )

    ds.show()

# ╭────────────────────────────────┬────────────────────────────────┬────────────────────────────────╮
# │ input_path                     ┆ output_path                    ┆ crop_result                    │
# │ ---                            ┆ ---                            ┆ ---                            │
# │ String                         ┆ String                         ┆ String                         │
# ╞════════════════════════════════╪════════════════════════════════╪════════════════════════════════╡
# │ https://las-ai-qa-online.tos-… ┆ https://las-ai-qa-online.tos-… ┆ https://las-ai-qa-online.tos-… │
# ╰────────────────────────────────┴────────────────────────────────┴────────────────────────────────╯