Description #

Audio speech quality evaluation operator – calculates multiple audio quality metrics based on ClearerVoice-Studio/speechscore

Key features #

• Unified encapsulation of SpeechScore, supports both reference-free and reference-based metrics
• The set of metrics to be calculated can be configured via the metrics parameter; by default, all supported metrics are calculated
• Supports input paths for local, TOS, HTTP, and S3; the reference audio column reference_audio_paths is optional
• Output is a structured result containing common metrics such as BSSEval, DNSMOS, NISQA, PESQ, STOI, SNR, SRMR, and more

Metric examples #

• Reference-free: DNSMOS (OVRL/SIG/BAK/P808_MOS), NISQA (mos_pred and others)
• Reference-based: BSSEval (ISR/SAR/SDR), PESQ, STOI, SISDR, and more

Operator parameters #

Input #

Output #

Structured result array, where each element is a struct containing multiple metrics with the following fields:

• BSSEval: struct
• ISR: float | null
• SAR: float | null
• SDR: float | null
• CBAK: float | null
• COVL: float | null
• CSIG: float | null
• DISTILL_MOS: float | null
• DNSMOS: struct
• BAK: float | null
• OVRL: float | null
• P808_MOS: float | null
• SIG: float | null
• FWSEGSNR: float | null
• LLR: float | null
• LSD: float | null
• MCD: float | null
• NB_PESQ: float | null
• NISQA: struct
• col_pred: float | null
• dis_pred: float | null
• loud_pred: float | null
• mos_pred: float | null
• noi_pred: float | null
• PESQ: float | null
• SISDR: float | null
• SNR: float | null
• SRMR: float | null
• SSNR: float | null
• STOI: float | null

Audio processing failures return a struct containing null values

Parameters #

If a parameter does not have a default value, it is required

Examples #

The following code demonstrates how to use daft to evaluate the speech quality of audio.

from __future__ import annotations

import logging
import os

import ray
import daft
from daft import col
from daft.las.functions.audio import AudioSpeechScore
from daft.las.functions.udf import las_udf

def configure_logging():
    logging.basicConfig(
        level=logging.INFO,
        format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
        datefmt="%Y-%m-%d %H:%M:%S.%s".format(),
    )
    logging.getLogger("tracing.span").setLevel(logging.WARNING)
    logging.getLogger("daft_io.stats").setLevel(logging.WARNING)
    logging.getLogger("DaftStatisticsManager").setLevel(logging.WARNING)
    logging.getLogger("DaftFlotillaScheduler").setLevel(logging.WARNING)
    logging.getLogger("DaftFlotillaDispatcher").setLevel(logging.WARNING)

configure_logging()

if __name__ == "__main__":
    TOS_INPUT_DIR_URL = os.getenv("TOS_INPUT_DIR_URL", "las-cn-beijing-public-online.tos-cn-beijing.volces.com")

    ray.init(dashboard_host="0.0.0.0", runtime_env={"worker_process_setup_hook": configure_logging})
    daft.set_runner_ray()

    samples = {
        "audio_paths": [os.path.join(f"https://{TOS_INPUT_DIR_URL}", "public/shared_audio_dataset/video_demo.mp3")],
        "reference_audio_paths": [os.path.join(f"https://{TOS_INPUT_DIR_URL}", "public/shared_audio_dataset/video_demo_denoised.mp3")],
    }
    df = daft.from_pydict(samples)
    df = df.with_column(
        "audio_speech_score",
        las_udf(
            AudioSpeechScore,
            construct_args={},
            num_gpus=1,
            batch_size=8,
            concurrency=1,
        )(col("audio_paths"), col("reference_audio_paths")),
    )

    df.show()
    #     ╭────────────────────────────────┬────────────────────────────────┬───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
    # │ audio_paths                    ┆ reference_audio_paths          ┆ audio_speech_score                                                                                                                                                                                                            │
    # │ ---                            ┆ ---                            ┆ ---                                                                                                                                                                                                                           │
    # │ String                         ┆ String                         ┆ Struct[BSSEval: Struct[ISR: Float64, SAR: Float64, SDR: Float64], CBAK: Float64, COVL: Float64, CSIG: Float64, DISTILL_MOS: Float64, DNSMOS: Struct[BAK: Float64, OVRL: Float64, P808_MOS: Float64, SIG: Float64], FWSEGSNR:  │
    # │                                ┆                                ┆ Float64, LLR: Float64, LSD: Float64, MCD: Float64, NB_PESQ: Float64, NISQA: Struct[col_pred: Float64, dis_pred: Float64, loud_pred: Float64, mos_pred: Float64, noi_pred: Float64], PESQ: Float64, SISDR: Float64, SNR:       │
    # │                                ┆                                ┆ Float64, SRMR: Float64, SSNR: Float64, STOI: Float64]                                                                                                                                                                         │
    # ╞════════════════════════════════╪════════════════════════════════╪═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡
    # │ https://las-public-data-qa.to… ┆ https://las-public-data-qa.to… ┆ {BSSEval: {ISR: 15.0130389824…                                                                                                                                                                                                │
    # ╰────────────────────────────────┴────────────────────────────────┴───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯