Volcengine Kubernetes Engine (VKE) has extended the capabilities of the native kube-scheduler and now supports Gang scheduling. This article describes how to trigger and use Gang scheduling.

Feature overview

Gang scheduling is a scheduling policy in concurrent systems that schedules multiple related processes to run simultaneously on different processors. This prevents the blockage of all related processes due to anomalies in some processes.

The primary principle of Gang scheduling is to ensure that all related processes can start simultaneously and that no abnormal processes are allowed, that is, All-or-Nothing. All-or-Nothing means that either all processes are started and the batch task runs successfully or if there are any abnormal processes, none of the processes can be started and the batch task fails.

VKE introduces the PodGroup CustomResourceDefinition (CRD) resource to implement Gang scheduling. A PodGroup in VKE consists of a set of related pods that need to be started for batch tasks to match the All-or-Nothing principle. To use Gang scheduling in VKE, create a PodGroup with a custom name, specify minimum replicas required for batch tasks to run as expected, and specify a timeout. Then, associate it with your workload by using instance labels.

Prerequisites

The scheduler-plugin add-on has been installed, and the Gang scheduling capability has been enabled. For specific operations, see scheduler-plugin.

Procedure

Step 1: Create a PodGroup CRD

Log in to the VKE console.

In the left navigation pane of the VKE console, click Cluster.

In the cluster list, click the name of the target cluster.

In the left navigation bar of the cluster management page, click Workload > Resource Explorer.

On the Resource Explorer page, click Create with Yaml to create a PodGroup CRD resource.

Kubernetes v1.26 and earlier
Kubernetes v1.28 and later

apiVersion: scheduling.sigs.k8s.io/v1alpha1
kind: PodGroup
metadata:
name: demo-podgroup # The name of the PodGroup.
namespace: default # The namespace of the PodGroup.
spec:
scheduleTimeoutSeconds: 10 # Timeout for waiting for the PodGroup to reach the minimum number of replicas.
minMember: 4 # The minimum number of replicas required for this PodGroup.

Click Confirm.

In the CRD category of the Resource Explorer, view the created PodGroup CRD resource. Click podgroup to enter the details page, where you can view the name of the PodGroup.

Step 2: Use Gang scheduling on workloads

The following content uses a Deployment as an example.

Use the console

In the left navigation bar of the cluster management page, click Workload > Deployment.

On the Deployment page, create a Deployment or update an existing Deployment. For details, see Creating Deployments and Managing workloads.

In the Advanced settings step, go to the Instance configuration section to add an instance label.

For clusters running Kubernetes v1.26 or earlier, set the label key to pod-group.scheduling.sigs.k8s.io and the value to <PodGroup name>.

For clusters running Kubernetes v1.28 or later, set the label key to scheduling.x-k8s.io/pod-group and the value to <PodGroup name>.

Use YAML

The following sample code shows how to configure Gang scheduling by using YAML:

Kubernetes v1.26 and earlier

Kubernetes v1.28 and later

Validating results

If the node resources in the cluster satisfy the resource requests of all Pods in the Deployment, all Pods in the workload will start as expected and enter the Running state. You can view the startup status of the Pod on the List of instances tab of the workload details page.

If the node resources in the cluster can only satisfy the resource requests of some Pods in the Deployment (for example, only requests of 3 out of 4 Pods can be satisfied), all Pods in the workload will remain in the Pending state.