Pain point scenarios #

Linux /dev/shm is a memory-based temporary file system (tmpfs), commonly used for inter-process communication and high-throughput temporary data exchange. In data and AI scenarios, the following situations often require larger shared memory:

• Ray workloads: Ray's object store prioritizes the use of shared memory to reduce copying and disk writes. When shared memory is insufficient, object spillover may occur or there may be noticeable throughput and latency fluctuations.
• Multiprocess data loading and preprocessing: For example, Python multiprocessing, data preprocessing pipelines, shuffle, cache, and so on, often use shared memory as a high-performance buffer.
• Large object or large batch data exchange: When large objects or large batch data need to be frequently transferred within the same node, a small /dev/shm becomes a bottleneck.

Solution overview #

• EMR Serverless supports configuring shared memory in Ray tasks (RayJob, Ray Cluster/Ray Serve compute groups) and CustomJob. The platform mounts /dev/shm (tmpfs) inside the container, with a default size of 70% of the container memory, to meet the needs of scenarios requiring larger shared memory.
• EMR Serverless provides the serverless.shm.enabled, serverless.shm.size, and serverless.shm.ratio parameters, supporting flexible shared memory configuration at the compute group and job levels in EMR Serverless.

  tip

  After setting the shared memory parameter at the computing group level, if the shared memory parameter is also set at the job level, the job-level setting does not take effect.

Configuration recommendations #

Setting shared memory too large may squeeze the available memory for processes, resulting in OOM. For Ray scenarios, it is recommended to start with the scale of the Ray Object Store and then adjust based on actual object size and throughput requirements.

Procedure #

Prerequisites #

• EMR Serverless has been enabled, and RayJob / Ray compute groups / CustomJob can be created normally.

Setting shared memory at the compute group level: Ray Cluster / Ray Serve compute groups #

You can add shared memory parameters in the parameter configuration Custom parameter configuration section (key-value, separated by multiple lines) when creating a Ray Cluster/Ray Serve compute group or modifying an existing compute group.
For example:

serverless.shm.size=4Gi

Important:

• When setting shared memory parameters for compute groups, the serverless.shm.ratio parameter is not supported for specifying the shared memory ratio. Only serverless.shm.size can be used to specify the shared memory size.
• This configuration applies to the Head and all Worker nodes. If the computing group contains multiple Worker specifications or groups, the platform calculates the corresponding shared memory limit for each role container based on its memory specification.
• After setting the shared memory parameter at the computing group level, if the shared memory parameter is also set at the job level, the job-level setting does not take effect.

For more details on creating and configuring computing groups, see Create and manage computing groups.

Job shared memory settings: RayJob #

When submitting a RayJob, locate Advanced settings in the job's Custom parameters, and enter the shared memory parameter key-value pairs line by line to complete the configuration.
For example:

serverless.shm.size=2Gi

serverless.shm.ratio=0.3

serverless.shm.enabled=false

For more information about RayJob configuration parameters, see RayJob jobs.

Job shared memory settings: CustomJob #

CustomJob supports configuring job behavior through parameters. You can add shared memory parameters when submitting a CustomJob (using the same set of parameters as Ray).

set serverless.shm.size=2Gi;
-- 或者：set serverless.shm.ratio=0.5;
-- 或者：set serverless.shm.enabled=false;

For more guidance on CustomJob operations, see EMR Serverless Custom Job usage instructions.

Job shared memory settings: AI Data Lake (LAS) job #

When using the job management feature of AI Data Lake (LAS) to submit a job, you can also set the shared memory parameter for the current job. In the job running parameters that pop up during job execution, you can set the shared memory size by enabling the "Shared Memory" switch.