Diagram showing an Amazon EKS scheduler placing two pods on one worker node, sharing an NVIDIA A10G’s 24 GB GPU memory.
Two Pods can share one GPU on Amazon EKS, but only with the right scheduler, driver and per-Pod memory limits. An AWS Machine Learning Blog walkthrough published on November 19, 2026 shows how. It uses NVIDIA Run:ai's GPU Fractioning Operator and KAI Scheduler to put two workloads on one NVIDIA A10G. By default, Kubernetes treats that GPU as one indivisible unit.

This piece covers what the lab validated, what it deliberately did not, and what an admin should check before copying it.

Diagram showing an Amazon EKS scheduler placing two pods on one worker node, sharing an NVIDIA A10G’s 24 GB GPU memory. The problem: one GPU, one Pod​

In the walkthrough's baseline, the NVIDIA device plugin exposes the single GPU as one nvidia.com/gpu resource. Sharing is disabled. Two Pods that each request nvidia.com/gpu: 1 can't run together on that node, even if their memory needs would fit side by side.

The test confirmed this. One whole-GPU Pod ran and the other sat in Pending. That is a familiar pain for teams juggling notebooks, short model evaluations and small inference services on a small GPU pool.

Where this fits among GPU-sharing options​

AWS lays out the choices plainly:

  • MIG gives hardware partitions, but only on supported GPUs.
  • Device-plugin time-slicing shares a GPU without replica memory or fault isolation.
  • MPS allows concurrent CUDA processes. Fault containment depends on configuration.
  • Run:ai GPU Fractioning builds on MPS. It adds Kubernetes scheduling integration, enforced per-Pod memory limits and per-Pod metrics. Its compute modes are time-slicing (the default) and sm-sharing.

AWS's own EKS best-practices guide also lists fractional GPU allocation as a software-scheduling option. It names NVIDIA's KAI Scheduler, part of the Run:ai platform, as the way Pods request fractional GPU resources. AWS says fractioning fits best when you need enforced memory limits on GPUs without MIG and can run fixed-size GPU node groups.

A caveat that appears throughout the post: a memory limit does not reserve proportional compute. The walkthrough tests memory limits only. It does not test fault containment, security isolation or throughput.

Background on the open-source release​

NVIDIA announced the open-source release of the Run:ai platform on October 13, 2026. NVIDIA's developer blog lists four modular components: KAI Scheduler, GPU Fractioning Operator, KAI Resource Management and Workload Map. GPU Fractioning Operator brings CUDA 13.4-native GPU fractioning to Kubernetes and integrates with KAI Scheduler.

NVIDIA's technical blog says the operator is hosted under the KAI Scheduler organization on GitHub and runs next to the NVIDIA GPU Operator, which provides the driver and container runtime setup. Third-party coverage describes the operator as scheduler-agnostic, though it integrates with KAI Scheduler for placement decisions. The AWS walkthrough uses only the first two components.

The tested stack​

AWS says the lab ran in us-east-2 on the following components. It describes the list as lab validation, not vendor support certification or a permanent support matrix.

ComponentValidated version
Amazon EKS1.36
GPUNVIDIA A10G (g5.xlarge), 23,028 MiB
NVIDIA driver / Container Toolkit615.71.10 / 1.20.1-1
containerd2.3.5 with NRI
GPU Operator / device plugin26.7.1 / 0.20.1
KAI Scheduler / Fractioning Operator0.18.5 / 0.1.7

An m7i.large system node hosts the operator controllers and KAI Scheduler. The GPU node carries an nvidia.com/gpu:NoSchedule taint to keep ordinary workloads off it.

AWS validated the walkthrough on October 22, 2026. It repeated every step except the image build on November 18, after updating the GPU node taint, and says the results matched.

Why a custom image is needed​

The main snag is the driver. At validation time, the EKS-optimized AL2023 NVIDIA image offered the R580 and R595 driver branches. Fractioning Operator 0.1.7 requires R615 or newer. R615 is short-lived and isn't the GPU Operator default.

So the sample uses Packer to build an encrypted custom AL2023 image on top of the standard EKS 1.36 AMI. The image installs the R615 driver and the NVIDIA Container Toolkit, and it configures the NVIDIA runtime. Details worth knowing:

  • NVIDIA_DRIVER_VERSION selects the branch, not an exact patch. The build records the installed version in .state/installed-driver-version.
  • For production, AWS advises pinning exact packages and the base image, and validating any driver-branch upgrade.
  • GPU Operator is installed with its driver and toolkit components disabled, because the image owns them.
  • AWS suggests moving back to an EKS-optimized NVIDIA AMI once its driver meets the operator's requirements. The sample scripts currently require the R615 driver and build metadata, so a stock AMI would need script changes.

The workflow, step by step​

The sample repository wraps each stage in a run.sh command:

  1. Clone and configure. Copy the example env file and restrict its permissions. Defaults are us-east-2, cluster runai-fractioning-blog, EKS 1.36 and g5.xlarge. Run ./run.sh check to confirm your AWS identity.
  2. Create the cluster with ./run.sh create-cluster. This gives you a dedicated VPC with one NAT gateway, OIDC and a private m7i.large node.
  3. Build the image with ./run.sh build-ami.
  4. Add the GPU node group with ./run.sh add-gpu-node. It is pinned at one On-Demand g5.xlarge, with eksctl's own device-plugin install disabled.
  5. Install GPU Operator with ./run.sh install-gpu-operator. This also runs a temporary privileged inspection Pod that reports host details. On the validated node that included cgroup2fs and an NRI socket under /run/nri.
  6. Run the baseline with ./run.sh baseline. Expect one Pod Running and one Pending.
  7. Install KAI Scheduler with ./run.sh install-kai. The Helm values enable NvFractions mode and the metrics agent, and they keep GPU-stack version checks on. Success shows FractiondReady, MpsdReady and Ready as True.
  8. Run the sharing test with ./run.sh share.
  9. Test the limit with ./run.sh limit.
  10. Clean up with ./run.sh cleanup. Use cleanup-all only if you no longer need the custom AMI and its snapshot.

What success looked like​

In the sharing test, each Pod carried 8 GiB memory request and limit annotations. Each selected kai-scheduler and the nvidia runtime class, and neither requested a whole GPU. Both Pods ran on the same node and the same GPU UUID. Each saw an 8,192 MiB device and successfully allocated 6,144 MiB.

The metrics deserve attention. The per-Pod series gpu_fractioning_gpu_memory_used_bytes showed about 6,394 MiB (roughly 6.24 GiB) per Pod. That is 250 MiB more than the probe allocated. AWS suggests CUDA context and runtime overhead, but says it didn't measure the breakdown. The practical lesson is to size contracts from observed usage plus headroom, not from the raw number in your code.

In the over-limit test, an 8 GiB contract attempted 9 GiB. cudaMalloc returned out-of-memory, and the probe handled it and exited cleanly. The Pod status was Completed, not OOMKilled. If you run this yourself, that is the expected result, not a bug. It shows allocation rejection, not process termination or broader isolation.

Limits and risks to weigh​

  • Fixed capacity only. The node group stays at one node. AWS did not validate GPU-fraction-aware autoscaling. It says Karpenter support for secondary schedulers is an open feature request. The Karpenter path with the custom AMI in an EC2NodeClass was not tested.
  • No EKS Auto Mode for the custom image. Auto Mode selects and manages the AMI. A mixed cluster is possible but untested.
  • Privileged components. The fractioning daemons use privileged containers, host PID access and NRI/MPS host mounts. Restrict deployment to trusted administrators and add admission rules for fraction requests.
  • Not a security boundary. Where stronger isolation matters, AWS recommends whole GPUs or MIG on supported hardware.
  • A second scheduler. KAI Scheduler is separate from the default one. Shops already running Kueue should decide which component owns GPU queueing.
  • One validated GPU. Hardware support alone doesn't establish fractioning compatibility. Smaller GPUs need smaller contracts and adjusted probes.
  • Cost. The EKS control plane, NAT gateway, instances, volumes and AMI snapshot are all billable. Treat the environment as disposable.

AWS also recommends the EKS node monitoring agent on GPU nodes. In its November 18 run, the agent reported healthy GPU conditions while MPS sharing was active, though that run did not test fault detection. Since GPU Operator already runs a DCGM host engine, point the agent at it rather than using its bundled one.

Takeaway​

This is a careful behavior test, not a benchmark or a production blueprint. It shows that, on a pinned stack, two bounded workloads can share one A10G. It also shows that over-limit allocations fail. The post explicitly declines to claim throughput gains or universal cost savings.

If you run mixed notebook, evaluation and small-inference workloads on a fixed GPU pool, it is worth a trial in a throwaway cluster. Before adopting it, test latency under concurrent load, pin your whole driver-to-scheduler combination, and review the security trade-offs. The AWS post carries a disclaimer that its content and opinions are the third-party author's, and AWS is not responsible for their accuracy.

 

References

  1. Using NVIDIA Run:ai GPU Fractioning on Amazon EKS - Amazon Web Services (AWS) Amazon Web Services (AWS) 2026-11-19T16:13:51+00:00
  2. NVIDIA Run:ai Is Now Open Source as Part of NVIDIA DSX AI Factory Platform developer.nvidia.com
  3. Memory-Enforced GPU Sharing in Kubernetes with NVIDIA CUDA 13.4 developer.nvidia.com