GPU snapshotting can significantly reduce cold starts for ML inference by restoring an already initialized process instead of loading model weights, initializing CUDA, and rebuilding runtime state every time a GPU worker starts.

But snapshotting a large model creates another problem: there is a lot of state to move.

For Qwen2.5-14B running with vLLM, our checkpoint was around 56 GB.

At that size, small inefficiencies in the snapshot and restore path become expensive very quickly.

We made two changes:

1. Instead of using the kubelet checkpoint API and creating a tar archive, our node agent invokes the checkpoint directly and writes the CRIU images to NVMe.
2. During restore, we modified CRIU to restore memory objects in parallel instead of processing them mostly serially.

For this post, I am intentionally keeping the storage constant ( NVMe )

Same model. Same node. Same local NVMe.

This lets us isolate how much improvement came from changing the software path rather than changing the storage underneath it.

Existing Snapshot Flow

CRIU does not naturally create one large checkpoint file. It produces a directory containing hundreds of image files representing process memory, file descriptors, shared memory, namespaces, and other process state.

Our original checkpoint path used Kubernetes' kubelet checkpoint API.

Conceptually, it looked like:

Application
    ↓
runc / CRIU
    ↓
CRIU checkpoint files
    ↓
Kubelet checkpoint API
    ↓
Create TAR
    ↓
Agent reads TAR
    ↓
Extract TAR
    ↓
Snapshot stored on NVMe

This works well from an API perspective because kubelet gives you a portable checkpoint artifact.

But for a 56 GB GPU snapshot, it means moving the same bytes several times.

CRIU first writes its checkpoint. The checkpoint is then read again to create the tar. The tar is written out. The agent reads it again.

Finally, the archive is expanded into the directory layout that CRIU needs during restore.

We were doing a lot of work just to eventually end up with the same CRIU files we started with.

1) Writing the Snapshot Directly From the Agent

Instead of asking kubelet to create the checkpoint, we moved checkpoint creation into our node agent.

The new path looks like:

Application
    ↓
Agent
    ↓
runc / CRIU
    ↓
CRIU checkpoint directory
    ↓
NVMe

The important change here is not CRIU itself. CRIU still writes essentially the same checkpoint. We simply removed the intermediate tar.

The agent invokes the checkpoint directly with the final NVMe directory as the image path. Once all files are successfully written, the agent writes a manifest which acts as the commit marker for the snapshot.

This also gives restore exactly the format it wants: a directory of CRIU images.

Snapshot Performance

We tested both implementations using the same model, same node, and same NVMe storage.

Snapshot creation went from roughly 21 minutes to under 5 minutes, a 4.3× improvement.

The disk counters made the reason much clearer.

For the 56.4 GB checkpoint:

The direct path writes almost exactly one copy of the final artifact to NVMe.

A 56.49 GB checkpoint resulted in 56.5 GB of NVMe writes — essentially a write amplification of 1.0.

With the kubelet path, the expensive part was not actually extracting the tar.

Most of the additional time came from creating it.

Within the 914-second checkpointing phase of the kubelet path, CRIU itself accounted for around 121 seconds. Roughly 793 seconds were spent creating the archive. So rather than optimizing tar creation, we removed the tar completely.

Restore Is a Different Problem

Once the artifact has been created, the snapshot optimization above does not automatically make CRIU restore faster.

Both snapshots eventually contain the same thing:

~56 GB of CRIU image files on NVMe

Whether those files went through a tar an hour earlier no longer matters.

In our measurements, NVMe snapshots produced using kubelet + tar restored in 43 seconds, while snapshots produced directly by the agent restored in 42 seconds.

Essentially identical. This was useful because it separated the two problems.

Snapshot optimization was about removing unnecessary I/O.

Restore optimization was about making CRIU process the necessary I/O faster.

2) Parallelizing CRIU Restore

Once the entire restore path was running against local NVMe, we looked at CRIU itself.

CRIU's restore path was leaving a lot of available storage performance unused.

A large part of our vLLM snapshot consisted of shared-memory and memfd backed objects. Stock CRIU restored these mostly serially.

Conceptually:

Read object 1
    ↓
Restore object 1
    ↓
Read object 2
    ↓
Restore object 2
    ↓
Read object 3
    ↓
...

That pattern is not ideal for NVMe.

Modern NVMe devices perform best when multiple I/O operations are in flight simultaneously. So we modified CRIU 4.2.1.

We made six changes to CRIU's restore path, including parallel shmem/memfd restore and asynchronous page reads.

But once we ran a clean stock-vs-patched control, the result was much simpler than the implementation: almost all of the improvement came from turning one serial memfd restore loop into eight parallel workers.

The async I/O work helped, but only by about two seconds.

Our fork ended up containing six restore-path changes. The largest improvement came from parallelizing shared-memory and memfd restoration.

Stock CRIU vs Patched CRIU

To measure the actual CRIU improvement, we ran the cleanest control we could.

The only thing we changed was the CRIU binary.

CRIU restore became 2.02× faster.

End-to-end restore improved from 79 seconds to 43 seconds.

Both implementations read exactly 56.4 GB from the same NVMe drive.

Effective read throughput increased from roughly:

Stock CRIU    0.71 GB/s
Patched CRIU  1.31 GB/s

The storage was capable of supplying the data. Stock CRIU simply was not keeping it busy enough.

Where the Improvement Came From

Looking deeper into the CRIU logs made the result even cleaner. Both restores contained 205 memfd objects. Stock CRIU restored those objects serially.

Our patched version distributed them across eight threads.

Everything after that phase was almost identical. The tail of the patched restore took 16.0 seconds. The stock restore took 15.6 seconds.

So almost the entire difference between the two implementations came from parallelizing that one part of the restore path.

End-to-End Result

Putting both changes together gave us improvements on both sides of the GPU snapshot lifecycle.

Snapshot : 4.3× faster

Kubelet + TAR -> Agent Direct
1279 seconds -> 297 seconds 

Restore: 1.84× faster end-to-end

79 seconds → 43 seconds

CRIU itself improved 2.02× 72.6 seconds → 35.9 seconds

All measurements were run on Azure Standard_NC24ads_A100_v4 nodes with an A100 80 GB GPU, 216 GB RAM and local NVMe, using Qwen2.5-14B-Instruct with vLLM 0.9.2. Cold restores were confirmed by observing the full 56.4 GB being read from NVMe, and every restore was validated by generating tokens from the model.

What We Learned

The interesting part of this work was that neither optimization required faster storage.

• The first improvement came from moving fewer bytes.
• The second improvement came from reading the same bytes with more parallelism.

For large GPU snapshots, storage performance is only one part of the problem. Sometimes the fastest way to move 56 GB is to stop moving it more than once. And when those 56 GB genuinely need to be read, the next question is whether the software above the disk is capable of keeping it busy.