GPU Snapshots for Reducing ML Inference Cold Starts
Before diving into GPU snapshotting, it's important to understand the foundation laid by CPU virtualization technologies. Firecracker is
A little bit
I'm an engineer passionate about tackling challenges and contributing to open source projects. In my free time, I love diving into books fiction, science, tech and human behavior
Stay updated with monthly newsletter
Work
Graph Data Structure for Static Analysis
Static code analysis is analysis performed without running the program by analyzing the structure and logic of the code, it
Container Image Acceleration and Smart Caching
Most of us are familiar with Docker and deploying containers for microservices, yet scaling container deployment for machine learning presents
Thoughts
Beyond Lazy Pulling: Why Signal-Driven Prewarming Is Better for Cold Starts
Container image cold starts remain a major challenge in serverless GPU infrastructure. Inference images often contain gigabytes of CUDA libraries,
Making GPU Snapshot and Restore Faster
GPU snapshotting reduces inference cold starts by restoring an initialized process instead of loading weights, initializing CUDA, and rebuilding runtime
Making AI Argue With Itself
I've been running background agents overnight, queuing up a list of tasks before bed, and waking up to
Building the harness for RL Coding Agents
Today's AI coding agents can generate impressive volumes of code in a single pass, spinning up entire modules,
High-Performance Model Weight Storage and Distribution in Cloud Environments
With the rapid scaling of AI deployments, efficiently storing and distributing model weights across distributed infrastructure has become a critical
Three-Tier Storage Architecture for Fast LLM Inference in the Cloud
Large Language Model (LLM) inference workloads deal with extremely large model files (often many gigabytes) that must be loaded quickly
AI-Assisted “Vibe” Coding - For Work / Play
I began exploring vibecoding for both my personal projects and Inferless, and I experienced firsthand how it enhanced both. In