GPU Snapshots for Reducing ML Inference Cold Starts
Before diving into GPU snapshotting, it's important to understand the foundation laid by CPU virtualization technologies. Firecracker is
A little bit
I'm an engineer passionate about tackling challenges and contributing to open source projects. In my free time, I love diving into books fiction, science, tech and human behavior
Stay updated with monthly newsletter
Work
Graph Data Structure for Static Analysis
Static code analysis is analysis performed without running the program by analyzing the structure and logic of the code, it
Container Image Acceleration and Smart Caching
Most of us are familiar with Docker and deploying containers for microservices, yet scaling container deployment for machine learning presents
Thoughts
High-Performance Model Weight Storage and Distribution in Cloud Environments
With the rapid scaling of AI deployments, efficiently storing and distributing model weights across distributed infrastructure has become a critical
Three-Tier Storage Architecture for Fast LLM Inference in the Cloud
Large Language Model (LLM) inference workloads deal with extremely large model files (often many gigabytes) that must be loaded quickly
AI-Assisted “Vibe” Coding - For Work / Play
I began exploring vibecoding for both my personal projects and Inferless, and I experienced firsthand how it enhanced both. In
2024 Wrapped
If I had to sum up 2024 in two words, they'd be "adventure" and "change.