samir-sarwar/samir-vllm: Small LLM inference engine from scratch in C++ and CUDA

This project implements a small LLM inference engine from scratch in C++ and CUDA, teaching first principles of model execution from weights to generated tokens. It includes direct model loading, a custom BPE tokenizer, CUDA kernels for embedding, normalization, and attention, as well as a shared KV cache and support for both static and continuous batching. Open article →