Llama.cpp and the official ggml WebGPU backend are here, finally letting you run GGUF-format large language models directly in your browser using your local GPU. The new backend cuts the cord with proprietary native clients and convoluted WebAssembly setups — it's pure on-device inference, with all data staying on your machine. This opens up a zero-configuration local compute pipeline for the web.
A paper published on May 20 reports that the WebGPU backend introduces static memory planning and efficient model loading, reducing VRAM overhead on the web by 29% to 33% compared to existing frameworks. On mainstream GPUs from Intel, Apple, and Nvidia, decoding throughput jumps by an average of 45% to 69%.
The web demo runs on the open-source wllama library, and recent low-level optimizations have achieved even better memory control than the paper describes. Llama.cpp can also be compiled natively using Google's Dawn, a C++ implementation of WebGPU, providing a benchmark for comparing Vulkan and WebGPU performance under the hood.