Varun Varia — Essays https://thevarunvaria.github.io/ Long-form essays and notes on building, serving, and governing modern AI systems. en-us thevarunvaria@gmail.com (Varun Varia) thevarunvaria@gmail.com (Varun Varia) Wed, 27 May 2026 00:00:00 GMT hand-rolled HTML Why You Can't Serve LLMs Like Regular Models (And How to Fix It) https://thevarunvaria.github.io/llm-inference.html https://thevarunvaria.github.io/llm-inference.html Fri, 01 May 2026 00:00:00 GMT Varun Varia AI Infrastructure If training an AI model is like sending a chef to culinary school for years, inference is putting that chef in a live Michelin-star kitchen on a Friday night at 8pm, taking real orders from paying customers who have expectations, dietary restrictions, and varying levels of patience.

In the world of Generative AI, the single most important number is Tokens Per Second (TPS). Traditional ML inference has predictable characteristics — fixed-size inputs, fixed computation per example, fixed-size outputs. Large Language Models break all three assumptions simultaneously, which creates five deep technical problems that simply do not exist in classical model serving.

Continue reading at thevarunvaria.github.io.

]]>