Imagine hiring a brilliant assistant, handing them a 200-page manual, and asking a question. They read all 200 pages, answer, and then promptly forget everything. Next question? They read all 200 pages again. And you pay them by the page. That’s roughly how large language models (LLMs) work by default. Prompt caching is the fix. […]
Read More →Your agentic app just ran a search. The tool returned 500 results as JSON. Your agent appended all of it and fired off an API call — 45,000 tokens to answer a question that needed maybe 4,500. Tejas Manohar, a senior engineer at Netflix, hit this problem every day. He was running out of tokens […]
Read More →Large Language Models (LLMs) all predict text, but they differ a lot in how they follow instructions, use context, handle tools, and optimize for safety, speed, or cost. If you treat them as interchangeable, you’ll ship brittle prompts. If you treat them as different runtimes with different affordances, you’ll get reliable results. This post explains the major differences across […]
Read More →A practical guide for software engineers navigating the evolving landscape of Large Language Models Introduction: Why This Matters As a developer in 2025, you’re likely interacting with Large Language Models (LLMs) daily—whether through coding assistants, chat interfaces, or integrated APIs. But here’s the thing: not all LLMs are created equal, and the way you communicate with […]
Read More →Listen up, engineers. We are past the “wow” phase. You know what an LLM is. You’ve likely integrated an API, generated some boilerplate code, and maybe even built a RAG pipeline. But here is the hard truth: if you are pasting the exact same prompt into GPT, Claude Sonnet, and Llama using the same strategy, […]
Read More →