KV Store: paying memory to avoid recompute
Without a KV cache, every new token re-attends over the whole prefix. With one, you trade that compute for GPU memory — about 40 GB of it on Llama 3 70B at 128K context, and PagedAttention decides how much of that you waste.