Is 99.9x% cache hit always better?

By ozer_23 · 2026-08-11 · 1 points · 0 comments

Support a problem needs at least 10k tokens new information. If cache hit is 99% ... input 0 cache created 10k (prefect) cache read 1M! Extremely, if a clever agent get all 10k information, and give the answer directly, the cache hit is 0%! Cache hit rate should be healthy, but…

Open the full discussion on BetterNews