← Teardowns

Engineering RFC · Worked example

A cache that can't fix the tail

A six-node cache cluster proposed to bring catalog p99 under 150 ms. At the expected 90% hit rate, every miss still costs at least 384 ms.

Objective
p99 catalog read latency under 150 ms at 18,000 req/s.
Hidden assumption
That a 90% hit rate moves the p99. The slowest 1% of reads are all misses.
Flip threshold
Hit rate ≥ 99.0%. The RFC expects 90%.

The full document, both reviews

Stakes$9,400 / mo · read path for every catalog pageAdversarial Review · 1/4

Source draft

591 chars

RFC: Redis cluster in front of the catalog database

Platform team · status: proposed

Problem. Catalog reads peak at 18,000 requests per second and our p99 read latency is 420 ms. The SLO is a p99 under 150 ms. Every catalog query against Postgres takes between 380 and 450 ms at peak. Proposal. Put a six-node Redis cluster in front of the catalog read path. A cache hit returns in 4 ms. A miss falls through to Postgres and populates the key. We expect a 90% hit rate based on the 80/20 access pattern we see in logs. Cost. $9,400 per month for the cluster. Two engineers for three weeks. Success. p99 catalog read latency under 150 ms at peak within one sprint of launch.

Adversarial review

pinned · quoted · computed
01

Pinned objective

p99 catalog read latency under 150 ms at 18,000 req/s.

Underneath: The success metric is a tail percentile; the proposal reasons about the average request.

02

Hidden assumption

A 90% hit rate moves the p99. It does not: p99 is set by the slowest 1% of reads, and with 10% of reads missing, that slowest 1% is all misses.

“We expect a 90% hit rate based on the 80/20 access pattern we see in logs.”
verbatim · highlighted in source

If it’s wrong: The cluster ships, average latency falls, and the SLO still fails at the same 380–450 ms tail.

03

Flip threshold

Cache hit rate at which p99 can fall under 150 ms

≥ 99.0%RFC expects 90%
Figures behind the threshold
Miss path: Postgres + cache lookup384–454 ms
Every miss vs the 150 ms SLOover, always
p99 ≤ 150 ms needs misses ≤1.0% of reads
p99 at the expected 90% hit rate≥ 384 ms

At 90% the SLO fails. The decision turns on hit rate above 99%, not on the cluster.

Cheapest test: Replay one peak hour of read logs against an LRU of the planned size and measure the miss rate before provisioning.

Is your next proposal built on a number like this?

Audit it before it ships. Free, two to three minutes, the draft is never stored.

Audit your draft

More teardowns