Last time I hand-rolled a 256-way sharded map and crowned it the best concurrent cache in Go’s standard library. The top question back: what about the popular libraries? So I added xsync and otter to the same pinned benchmark. The answer surprised me — for reads, the library wins.
I built the clients → API server → queue → backend pipeline you’ve seen a hundred times, pointed a ten-second load spike at it, and watched it spend the next minute finishing a quarter-million requests that nobody was waiting for anymore — with its throughput graph perfectly, reassuringly flat. TL;DR: under overload a queue’s throughput (work finished per second) can hold rock-steady while its goodput (work finished while a client still wants the answer) collapses to zero — and a client retry loop keeps it there long after the spike is gone. The culprit is the queue discipline. Changing FIFO to drop work the backend can’t finish in time took goodput from 19% back to 100% and shrank the peak backlog 100×, in about thirty lines of code. ...
I built the same in-memory cache six ways with the Go standard library and benchmarked them across read/write mixes and core counts. Lock striping wins by up to 8×, sync.RWMutex turns out to be a trap, and one design gets slower the more cores you add.