Theory of Constraints (TOC), as popularized by the business novel The Goal in 1984, and recently resurrected for DevOps and IT in The Phoenix Project, holds that any given system is limited in throughput by only a few key bottlenecks. Improvements anywhere else won’t speed things up, but improvements at the bottleneck will have a dramatic impact on the whole system.
On a network, there are two kinds of bottlenecks: bandwidth and latency. While these concepts are familiar to most of us, I’d like to highlight them as a base for the techniques in this post. Bandwidth is the ability to move a large amount. I like to think of bandwidth as a cargo ship: lots of containers, lots of capacity, but it takes days to make the journey. Latency is more like a courier: get a small package there as fast as possible. While there’s some overlap between the two – for example, airplanes are fast and have lots of storage – the two concepts have different effects on your data, and different data has a different mix of demands.
Latency is the slowness of moving data. It’s measured in time, and adds up from end to end. Once you lose the time at one location, it’s impossible to make up for it elsewhere. That’s why finding a bottleneck is so important: a single device which adds significant latency slows down the entire trip.