Ok, we get it. It's not the network.
Over the past few years, network engineers have slowly been winning the battle of blame. Even though most people still say "The network is slow" when there is a performance problem, many engineers now have the tools to prove otherwise. Whether they use SNMP to check their infrastructure devices, flow tools to watch for utilization spikes, packet analysis engines to watch-dog applications, or automated performance analyzers with built in alarms, most environments have some type of monitoring in place to help the engineer prove his case.
However, the problem now becomes - what next?
Not long ago I was on a consulting job, assisting a client in tracking down the root cause of a slow file transfer. The server guys were upset that the network was only providing their precious application with 25% of the bandwidth they expected, and demanded that the network engineer fix his slowness problem. After a few packet captures, we clearly saw that the network was exonerated. We saw no packet loss, had good roundtrip times, and there were no ICMP messages warning of flakey links. The network was clean and clear as a whistle.
However, what was the next step for the network engineer? Was he supposed to toss the capture to the server guys and explain how it is no longer his problem? What are server/application guys going to do with the capture? - Likely nothing.
So, in a standoff like this, who has the responsibility to continue troubleshooting and find the root cause?
In my opinion - the network guy.
He is the one that has the tools to collect and interpret the traffic, while the server and application teams likely don't. Even if the tools make the network look like a dream, the network engineer will need to step in and help the application team find root cause. This means he will need to take ownership of the transport layer and above, providing other teams the details they need to be able to take action.
Now this may not sound fair, kind of like being responsible for somebody else's problems, which is true. However, it's time to get over it. Problems linger. Blame games go on far too long. It's time for the network engineer to use the tools he has available to go beyond "It's not the network". It's time for that role to take responsibility for finding the root cause, even if it isn't his job to completely fix it.
Sound harsh? Well, armed with the right gear it's not! In fact, if the network engineer has the right tools and can read them, he'll become a Network Ninja, instead of just a finger pointer.
Author Profile - Chris Greer is a Network Analyst for Packet Pioneer. Chris has many years of experience in analyzing and troubleshooting networks. He regularly assists companies in tracking down the source of network and application performance problems using a variety of protocol analysis and monitoring tools including Wireshark and ClearSight Analyzer. Chris also delivers training and develops technical content for several analysis vendors. He can be contacted at chris (at) packetpioneer (dot) com.