When an application is not responding, or is responding slowly, the finger pointing starts, usually with the finger (hopefully only the index finger) pointing at the network. Now, as a network engineer, you know it’s not your network, but the burden of proof is left to you. So rather than wallow in self-pity, here’s a quick, step-by-step process for quickly isolating the offending application and proving (hopefully) that it’s not your network that’s at fault.
- Find your favorite packet-based network analysis solution. Ours is OmniPeek network analyzer (we hope it’s yours too).Better yet, maybe you already have packet-based network analysis appliances running in your network performing 24x7 monitoring and analysis.
- Choose the best place to monitor the offending application. This is the most subjective step in the process, as your choices may be either quite varied or very limited. It’s important to keep in mind where the users are located, and whether it’s a single user that’s having trouble or a broad range of users. If it’s a single user, try to monitor as close to the user as feasible. If it’s a broad range of users, monitoring closer to the application server will allow you to analyze all of the users simultaneously. If you can capture at both locations and synchronize the time stamps you’ll have an even more complete data set for proving your case.
- With your monitor point established, start collecting packets. If you’re pretty sure the problem is isolated to a specific application, then filtering on just that application will make your analysis a bit easier. The same is true if you’re trying to isolate a single user. Or, with many systems, you can start a “wide open” capture and then filter on the fly as you become more confident of where you want to focus.
- With the offending application/users identified, begin the real analysis.
This is where the real work is done. If you haven’t already, isolate a conversation where the problem is occurring. If your network analysis system employs expert analysis, use that as a guide. Look specifically at the type of expert events being logged, and what layers they are occurring in. Events in the application or client/server layer are leading you towards an application problem. Those at the transport layer are implicating your network.
Figure 1 Events at the application or client/server layer imply application problems while those at the transport layer imply the network.
- Based on expert analysis, drill in deeper.
OK, you’re getting close. Based on the expert analysis it’s pretty clear where the problem is (at least network vs. application), but if you’re going to claim it’s an application problem you need definitive proof. Find a conversation exhibiting some or all of the expert events and expand that conversation to show the packet by packet detail, with timing (often called visual expert or bounce diagram). This diagram just about always provides the information you need to determine if it’s the application or the network.
Figure 2 A bounce diagram provides all the detail you need to determine if a problem is network-based or application-based.
For example, if you see client requests followed by quick acknowledgements (ACKs) and then long delays in data packet delivery (Figure 2) then you can be pretty sure the problem is with the application or the hardware running it.
So in this guilty until proven innocent world of network management, rest assured that the proof you need to keep your network’s reputation untarnished is just a few clicks away.
Author Profile - Jay Botelho is the Director of Product Management at WildPackets, Inc., a leading network analysis solutions provider for networks of all sizes and topologies. Jay holds an MSEE, and is an industry veteran with over 25 years of experience in product management, product marketing, program management and complex analysis. From the first mobile computers developed by GRiD Systems to modern day network infrastructure systems, Jay has been instrumental in setting corporate direction, specifying requirements for industry-leading hardware and software products, and growing product sales through targeted product marketing.