When analyzing an application problem, it is critical to capture traffic in the right location, with the right method (SPAN or TAP), on the right server, using the right analyzer – one that won't drop traffic.
If the stars somehow align and this is all possible, the final piece of the 'capture' puzzle is collecting traffic at the right time. The problem needs to be in progress while the capture is running. This may be the most difficult part of the process, since many application problems are intermittent, and cannot be reproduced on-demand.
To address this problem, many engineers choose to use a Network Recorder to collect every packet on a connection over a long period of time, allowing them to analyze problems that have occurred in the past. However, this analysis tool can only capture traffic that has been properly sent to it. As has been repeatedly discussed on this site, a packet can be captured in three basic ways – using a SPAN/Mirror port on a switch or a network tap. The SPAN/Mirror port or tap must be able to handle the traffic stream being monitored, which can be tricky in a high-throughput environment.
When capturing, make sure that the traffic level is well under the threshold for the capture method, and that a capture device such as the Network Time Machine is used, which can capture at line-rate in high-traffic environments, 24/7.
If a packet is lost by the capture method, meaning the SPAN or Tap, this will appear as packet loss to the analyzer. Time may be spent chasing a false alarm such as packet loss, when the issue all along was with packets not making it to the analyzer. Considering these things ahead of time and taking appropriate action will save tons of time when an application problem strikes.
Capture once. Capture smart!Author Profile - Chris Greer is a Network Analyst for Packet Pioneer. Chris has many years of experience in analyzing and troubleshooting networks. He regularly assists companies in tracking down the source of network and application performance problems using a variety of protocol analysis and monitoring tools including Wireshark and ClearSight Analyzer. Chris also delivers training and develops technical content for several analysis vendors. He can be contacted at chris (at) packetpioneer (dot) com.