Author Profile - Chris Greer is a Senior Network Analyst for Network Protocol Specialists, a Seattle based Network Consulting company. Chris has 10 years of experience in analyzing and troubleshooting networks. He regularly assists companies in tracking down the source of network and application performance problems using a variety of protocol analysis and monitoring tools including Wireshark. When he isn’t hunting down problems at the packet level, he can be found teaching various analysis workshops at Interop and other industry trade shows. Chris also delivers Fluke Networks public courses and protocol analysis themed webcasts. He can be contacted at chris (at) nps-llc (dot) com.
Fact-Based Troubleshooting – Fighting the Three-R’s
Replace ... Reboot ... Reimage.
When we are called on to help troubleshoot a problem in a company, these are the three most common ‘fixes’ that come out of the IT group. Often because of fables and old-wives tales, these three actions are perceived to be the fix for many problems out there, and rarely with any solid proof that these would actually resolve the issue.
As an example, I was at a site not too long ago doing a network assessment, when we found that several people in a conference room were not able to access the internet. They were using a small unmanaged switch for connectivity to the production network. After performing absolutely zero tests, one of the IT people said that the switch was ‘dropping packets again’, and ran out to get a managed Cisco switch. (R number 1).
After that did not fix the problem, all in the conference room decided to reboot their laptops (R number 2). Once again, we were still sitting there with the same problem, and two of the three ‘R’ actions had been taken, with no facts to back up those actions. In the end, it turned out to be a problem with DHCP, but this was only found after taking a packet capture and analyzing the trace.
This example is quite common in our industry. Actions are taken to resolve problems before facts about the problem are discovered. In many cases, those actions don’t address the root cause, which leads to the problem going unresolved, or only going away for a short time.
On LoveMyTool.com there are tons of articles describing tools that can be used to find out facts about an issue BEFORE taking action. Analyzers such as Wireshark go a long way toward proving what is really happening on the wire, then researching a possible resolution before ever making a network, server, or application change.
So why aren’t these tools used when a problem strikes?
As humans, we are creatures of habit, and it can be easy to think that the same thing that worked last time will work again. This is one reason why the three R’s are so common and executed before a problem is analyzed. Just like anything, the more you use an analyzer, the faster you will get at finding the real problem. How do you get started with the analyzer? Stay tuned to LoveMyTool.com for articles about capturing and interpreting problems, as well as descriptions about other tools that will help to find the facts!
So the next time a problem hits, especially if it is the same problem you have seen in the past, try your best to fight the three R’s. Don’t take action on a problem until you have facts to steer you to take the right one.
Note: This article is not saying that the three R’s never work. There are many times when the facts will lead us to replace a card, or perhaps even a switch, but this action is only taken after the problem was thoroughly analyzed.
Continue reading other LoveMyTool posts by Chris Greer »
Recent Comments