Editor’s Note - I am very pleased to introduce to our readers, my friend and a very trusted technologist Tom Tosh. Tom has agreed to write a series of articles about his unique and well proven solution and resolution focused techniques. These techniques have been honed from his many years of hands-on experience in helping find the problems that have plagued and solutions that were needed by many network managers and companies that he has helped. His company, Chi-Metrix and his efforts are all based on these “Metrics” that he will be sharing with you in these articles.I hope you enjoy reading this series as I have and I wish everyone Great Success … Oldcommguy.
Part 1: Tools, Process and Metrics for Network Managers
Part 2: Measuring for Business Impact – The “Level 4 Measurement”
Tools, Process and Metrics for Network Managers (By Tom Tosh)
You have probably seen the marketing messages: “See everything - Know everything” or “The Source of Network Truth.” These messages, while effective in getting our attention, imply the following: “Once you put our tools in place, their inherent worth and value will become apparent as if by magic.”
Having spent as much of my career helping network operations teams deploy and make more effective use of analysis tools, I know there is quite a steep ramp between the promise and the reality: Tools alone do not create knowledge. Seeing is not the same as knowing.
In this series of articles, I would like to broaden the focus on tools beyond features alone, and discuss the environment and processes in which tools are used to provide tangible benefits. I would like to share and advance some concepts on how to better evaluate and get more out of the tools we invest in. This, after all, has been the main thrust of my work over the past five years.
If you can’t measure, you can’t really manage. A site dedicated to tools should ultimately be keenly interested in the overall effectiveness and the success of the network manager. (And how do we propose to measure that?) The measurements we choose should also be subject to scrutiny and assessment, since nothing can lead to greater waste of effort than measuring the wrong things or measuring in the wrong way.
The Power of a Simple Metric
One of my early consulting engagements had me helping a large commodities-trading firm deal with a performance problem that was impacting the trading floors during peak hours. The condition had been growing progressively worse over a period of months and finally had gotten to the point where action had to be taken.
I have purposely avoided delving into the technical aspects of this case so that the human and organizational factors might be drawn out more clearly.
After a day or so of collecting data related to the network environment and the symptoms of the problem, the root cause revealed itself to be an issue related to the initial design of the network. Correcting the problem would involve making changes to a multitude of core and distribution-layer switches and, despite the visible impact of problem, the organization was resistant to making any changes. The reluctance could be described as a condition where it was feared that things would go from bad to worse.
So, not only did we have a serious network performance problem that had to be addressed, we also had an organizational resistance to making a positive change that had to be met and overcome first. During some very intense discussions around implementing the proposed changes, I was asked if I could “guarantee” that the proposed changes would make things better while, at the same time, not introduce any new problems.
It was during some brainstorming of how to provide the necessary reassurance that we came up with the idea of creating a test (virtual) network as part of the trading floor system of VLANs, but yet logically separated from them. Moreover, the test network would be set up with the proposed changes in place on the switches feeding it, so that the client could then test for “reach-ability” from any other part of the network.
Part-One Conclusion
To make a long story short, one of the things we also came up with was a test whereby we would move a reference file from the test network in the core to any access segment and back again, while timing the transfer. Then began a weeks-long process of applying the changes in an incremental fashion, and re-running the simple timing test after each change, to track the improvement.
Funny to me was that what I started to silently refer to as the “silly timing metric” became the thing that everyone cared about and asked about. By Week Three, people I never met before would see me in an elevator and ask what the day’s readings were before giving me a thumbs-up. Many were executives of the organization.
Looking back, the metric was anything but silly. As the changes we made caused our recorded file transfer time to decrease from an initial reading of nearly two minutes to around 20 seconds, the positive impact on the performance of the trading floors was evident to all. Everyone accepted that the metric directly correlated to what was most important to them.
What I also found interesting was that, after the first round of changes had been made, we still noted some problems that still existed on the network that needed to be corrected. But the organization reverted back to its original conservative approach towards change – getting our metric down from 20 to an ideal state of around 18 seconds was not worth the risk to them.
This is a good place to end Part One. In the next installment, I will use this case to introduce the concept of four distinct levels by which we can measure or assess anything that’s worth measuring.
Part 1: Tools, Process and Metrics for Network Managers
Part 2: Measuring for Business Impact – The “Level 4 Measurement”
Author Profile - Tom Tosh is a seasoned trainer and proven senior level consultant and analyst. He spent the last 10+ years at Network General where he was there Senior Consultant and Project Manager. He has recently founded Chi-Metrix where Tom prides himself on bringing his considerable skills to help network customers with their network issues. From planning to troubleshooting, Tom is one of the best and is a long-time Oldcommguy’s associate and a proven solution focused technologist!
Tom’s skills, training and experience allows him to -
- Diagnose network and application performance issues.
- Perform network assessments, health checks and application baselines
- Provide coaching and training to help IT groups make better use of network tools.
- Derive actionable information from captured trace files
- Evaluate effectiveness of problem management practices (per ITIL)
- Develop visibility assessments to better target investments in tools
- Design and manage projects related to application rollouts and network changes.
Tom has served a wide variety of clients (corporations and non-profit organizations, as well as local, state and federal government agencies) all over North America, Australia and Japan.
Tom can be reached at - tom.tosh (at) chi-metrix (dot) com.
Recent Comments