LoveMyTool - Open Community for Network Management and Monitoring

Still Not Convinced That Application Block Size Matters? (by Mike Canney)

Small application block size kills your application performance.  It not only increases your application turns but can have other detrimental effects.  

In this video, we examine a lab replicated example of a customer that was experiencing poor file transfer performance using FTP.  I have seen this same root cause of this problem in many applications including: MS SQL, VB front end to Data Bases as well as out of the box IIS implementations.

 

 

 

    

Author Profile: Mike Canney is the President of getpackets.com, specializing in providing application and network performance consulting services. 
 Over the past 23 years Mike has helped 100's of companies identify and resolve their application and network performance issues. Mike has also developed coursework and taught thousands of engineers how to identify, remediate, and prevent network and application issues by analyzing traffic flows at the packet level. Mike can be contacted at canney (at) getpackets (dot) com 

 

Troubleshooting Application Performance Issues (by Mike Canney)

This presentation was recorded at the Sharkfest 2011, Wireshark Developer and User Conference, held at the Stanford University between June 13th and 16th.

As network speeds continue to grow at an unprecedented rate, the need to be able to quickly analyze multi-Gigabyte sized trace files has become increasingly important. In this session, students will learn how to capture, monitor and analyze extremely large trace files to quickly resolve some of the most common application performance problems seen today.

Topics to be covered:
• How to quickly create a capture to disk appliance
• Using Pilot to go “back in time” for troubleshooting issues
• Using Pilot to filter and send only pertinent data to Wireshark
• Top Causes for application performance issues: Application Turns, TCP Window size, Inflight Data
• Using Wireshark Profiles to help troubleshoot SMB issues

Mike Canney, well-versed in multiple sniffer technologies, specializes in providing application and network performance consulting services: specifically Application Network-ability Assessments (ANA), network performance troubleshooting, and deep level packet analysis. Over the past 22 years, Mike has helped hundreds of companies identify and resolve their application and network performance issues. Mike has also developed courseware and taught engineers how to identify, remediate, and prevent network and application issues by analyzing traffic flows at the packet level. Mike has been a guest speaker at many industry trade shows (Networld Interop, Cisco Networkers, e.g.) throughout the United States on the topic of application performance analysis.

Right click to download the corresponding PowerPoint presentation.

Continue viewing other Sharkfest videos »

A WAN in a Can! (by Mike Canney)

I often run across the need to do “What if” type studies for many of my clients.  What if we move the application server from one data center to another?  What if we add an additional office to the network? How much bandwidth are we going to need?   I have also used many open source tools to assist with this.  Dummynet, NIST Net, WANEM etc.  While I am a firm believer in open source, sometimes these “free” tools just don’t cut the mustard. 

Recently I have been doing a considerable amount of research and testing using the popular WANEM.  I have to say, WANEM is nice.  It’s easy to use/setup and seems to have most of the features needed for small scale simulations.  It was only when I started really cranking up the traffic that it started to fall apart.  Now granted, I was using a decommissioned PC (which I suspect most people would) and it wasn’t the beefiest of boxes. 

When I started looking at the packets with Wireshark, I was noticing a great deal of dropped packets.  That would have been great if I had told the emulator to inject packet loss :).  This really got me looking for an alternative solution.  As I am sure you are aware, when doing this type of work (predicting response times) it is critical to be correct.  

The first time you put your name behind a prediction and it goes south, will be the last time anyone ever seeks your advice.  I just don't have faith in a software based emulator after what I experienced when actually analyzing the packets while using WANEM.  There are too many variables with a software based emulator.  It's great for messing around in my own lab, but would never use it when my reputation is on the line.

As a consultant, I knew what I needed had to be small and it had to be able to fit in my backpack.  So the search began…

I was able to get my hands on a Linktropy Mini2 from Apposite Technologies (http://www.apposite-tech.com).  I can now say, the search is over.  This thing is fantastic!  It has a slick web interface for configuration and could easily stand up to any traffic I could throw at it (without unwanted packet loss).  This sub $2,000 device should be in any serious network engineer’s tool bag.

 

    

Author Profile: Mike Canney is the President of getpackets.com, specializing in providing application and network performance consulting services. 
 Over the past 22 years Mike has helped 100's of companies identify and resolve their application and network performance issues. Mike has also developed coursework and taught thousands of engineers how to identify, remediate, and prevent network and application issues by analyzing traffic flows at the packet level. Mike can be contacted at canney (at) getpackets (dot) com 

 

Pilot, Where Have You Been All My Life? (by Mike Canney)

Let me start off by saying, over the past 22 years, I have used just about every protocol analyzer commercially made.  The one thing that all of these analyzers had in common is that none of them could handle large trace files well. 

Back in the early 90’s, a large trace file was maybe around 64 MB.  Today, as a consultant, I am constantly being sent trace files that are over 1 Gigabyte in size. Most analyzers come to a screeching halt with these types of trace files.

 I have a challenge for you.  Grab your favorite protocol analyzer and pull in a 5 Gig trace.  What? Your analyzer can’t do that?  Pilot can, and it can do it in ~10 seconds!

Pilot slices through large trace files like a hot knife through butter.  I am not kidding.  I was blown away the first time I saw this product. 

I was recently investigating a customer problem where end users were complaining the “Network” was slow (imagine that).  I asked the typical questions of: Who is having the problem?  What application were they having problems with? What time did the problem occur?  Etc.

I also got the typical response….

The Network is Slow! (by Mike Canney)

Author Profile: Mike Canney is the President of getpackets.com, specializing in providing application and network performance consulting services. 
 Over the past 22 years Mike has helped 100's of companies identify and resolve their application and network performance issues. Mike has also developed coursework and taught thousands of engineers how to identify, remediate, and prevent network and application issues by analyzing traffic flows at the packet level. Mike can be contacted at canney (at) getpackets (dot) com

 “We need more Bandwidth!”

-A tale of an ancient file transfer protocol.

 “We need more bandwidth”.  As a network engineer, have you ever heard that before? My guess is that you just heard that yesterday morning at 8 AM, or have a couple of voice mails waiting for you, right now, with those exact words. 

 I was at a client site a while back where they were doing a great deal of file transfers between two data centers.  They upgraded to a DS3 because they wanted to decrease the time of these file transfers.  When the file transfers times were still unacceptable, they called me in to try to figure out what was going on. 

 The customer was copying files to and from a share drive and couldn’t figure out why they were only seeing ~2.3Mbps (maximum) throughput on their 44Mbps circuit.  I took a quick packet capture and this is what we found.

As you may be aware, a file utilizing a share drive uses SMB (CIFS) for its file transfer protocol.  Server Message Block has been around a LONG time.  It has gone through many slight changes over the years, (new dialects, marketing makeovers, etc.).  SMB/CIFS has become a necessary evil for most organizations and most have just accepted that it doesn’t always work well in WAN environments. 

If my users need to copy files from their desktop to a server we just map a drive and copy the file over.  This works great in a LAN environment. It’s fast, it’s easy, and it all makes sense.  So why when I copy files over the WAN does it all of the sudden become the dog that it is?

The Hidden Value of End User Monitoring (by Mike Canney)

Author Profile: Mike Canney is the President of getpackets.com, specializing in providing application and network performance consulting services. Over the past 22 years Mike has helped 100's of companies identify and resolve their application and network performance issues. Mike has also developed coursework and taught thousands of engineers how to identify, remediate, and prevent network and application issues by analyzing traffic flows at the packet level. 

The Hidden Value of End User Monitoring

End user performance monitoring is currently a very hot topic amongst IT management these days.  Take a quick walk around the vendor pavilion at Interop and you will count at least 5 companies touting that this is what they do and that they do it better than anyone else.

The concept totally makes sense.  As a Network Engineer for that past 22 years, the ability to see what the end users are doing as well as being alerted to “slowdowns” before the phone rings is very cool.  This value alone is worth investing IT budget dollars into but the real value of a solution like this is a “hidden” value.

Most of us engineers have taken calls when things were reported, as “The Network" is slow.   The typical first step is to get your analyzer in place, grab trace files and begin troubleshooting.   The really frustrating thing is when everything looks fine, and the problem just “magically” went away.  I am sure most of you have spent countless hours troubleshooting these types of issues.  I have an old buddy of mine that used to jokingly say, “Well, the Sniffer must have fixed it”.  Why is this the case?  Why do these problems seem to all of the sudden go away once you get your analyzer in place?

 I am going to go out on a limb here and say that most of the time…

THE END USER IS NOT BEING HONEST!!!  Duh!

Ok, well that may be a bit strong but I have seen in countless cases where we are proactively monitoring end user experience that help desk calls go dramatically down when the user knows that you can see what they are doing.

 I had a customer tell me the other day that a user of his was constantly complaining that SAP was slow.

 The user would constantly raise a stink with his management that the “network” was slow and that he couldn't get his work done because of this. 

 The end user knew that SAP was a hot button with management and he could get all kinds of management attention if he just mentioned the word “SAP”. 

 Little did the end user know that their IT staff was now watching every end user transaction on the network!  This time, when he complained that SAP was slow, the network staff pulled up the end user’s transaction report at the time of the supposed “slowdown of SAP” and it showed that the user’s SAP transactions were in the 1-2 second range. 

 On the other hand, his FaceBook and YouTube response times were definitely slow.

 How many hours do you think you have wasted over the years trying to troubleshoot a problem that never existed in the first place?  To the end user, SAP was the same as FaceBook and YouTube.  They were all in the “network”.

Now the fun begins: What do you think happened when their manager saw that SAP was fine but in actuality, what the user was really complaining about was his (non work related) facebook page being slow?

Exactly, no more phone calls complaining that the network was slow from this end user.  Ever!

Good Luck! Have fun Monitoring! Mike.

Continue reading other LoveMyTool posts by Mike Canney »