Editor Profile - Tim O’Neill is an independent technology consultant. He has over 30 years experience working in the WAN, Analog, ISDN, ATM and LAN test market. Tim has worked with companies like Navtel, Network General, Ganymede and ClearSight Networks and is now helping companies get lab recognition and technology verification. Tim is also the Chief Contributing Editor for LoveMyTool.com, a website designed to help network managers gain access to valuable information and real solution stories from other customers. Tim is a patent holding, published and degreed engineer, who has seen this technology grow from Teletype (current loop) data analysis to today’s 10 Gigabit LAN’s focused on business applications with heavy compliance demands. Tim can be reached at oldcommguy (at) bellsouth (dot) net.
Is RSPAN something one should be using? Or is it something to be avoided?
Recently I wrote an article about SPAN that details the overall issues with using SPAN ports as the monitoring data access point, providing evidence that SPAN access for monitoring and analysis is not in the best interest of network managers who need to fully understand their networks.
The list of problems with using SPAN ports for network monitoring and analysis are many and in particular, I pointed out that SPAN ports will not meet the demands of compliance requirements for high fidelity data access nor for analysis of synchronized traffic such as VoIP.
So you should not be surprised when I tell you that RSPAN is even a bigger potential mess and that its usage is not only not recommended but is in fact prohibited altogether in many high-end customer sites!
What is RSPAN?
RSPAN stands for Remote Switched Port for ANalysis.
RSPAN is similar to SPAN except that it utilizes a “remote” switch to send data back through the “production” network for monitoring.
RSPAN has all the issues of SPAN and more
Recall that,
- SPAN’ing or mirroring changes the timing of the frame interaction (what you see is not a true representation of your network traffic).
- The SPAN algorithm is not designed to be the primary focus or the main function of the device like switching or routing. This means that the first priority is not SPAN and if replicating a frame becomes an issue, the hardware will temporally drop the SPAN process.
- If the speed of the SPAN port becomes overloaded frames are dropped.
- Proper SPAN port setup requires that a network engineer configure the switches properly and this takes away from the more important tasks that network engineers have. Many times configurations can become problematic (constantly creating contention between the IT team, the security team and the compliance team).
- SPAN port drops all packets that are corrupt or those that are below the minimum size, so all frames are not passed on. For IPv6 knowing the amount of fragments is an important measurement criterion.
All of these events can occur and no notification is sent to the user, so there is no guarantee that one will get any, much less all, the data required for proper analysis or reporting/statistics.
Now we add RSPAN, this means that the above issues are compounded with more delay, more loss, more filtering and data grooming. In addition, the RSPAN packets are transported over the production network. This is just unreliable for any type of serious monitoring or analysis techniques.
How does RSPAN really work?
The following shows Cisco’s schematic for RSPAN.
The Source switch is taking frames from the source port and encapsulating them in newly-created RSPAN frames. The switch then sends these new RSPAN frames through the production network to a Destination switch which then removes the encapsulating headers and presents the frames to a capture device on the destination port. From a simple frame timing and performance monitoring perspective, what appears at the ultimate destination port bears no reliable resemblance to what actually occurred at the source port or on the network being monitored.
Summary
SPAN does not deliver high fidelity data access and is a potential failure point in your network. RSPAN just adds to the issues of SPAN and is not recommended for any application of monitoring or analysis for today’s networks.
If one must have remote access to data, my recommendation is to use real access taps such as those manufactured by Garland Technologies or a secure remote access device like the Anue Systems data access switch. This will give you all the packets required for meeting today’s critical analysis and monitoring needs (and save you lots of headache).
Comments from Network Experts –
Betty DuBois – Sniffer Expert, Network Consultant and Wireshark instructor
Tim O’Neill hits the nail on the head again. So often I will hear customers tell me “the sales guy said that with RSPAN I could see my whole network without having to move the capture device”. Just remember the acronym for the OSI model. Please Do Not Take Sales Person’s Advice on this one. If all you need is something quick and dirty just to see if the client and server can talk to each other, then it is fine. But RSPAN is not a viable solution if you are trying to prove whose fault it is, the delta times for the packets are completely skewed.
Scott Haugdahl – Network Expert and Founder of Bitcricket
Editor's Note - Scott has also written previously about RSPAN, please refer here.
Tim, I agree that RSPAN is generally bad for enterprises with a high volume of traffic. There are certain situations where you cannot for practical reasons, put a tap on every segment in your infrastructure, especially low volume local or remote edges. In such situations, save taps for higher volume traffic and permanent installs and use an SNMP tool to monitor switch stats for port CRC counts (i.e. packets not passed through the SPAN).If the RSPAN VLAN is sharing the same physical segments as corporate data, one might look into lowering the switch VLAN priority for passing RSPAN traffic. Sure, this can worsen the timing but I can still obtain valuable troubleshooting information even if the packet timing at the analyzer is not precise. Finally, using a security ACL as a filter (such as TCP port 80) on the RSPAN VLAN (supported by higher end Catalyst switches in certain configurations), is worth a look to cut down the traffic volume. Regardless, tread *very* lightly!
Tom Tosh – Sniffer Expert – Senior Consultant and Network Expert
RSPAN is one of those features that, when I first heard the concept, I thought “Hmm…. Interesting,” but on a second look, after coming to understand how it works, soon asked, “When and how could that possibly be useful?” I believe that a comprehensive visibility strategy for networks requires some understanding and consideration of RSPAN, its limitations and its caveats. When we have no other visibility into a remote switch, and the answers we seek regarding the traffic seen at any specific, user-based port on that switch are not performance-based, RSPAN can likely provide some usable information. And in that statement are the key caveats of RSPAN:Limit RSPAN sourcing to a single, user-based port in order to minimize RSPAN’s impact on the switches and network path. This means no multiple-port spanning, and no spanning of trunk or server ports.
Apart from basic network conversations seen within the RSPAN-sourced frames – for example: “this user is definitely talking to this server,” refrain from making any conclusions as to the performance occurring on those conversations. If you have any alternative means of getting the limited answers that RSPAN can provide by all means use them prior to resorting to RSPAN.
Jenny Wilson – Sniffer Expert, Network Expert, Consultant and Trainer
Excellent article Tim! When it comes to RSPAN, if we're not aware of the risks and disclaimers, we can unwittingly sabotage ourselves - either causing additional problems, or spending months troubleshooting a problem that could have been solved in a few minutes with accurate information.
Tony Fortunato - Network Performance Consultant, Certified Wireshark and Fluke Networks instructor with over 15 years experience
Tim you are right on as there are many problems with SPAN and RSPAN! I’ve personally experienced when spanning a server port and capturing from the client’s computer creates duplicate packets and cause false positives. I have seen cases where the worst case scenario involves technician blindly spanning multiple ports or an entire VLAN causing significant switch performance degradation and possible complete switch failure. I’ve been onsite where arguments leading to physical threats between the IDS group who needs the SPAN port and the Network Technician who wants to use the same port for his troubleshooting. Full and direct access is always best!
Richard Bejtlich - Founder of TaoSecurity and author of many security books including Tao of Network Security Monitoring and Extrusion Detection
This is the simplest way for me to compare SPAN ports to taps: a SPAN port is a girlfriend, but a tap is a wife. It takes a real level of institutional commitment to install a tap, and the rewards are long-lasting. A SPAN port is a temporary fling subject to break-up (i.e., deactivation). Furthermore, I really liked [Tim's] emphasis on SPAN configuration as a change that must be allowed by the change control board in any semi-mature IT shop. The only CCB action needed for a tap is the initial installation. Any change to a SPAN port configuration should be authorized by the CCB.
Recent Comments