The pcap capture file format has been the universal packet capture format since the early days of computer networking. Almost all capture tools support the pcap format. And while vendors have created new formats over the years, most tools support conversion into the pcap format.
While pcap continues to be used today, it does have some limitations that make other formats more attractive. A new format called “pcapng” has been under development for a number of years. The goal of pcapng is to address some of the shortfalls of pcap and create a flexible format for the future. With the adoption of pcapng as the default format for WireShark 1.8, there appears to be a growing push to support pcapng.
CloudShark’s recent 1.6 release also pushes ahead with improved pcapng support. The addition of pcapng allows CloudShark to unify pcapng’s per packet comment capability with CloudShark’s annotation capabilities.
So, what are some of the advantages of pcapng and why should you make the switch? Are there any limitations that need to be addressed before we’ll see large scale adoption of the new format?
Where did Pcapng come from?
The original “.pcap” file format is part of an API for performing captures on a network interface. In Unix/Linux, this is implemented through the libpcap library. In Microsoft Windows, it is implemented through the WinPcap library. These libraries have been used to create a number of tools throughout the years, both proprietary and open sourced, but all able to output packet captures to disk using the pcap file format.
The proponents of pcapng took a different route and sought standardization. The result is an IETF (Internet Engineering Task Force) Internet Draft that specifies the next generation pcap file format. Through the use of standardized blocks and fields, the pcapng format is logical, extensible, and future proof.
Pcapng Advantages
1. Supports captures from multiple interfaces
While the pcap format does contain some information about the capture interface, the interface information is part of the common header and not stored on a per packet basis. This makes it difficult to mingle packets from different interfaces in the same capture file. The interface information for each packet is lost. This issue is solved by pcapng which allows a capture file to define multiple interfaces using “Interface Description Blocks”. Each packet can be associated with a specific interface. This opens the door to capture files with packets from multiple interfaces. This also opens the door to new analysis approaches that take advantage of this capability.
2. Improved timestamp resolutions
One of the major complaints of the older pcap format is its limitations with regards to resolution on packet time stamps. In the pcap format, each packet contains a header with a time resolution that is only accurate to microseconds (10^-6 seconds). While this seemed totally reasonable when this format was first created, today’s common high speed networks require higher resolution to accurately measure timestamps. Microsecond resolution provides a resolution of 999,999 packets per second; even common 1 gigabit Ethernet links can easily exceed this rate. As a result, microsecond resolution is not be able to distinguish the true timing between frames at these speeds and makes packet capture analysis much less useful.
The pcapg format overcomes pcap’s time resolution limitation by defining a flexible format that can be used to adjust the resolution. Timestamps are now expressed as a single 64 bit time unit, representing the number of time units since 1/1/1970. An associated resolution field (if_tsresol) specifies what the time units mean. The resolution defaults to microseconds (i.e. 10^-6), but may be altered by setting the if_tsresol option in the Interface Description Block, allowing resolutions much deeper for ever increasing network speeds.
3. Embed comments right in the capture file
Troubleshooting network traces can be difficult work, especially when trying to pass information between colleagues or customers. While CloudShark has always supported the ability to annotate packets, it is now possible to generate a pcapng file that embeds both top level and per packet comments. Tools such as Wireshark now offer the ability to edit these comments for improved analysis.
CloudShark has integrated its own comment and annotation abilities with the pcapng format so you can easily import pcapng files, preserving the comments, and export CloudShark captures into the pcapng format.
4. Additional meta data stored in capture file
While top level and per packet comments are incredibly useful, the pcapng format allows other pieces of meta data to be stored in the capture file. As an example, the pcap format does not contain any information about the capture source or how the capture was created. In some cases, this information can be extremely valuable. With pcapng, the Interface Description Block allows a description field, an OS field, and a filter field that can provide additional details about the capture source.
5. Extendable format
One of pcapng’s biggest advantages is it’s extendable file format. The format deploys a generic block structure that allows tools to ignore blocks that are not understood. This allows the format to evolve over time. Specific blocks are defined for interfaces and packets. Additional meta data may be stored in optional blocks such as the name resolution block or the interface statistics block. A number of experimental blocks have also been defined. Tools only need to implement a small number of blocks to support the same level of functionality as the original pcap format. The inclusion of meta data along with the packet really opens the door to a whole new set of compatible analysis tools.
Making the Transition
While pcapng is certainly the packet capture format of the future, not all of the features of pcapng are widely implemented (and it has been around for several years). While many tools exist to perform packet capture, most of them rely on the original libpcap library to generate capture files, which still generates saved captures in the older pcap format.
That said, the adoption of pcapng as the default file format in Wireshark and Tshark version 1.8 will hopefully aid in this push. As more applications, such as CloudShark, arise that are willing to take advantage of pcapng’s extensible fields, more tools will begin to use this highly useful and standardized method of storing network captures. What possiblities can you see for pcapng?
Jason Walls is Director of Technical Marketing for QA Cafe/CloudShark. Jason has been involved with computer networking since 2001, focusing on broadband access networks, customer premises equipment, and home networking. He is currently the chair of the BroadbandHome Working Group at the Broadband Forum. A packet geek like most of his peers, Jason spent ten years in trenches of network testing and analysis at the University of New Hampshire's InterOperability Laboratory before coming on board as the Marketing Director at QA Cafe.
Recent Comments