Metadata – we all need it now!
Not so long ago, flow analysis was one of the tools of choice when it came to troubleshooting security or operational problems on networks. Many vendors developed tools which could take these flow records and store them in a data base, so that you could get real-time and historical reports.
However, metadata analysis is now seen as the must have pieces of technology for keeping modern networks running both securely and efficiently. Metadata analysis systems typically use network traffic or packets as a data source. You can typically source these via SPAN, mirror ports or TAPs. The clever part of metadata analysis involves data reduction. This is where you take raw network traffic and capture interesting pieces of data like IP addresses, website names or filenames. In some instances, you end up with a 4000:1 compression ratio. For example, if I transfer a 4MB file across the network, I may capture 1KB of metadata.
The screen shot below from our own (NetFort) LANGuardian system is a good example of this data reduction.