Network Flow Monitoring: The ABCs of Network Visibility
2021-07-30 | 8 min read
In this blog I will review an important concept of network management: flow monitoring. A network flow is the series of messages exchanged between the opening and closing of a communication session. Flow data is used by tools like network flow analyzers and flow collectors to generate insight into network performance and assist with problem resolution. Flow data is an aggregated form of data and is also referred to as metadata.
Flow monitoring is a useful way to generate information engineers use to troubleshoot network issues. Flow data is aggregated and therefore different from packet data, which is a copy of the detailed data inside network packets. Most monitoring tools process only one or the other type of data. Some organizations treat the two types of data as mutually exclusive, but combining the two provides administrators with superior insight for issue identification and resolution.
NetFlow was the first flow technology and was developed by Cisco in 1996 as a proprietary protocol. The company later harmonized their tool with the standards known as Internet Protocol Flow Information Export (IPFIX). NetFlow is considered by many to be a de facto standard. There are also other flow formats such as jFlow (Juniper), NetStream (3Com/HP), and sFlow (various vendors).
Flow monitoring is often the best way to resolve intermittent network performance problems. Tools track flows related to applications and services over all areas of the network and offer insights into bandwidth utilization. Flow monitoring can also map out historical trends for capacity planning, as well as proactively identify security issues.
The strength of flow monitoring is that it gives administrators a very effective high-level view by providing timestamps, senders’ / receivers’ IP addresses, the ports communicated on, the length of the conversation, and the amount of data transferred. Because flow data is summary information it doesn’t take up a lot of storage space. This means more historical data can be archived to allow analysts to go back months or even years in time to research complex performance issues.
The downside is that flow data doesn’t provide nearly the level of detail that full packet capture data provides. While flows are useful for alerting the operations center to potential issues, they can’t necessarily explain exactly what happened to rebuild files that have been corrupted. The other issue is that many flow collectors are configured to process sampled data, rather than looking at every session.
TYPICAL USE CASES AND BENEFITS
Real-time bandwidth monitoring
Administrators use real-time monitoring tools to identify the interfaces, links, applications, users, and protocols taking up network bandwidth. A flow monitoring tool can examine bandwidth utilization over the LAN, WAN links, and specific devices. It also identifies internal and external traffic sources and destinations. Flow monitoring allows administrators to know the Top Senders, Top Protocols, and Top Applications that consume use up bandwidth.
Applying Quality of Service policies
Flow monitoring can help administrators manage QoS policies for specific services. By default, each network channel operates on a best-effort basis—every application gets equal priority, whether it is a business-critical VoIP service, or a user streaming video content. Enterprises must set QoS polices to ensure business-critical applications get sufficient bandwidth.
Identifying historical trends
By analyzing traffic patterns and usage over a period of time, network flow monitoring tools can identify trends in bandwidth usage and potential bottlenecks. Historical data can also aid administrators in capacity planning and verifying bandwidth-based billing, including “burstable” bandwidth services.
Identifying abnormal bandwidth usage
Flow monitoring also proactively identifies DDos attacks, unauthorized downloading, and other suspicious and potentially malicious network behavior. Flows can be your best option for security forensics and analysis. Monitoring tools automatically identify high traffic flows to unmonitored ports, expose unauthorized applications like file sharing and video streaming, monitor traffic volumes between pairs of source and destinations, and detect failed connections.
Considerations For Flow Monitoring
Efficient, dual data generation
You can use the same network visibility platform you use to aggregate and process network packets to manage delivery of flow data to your flow monitoring tools. A visibility platform lets you offload flow generation from routers and other network devices to help them work more efficiently. You will need a solution that generates the specific type of flow data your tools are designed to process. Some network visibility platforms, such as the Keyisght's Vision ONE network packet brokers (NPBs), generate your choice of NetFlow or IPFIX compatible data.
You want to make sure that the solution you use for generating flow data has enough processing power to keep up with your traffic volume and support all your flow monitoring tools. Keyisght Vision ONE NPBs, for example, are capable of simultaneously generating flow data, decrypting secure traffic, and filtering data based on application type. Keyisght’s high-performance processing engine generates flow records for up to 300K TCP sessions per second and supports up to ten flow monitoring tools (or collectors).
Enrichment of flow data
Another benefit of a platform that handles both flow and packet data is the ability to enhance flow data with value-add extensions. With Keysight’s solution, you determine what additional information to send to your monitoring tools. You can include geographical information, application ID or name, browser type, and SSL cipher as part of the information flow to your tools. For subscriber-aware reporting, you can provide detail on applications and handset/device type for mobile users.
Use flow data for efficient on-the-fly monitoring and keep your team up-to-date with network events as they happen. And strengthen overall network monitoring by also deploying packet-based data capture and monitoring. With access to network packet history, you can quickly drill down to packet level, examine incidents and determine their root cause and severity.
Combining these two monitoring techniques helps network and security analysts stay on top of the mountain of alerts they receive to ensure an unexamined issue doesn't escalate to become a serious outage or network breach.