The Internet NG Project
D5.1 Contents
Real-Time Flow Measurements
The Real Time Flow Measurements (RTFM) working group is an IETF working group. The focus of this working group is on developing a solution for metering Internet traffic flows for accounting purposes, in this context accounting should be taken broadly: characterizing network behavior, network planning, monitoring network performance, verifying quality of network service and attribution of network usage to users.
Already in 1992 an RFC was published on the backgrounds of Internet accounting (RFC 1272). The RTFM took this RFC as a basis for furher work and this has resulted in three new RFCs:
- Traffic Flow Measurement: Architecture (RFC 2063).
- Traffic Flow Measurement: Meter MIB (RFC 2064).
- Traffic Flow Measurement: Experiences with NeTraMet (RFC 2123)
The architecture is inspired by working drafts of the OSI accounting model (Management Framework - Part 4 of Information Procesing Systems Open Systems Interconnection Basic Reference Model, ISO 7498-4).
New Internet developments, in particular DIFFSERV, RAP, POLICY FRAMEWORK and AAA, are strong drivers for RTFM to improve and extend on the current RTFM architecture and Meter MIB.
RTFM Architecture (RFC 2063)
The 'objects' of interest are traffic flows, which is a uni-directional (IP-) packet flow with a specified source and destination. The source and destination of a traffic flow are specified by a set of attributes such as: ...
Typically, a flow exists for a certain period of time, i.e. it starts at some point in time (start-time) and it ceased to exists at some later point in time (stop-time). The stop-time may either be the current time (indicating that it still exists) or some previous point in time (indicating that it ceased to exist).
RTFM Architectural Components
The Traffic Flow Measurement Architecture is based on the following four functional entities:
- Meter: records network activity (for instance by counting packets and bytes) at a specific point in the network as directed by its configuration settings. A meter may process recorded data prior to storage, this data is called usage data.
- Meter Reader: Transports (reliably) usage data from a meter as to make it available for further processing by an analysis application.
- Manager: this entity configures and controls Meters and Meter Readers.
- Analysis Application: processes usage data for information and report purposes.
Component Interactions
Given the above four components, four different interactions are envisaged in the RTFM Architecture: Meter - Manager, Meter Reader - Manager, Meter - Meter Reader and Meter Reader - Analysis Application. These interactions are illustrated in Figure 1.
The nature of these interactions are:
- Meter - Manager Interaction: a Manager configures and controls a Meter. Configuration includes: flow specifications, meter control parameters. (e.g. maximum flow table size, sampling rate).
- Meter Reader - Manager Interaction: a Manager controls and configures a Meter Reader, this includes: Meter's identity from which the Meter Reader should collect usage data, rate at which usage data should be collected from the Meter and the portion of the Flow Table that should be collected.
- Meter - Meter Reader Interaction: A Meter Reader collects the usage data hold by the Meter, the Meter stores this usage data in a Flow Table. The Meter Reader may either collect the entire Flow Table or any portion thereof. Usage data may be retrieved at any point in time, i.e. fully asynchronous.
- Meter Reader - Analysis Application Interaction: Usage data collected by the Meter Reader can be processed further by an Analysis Application. However, the interactions required are considered to be outside the scope of the RTFM Architecture.
There may be interactions between a Manager and an Analysis Application and Manager to Manager interactions, however these are considered outside the scope of this architecture.
Component Composition Rules
A number of component composition rules apply in the RTFM architecture. In short these are:
- A Meter Reader collects usage data from one or more Meters.
- A Manager configures and control one or more Meters.
- A Meter is controled by one Manager.
- A Manager configures and controls one or more Meter Readers.
- A Meter Reader is controled by only one Manager.
Traffic Flows and Attributes
The RTFM Architecture document (RFC 2063) defines a set flow attributes and are defined in more detail in the Meter MIB (RFC 2064).
Table x: Flow Address Attributes
| Source Address | Destination Address |
|---|---|
| Source Interface | Destination Interface |
| Source Adjacent Type | Destination Adjacent Type |
| Source Adjacent Address | Destination Adjacent Address |
| Source Adjacent Mask | Destination Adjacent Mask |
| Source Peer Type | Destination Peer Type |
| Source Peer Address | Destination Peer Address |
| Source Peer Type | Destination Peer Type |
| Source Trans Type | Destination Trans Type |
| Source Trans Address | Destination Trans Address |
| Source Trans Type | Destination Trans Type |
By using the mask attributes it is possible to define flows that are not host-to-host based
but that is e.g. based on a source domain or ethernet segment to another domain or some other
grouping of hosts.
Given a particular flow, defined in terms of flow attributes, the Meter keeps track of
the number of packets and bytes that pass in upward and downward direction seperately.
Meter MIB (RFC 2064)
The Meter MIB support multiple protocol stacks on which Traffic Flows can be based. The protocol entities supported are:
- Adjacent Address: Ethernet (MAC address), FDDI, TokenRing.
- Peer Address: IP, CLNS, IPx (Novell), AppleTalk, DECnet.
- Trans Address: UDP or TCP port number.
Implementation
IBM has developed an implementation of the Meter MIB.
A public domain implementation, called NeTraMet, has been made by Nevel Brownlee of the University of Auckland. The implementation consists of a number of packages:
- NeTraMet - NeTraMet is an implementation of a Meter MIB
- NetFlowMet - Is a software package that allows Cisco's NetFlow to appear as a Meter. As it were, it transforms Cisco's proprietary solution for flow measurements to a RTFM architecture compliant Meter.
- NeMac - NeMac is a combined implementation of a Meter Reader and a Manager.
- NIFTY - Nifty is a combined implementation of Meter Reader, Manager and Traffic Flow Analysis Application.
- FD_FILTER - FD_FILTER is a utility (i.e. program) for processing flow data files. It basically reads a data flow file, performs some processing, and writes it to a new flow data file. Possible processing operations are: computation of flow-rates (in terms of Octets or PDUs), chance file format, filter out flows, filter out NeTraMet statistics records.
- FD_EXTRACT - FD_EXTRACT is yet another flow data file processing utility. It reads a flow data file and produces a column list of a matrix.
- NM_RC - NM_RC is a remote console for NeTraMet, it combines NeMac (i.e. a Meter Reader and a Manager), FD_FILTER a 'Display Formatter'.
The following table gives an overview of how the NeTraMet packages relate to the functional entities as defined in the RTFM architecture.
| Package | Meter | Meter Reader | Manager | Application | 'Utility' |
|---|---|---|---|---|---|
| NeTraMet | X | - | - | - | - |
| NetFlowMet | X | - | - | - | - |
| NeMac | - | X | X | - | - |
| Nifty | - | X | X | X | - |
| Nm_rc | - | X | X | - | X |
| Fd_filter | - | - | - | - | X |
| Fd_extract | - | - | - | - | X |
Furthermore, NeTraMet can be run on several operating systems such as: DOS, Linux, Solaris, SunOS, UNIX. It is standards to use NeTraMet to monitor Ethernet links, but there exists also NeTraMet support for OC3 lines.