nfacctd: NetFlow v9 / IPFIX, correlate Options data to Flow data (ie. VRF ID -> VRF Name)

[ThomasJLLN]

Description
Hello,

First of all, thank you for developing & maintaining the pmacct tools suite that we have been using for a few years in our company!

I'm trying to setup nfacctd to collect IPFIX flows. I would like to retrieve the correspondence between vrf_id_ingress & vrf_name from Arista devices.
I wanted to clarify my understanding of the possibilities of nfacctd to perform my request.

To ease the understanding of my question, I used the following terms:

• option template: template sent to describe the mapping format between vrf_id & the vrf_name.
• option data: data that corresponds to the option template above (vrf_id:1 <-> vrf_name:toto)
• flows: IPFIX flows that contain information about connections from different IPs

Is there a way to automatically integrate the vrf_id_ingress <-> vrf_name mapping from the option template & corresponding option data sent by the network device to the flows?

It corresponds to having flows containing the following information: src_ip, dst_ip, src_port, dst_port,..., vrf_id_ingress, vrf_name

Following my research, the only way I found to retrieve the vrf_name is in the QUICKSTART page, section Va. NetFlow daemon & accounting NetFlow v9/IPFIX options
It seems possible to extract and log the option data in another output than the flows, but merging it directly to flows seems impossible.

Could you please let me know if there is an option to merge the vrf_name directly into data flows?

Thank you for your help.

Version
NetFlow Accounting Daemon, nfacctd 1.7.9-git [DAILY]

Arguments:
'--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu' '--program-prefix=' '--disable-dependency-tracking' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--localstatedir=/var' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--enable-static' '--enable-jansson' '--enable-kafka' 'build_alias=x86_64-redhat-linux-gnu' 'host_alias=x86_64-redhat-linux-gnu' 'CFLAGS=-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' 'LDFLAGS=-Wl,-z,relro -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld' 'PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig' 'CXXFLAGS=-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' '--enable-l2' '--enable-traffic-bins' '--enable-bgp-bins' '--enable-bmp-bins' '--enable-st-bins'

Libs:
cdada 0.5.0
libpcap version 1.9.1 (with TPACKET_V3)
rdkafka 0.11.4
jansson 2.14

Plugins:
memory
print
nfprobe
sfprobe
tee
kafka

System:
Linux 5.16.12-1.el8.x86_64 #1 SMP Mon Mar 7 15:36:33 UTC 2022 x86_64

Compiler:
gcc 8.5.0

For suggestions, critics, bugs, contact me: Paolo Lucente paolo@pmacct.net.

[paololucente]

Hi Thomas ( @ThomasJLLN ),

you are right, your understanding is total accurate about the status quo. Rationale being that probably options (be that translation of VRFs, interface names, etc.) are generally long strings and maybe more suitable to be loaded in a lookup table than being used for enriching the actual data stream. Nevertheless, there is on the radar an effort to make enrichment possible (so lookup id -> name inside pmacct) as an option.

Let me know your thoughts. The effort did not start yet and i can't for sure say when it will be delivered. We could slightly rename this Issue and mark it as Enhancement. Also to count further interest in this feature.

Paolo

[ThomasJLLN]

Hello Paolo,

Thank you for getting back to me. It's crystal clear now.
I agree with the idea of moving on an enhancement request. Would you like me to rename the issue?

[paololucente]

Thanks Thomas ( @ThomasJLLN ), i just did rename it and added the enhancement label.

[KandhlaChandi]

Hi Paolo,

As you're aware, we've been working on correlating VRF ID -> VRF Name using pretag.map.
on the pod system we're running, the key 'sample_type' and 'mpls_vpn_id_in'(the new pretag.map supported key) show WARN message as
WARN ( default/core ): [/etc/pmacct/pretag.map:1] unknown key 'sample_type'. Ignored.
WARN ( default/core ): [/etc/pmacct/pretag.map:2] unknown key 'sample_type'. Ignored.
WARN ( default/core ): [/etc/pmacct/pretag.map:7] unknown key 'mpls_vpn_id_in'. Ignored.
we're on nfacctd 1.7.10-git
Also, the jeq/label construct is not working throwing the following error message
WARN ( default/core ): [/etc/pmacct/pretag.map] Unresolved label 'enrich_flows_vpn_name'. Ignoring it.
WARN ( default/core ): [/etc/pmacct/pretag.map] maps_index: not supported for field(s) 100000. Indexing disabled.
i suspect this to be related to unknown key 'sample_type'