Uninstrumented functions in Nitro NITF parser

[kaoudis]

When I parse a nitf file with the Polytracker instrumented version of Nitro, DFSan notes uninstrumented functions.

How to reproduce

1. Build the Dockerfile-nitro-nitf.demo and attach to the Docker container
2. create a shallow copy of the test files directory from the FAW repo:

mkdir FAW && cd FAW
git init && git remote add origin https://github.com/GaloisInc/FAW.git
git config core.sparseCheckout true
echo "/test_files/nitf" >> .git/info/sparse-checkout
git pull origin master

3. Run the instrumented nitro_track binary (the uninstrumented version is show_nitf++!) on each file. Example:

/polytracker/the_klondike/nitro/build/nitro_track FAW/test_files/nitf/i_3034c.ntf

This will produce some Polytracker related DFSan and taint sink output in stderr around the NITF file related output the show_nitf++ uninstrumented binary produces in stdout, and should also result in a Polytracker output file in the working directory (i.e. POLYDB: polytracker.tdag).

DFSan output lines are generally like this, though DFSan mentions a variety of uninstrumented functions:

==18488==WARNING: DataFlowSanitizer: call to uninstrumented function newlocale

For a parsed nitf, get just the names of uninstrumented functions:

/polytracker/the_klondike/nitro/build/nitro_track FAW/test_files/nitf/i_3052a.ntf 2>&1 > /dev/null | awk '{print $NF}' | sort -u

Get count per fn:

/polytracker/the_klondike/nitro/build/nitro_track FAW/test_files/nitf/i_3052a.ntf 2>&1 > /dev/null | awk '{print $NF}' | grep wcrtomb | wc -l

List of functions by FAW sample NITF (counts):

i_3034c.ntf

• __ctype_get_mb_cur_max (1)
• iswspace (8)
• lseek64 (395)
• newlocale (1)
• vsnprintf (14)
• wcrtomb (2)

i_3052a.ntf

• __ctype_get_mb_cur_max (1)
• lseek64 (297)
• newlocale (1)
• vsnprintf (9)
• wcrtomb (2)

i_3063f.ntf

• __ctype_get_mb_cur_max (1)
• lseek64 (297)
• newlocale (1)
• vsnprintf (9)
• wcrtomb (2)

i_3301k.ntf

• __ctype_get_mb_cur_max (1)
• iswspace (11)
• lseek64 (427)
• newlocale (1)
• vsnprintf (22)
• wcrtomb (2)

ns3033b.nsf

• __ctype_get_mb_cur_max (1)
• iswspace (10)
• lseek64 (427)
• newlocale (1)
• vsnprintf (18)
• wcrtomb (2)

ns3034d.nsf

• __ctype_get_mb_cur_max (1)
• iswspace (12)
• lseek64 (387)
• newlocale (1)
• vsnprintf (18)
• wcrtomb (2)

ns3114i.nsf

• __ctype_get_mb_cur_max (1)
• lseek64 (273)
• newlocale (1)
• vsnprintf (9)
• wcrtomb (2)

ns5600a.nsf

• __ctype_get_mb_cur_max (1)
• iswspace (11)
• lseek64 (991)
• newlocale (1)
• vsnprintf (16)
• wcrtomb (2)

[kaoudis]

The joesweeney/nitf-demo branch Dockerfile Dockerfile-daedalus-nitf.demo produces a NITF parsing version of DaeDaLus which also sees uninstrumented fns. I'd like to record them here by FAW sample file and count, so that I can use this data later potentially if I (or anyone else!) might decide to instrument more functions or update the Polytracker ABI list to mark any of these as sufficient without a custom implementation.

==Also worth noting: DaeDaLus doesn't appear to parse .nsf files (it errors out on them), so the uninstrumented functions called on those inputs are likely from setup routines which occur before erroring out on the input type or something about the format.==

List of functions by FAW sample NITF (counts):

i_3034c.ntf

• __ctype_get_mb_cur_max (1)
• __fxstat (1)
• newlocale (1)
• vsnprintf (1)
• wcrtomb (2)

i_3052a.ntf

• __ctype_get_mb_cur_max (1)
  • __fxstat (1)
• newlocale (1)
• vsnprintf (1223)
• wcrtomb (2)

i_3063f.ntf

• __ctype_get_mb_cur_max (1)
• __fxstat (1)
• newlocale (1)
• vsnprintf (1299)
• wcrtomb (2)

i_3301k.ntf

• __ctype_get_mb_cur_max (1)
• __fxstat (1)
• newlocale (1)
• vsnprintf (477)
• wcrtomb (2)

ns3033b.nsf

• __ctype_get_mb_cur_max (1)
• __fxstat (1)
• newlocale (1)
• vsnprintf (1)
• wcrtomb (2)

ns3034d.nsf

• __ctype_get_mb_cur_max (1)
• __fxstat (1)
• newlocale (1)
• vsnprintf (1)
• wcrtomb (2)

ns3114i.nsf

• __ctype_get_mb_cur_max (1)
• __fxstat (1)
• newlocale (1)
• vsnprintf (1)
• wcrtomb (2)

ns5600a.nsf

• __ctype_get_mb_cur_max (1)
• __fxstat (1)
• newlocale (1)
• vsnprintf (1)
• wcrtomb (2)