FIPS

[kobelb]

Some users require only FIPS compliant cryptographic algorithms, so we'll have to look into running NodeJS with the --force-fips flag and see what all we have to change to make this work.

We could potentially run this for all installs of Kibana, or optionally allow it to be turned on. It all depends on the limitations when running in this mode.

The --force-fips flag doesn't work out of the box, and instead we'll have to build NodeJS with FIPS support. v8.11.1 supports it per: https://github.com/nodejs/node/blob/v8.11.1/BUILDING.md

but they removed support in master per:
https://github.com/nodejs/node/blob/master/BUILDING.md#building-nodejs-with-fips-compliant-openssl with further reasoning provided by @legrego below.

ES Issue for tracking FIPS compliance: elastic/elasticsearch#29851

[legrego]

but they removed support in master

I found meeting notes from the Node TSC where they discussed this. Master was recently updated to support OpenSSL 1.1.0, which does not yet have a FIPS module. The TSC discussed dropping FIPS support or moving to a FIPS-compliant SSL library, such as BoringSSL.

As of right now, it appears that Node v10 will support both versions of OpenSSL, at least for a certain period of time. This will allow us to build Node 10 using the OpenSSL FIPS module, as long s we target the previous OpenSSL version. All this is subject to change based on Node's and OpenSSL's roadmap and release schedule. I don't want to recap it again here, and risk being out-of-date, but there is a great writeup in this PR which details the Node TSC's OpenSSL Strategy going forward.

[legrego]

Research

Using https://github.com/legrego/docker-node-fips, I was able to get a FIPS-enabled Node.js runtime on my machine. Once that was up, I cloned the kibana repo into the container and tried running node ./scripts/kibana to see what would happen.

Reporting

Reporting failed to load. Reporting uses PUID, which uses MD5 to generate a machine uid. I opened a PR against this to change the algorithm to something that can run in a FIPS environment.

Optimizer

Webpack and the various loaders use quite a bit of hashing for file names, caching, etc. I managed to trace down and "fix" a number of the usages (see https://github.com/legrego/kibana/tree/fips-compliance for my WIP), but our version of webpack (3.6) has a hard-coded reference to MD5 that I can't find a way around. It appears that v4.6 (??) made this configurable via webpack/webpack@2acd0d4, but I'm not sure if it will fix our problem or not.

Short URL

The Saved Object ID for Short Links is generated using MD5, and should be updated to use a different algorithm. If I understand this correctly, it shouldn't be a breaking change to update this, but I'd love for someone else to help confirm this.

X-Pack Test Suite

The X-Pack test suite cannot currently be run within a fips environment, for two reasons (so far):

1. Gulp depends on v8flags , which has a hardcoded use of md5. The test runner fails immediately here. Patching this locally got my past this error, which leads to the second issue.

x-pack@7.0.0-alpha1 /Users/larry/repos/kibana/x-pack
└─┬ gulp@3.9.1
  └── v8flags@2.1.1 

2. The Optimizer step of the functional test runner fails here (even after applying the changes in the section above) because of another hardcoded MD5 reference that's used as part of the SourceMapDevToolPlugin:

Running "run:browserTestServer" (run) task
  log   [16:36:38.817] [info][optimize] Optimizing and caching bundle for tests. This may take a few minutes
crypto.js:81
  this._handle = new binding.Hash(algorithm);
                 ^

Error: error:060A80A3:digital envelope routines:FIPS_DIGESTINIT:disabled for fips
    at new Hash (crypto.js:81:18)
    at Object.createHash (crypto.js:611:10)
    at getHash (/kibana/node_modules/webpack/lib/ModuleFilenameHelpers.js:41:33)
    at Object.createFilename (/kibana/node_modules/webpack/lib/ModuleFilenameHelpers.js:63:10)
    at chunk.files.forEach.file (/kibana/node_modules/webpack/lib/SourceMapDevToolPlugin.js:103:71)
    at Array.forEach (<anonymous>)
    at /kibana/node_modules/webpack/lib/SourceMapDevToolPlugin.js:90:18
    at Array.forEach (<anonymous>)
    at Compilation.<anonymous> (/kibana/node_modules/webpack/lib/SourceMapDevToolPlugin.js:89:12)
    at Compilation.applyPlugins1 (/kibana/node_modules/tapable/lib/Tapable.js:75:14)
    at self.applyPluginsAsync.err (/kibana/node_modules/webpack/lib/Compilation.js:661:11)
    at next (/kibana/node_modules/tapable/lib/Tapable.js:202:11)
    at Compilation.compilation.plugin (/kibana/node_modules/webpack/lib/BannerPlugin.js:67:5)
    at Compilation.applyPluginsAsyncSeries (/kibana/node_modules/tapable/lib/Tapable.js:206:13)
    at self.applyPluginsAsync.err (/kibana/node_modules/webpack/lib/Compilation.js:657:10)
    at next (/kibana/node_modules/tapable/lib/Tapable.js:202:11)
    at Compilation.<anonymous> (/kibana/node_modules/extract-text-webpack-plugin/dist/index.js:275:11)
    at Compilation.applyPluginsAsyncSeries (/kibana/node_modules/tapable/lib/Tapable.js:206:13)
    at sealPart2 (/kibana/node_modules/webpack/lib/Compilation.js:653:9)
    at next (/kibana/node_modules/tapable/lib/Tapable.js:202:11)
    at /kibana/node_modules/extract-text-webpack-plugin/dist/index.js:244:13
    at /kibana/node_modules/extract-text-webpack-plugin/node_modules/async/dist/async.js:473:16

Test Results

I did manage to run the OSS test suite (yarn test) and a partial X-Pack test suite before the functional tests kicked off, and there were no surprising test failures with FIPS enabled. The only failure I had was one that explicitly tested that an MD5 hash was created for the short url.

I'm still working through this -- I'll udpate this comment as I learn more.

[epixa]

Since edits don't notify subscribers, can you drop a new comment whenever you make progress in the research? Even if that new comment is just "updated research with more progress" :)

[legrego]

"Updated research with more progress" -- Added findings for Short Links, the x-pack test suite, and general test results.

I'm going to set this aside, for now, to focus on Spaces/RBAC work, but I'll pick this up again as time/priorities permit.

[legrego]

An update on the current state of affairs:

OpenSSL's FIPS Object Module is only compatible with OpenSSL versions 1.0.1 and 1.0.2 . See [1].
The last version of Node.js to use one of these compatible versions was the 8.x series, which went EOL 2019-12-31. See [2].
The last version of Kibana to use a compatible version of Node.js (8.14.0) was the 6.5.x release. See [3]. Kibana 6.5.x will go EOL on 2020-05-14, so a little less than a month from now.

So to summarize, Kibana runs on Node.js, and (based on my research) there are no supported versions of Node.js which are able to take advantage of FIPS.

OpenSSL has plans to support FIPS again though. Their original goal was to have FIPS mode completed by the end of 2019, but that didn't happen.
They posted an update saying that they expect to have coding completed by the end of Q2 2020, with a release in Q4 of 2020. See [4].

Once OpenSSL finishes their work, then Node.js will need to release an update to support the new version of OpenSSL.
Once Node.js releases its update, then Kibana can start to take advantage of it.
Once that happens, then we can start evaluating what needs to change within Kibana.

[1] https://wiki.openssl.org/index.php/FIPS_modules
[2] https://github.com/nodejs/Release#end-of-life-releases
[3] https://github.com/elastic/kibana/blob/6.5/package.json#L391
[4] https://www.openssl.org/blog/blog/2019/11/07/3.0-update/

[bytebilly]

Related: nodejs/node#37072

[D4V3M0NK]

Could I also point out that if you're using a distribution that has a certified FIPS modules (Ubuntu, RedHat and others have gone through the considerable expense of certifying OpenSSL libraries on their platforms) then it would be reasonable to assume people use those distros (in some part) because of those certifications: when working in the public sector space, one has to use FIPS certified libraries when it comes to encryption. With all due respect to both the fine Node programmers and OpenSSL themselves, relying and on a non-profit to be able to go through the same expense in getting them certified is (IMHO) a little shortsighted, as that means that anyone wanting to use Node in a public sector setting has to revert back to OpenSSL 1.0.2g - and in doing so, one opens themselves up to the vulnerabilities of a library that aged.

Having Node be able to be built by dynamically linking to those certified libraries would mean that Node would be FIPS compliant, and so, therefore, would any application that relies on Node that sits on top of it.

[watson]

It's no longer required to build a custom version of Node.js to support FIPS: https://nodejs.org/api/crypto.html#fips-mode

I haven't spend enough time looking into this to figure out which version of Node.js this became possible as these docs changes hasn't been backported to the previous versions. But I think it might have been added in Node.js 17 (Kibana is currently running Node.js 16 with a Node.js 18 upgrade coming soon).

[watson]

Next steps as I see them:

1. Upgrade Kibana to Node.js 18
2. Decide if we want to allow enabling of FIPS mode by either a combination of an OpenSSL configuration file and a Kibana config option (e.g. xpack.security.fips: true), or by the OpenSSL configuration file alone.
   3.1. Possibly implement the Kibana config option if required.
3. Add test to verify that Kibana can be started in FIPS mode if the OpenSSL requirements are met.
4. Document steps to enable FIPS mode.

[watson]

We want to check that enabling FIPS mode in Node.js doesn't somehow mess the places we need to run using --openssl-legacy-provider. According to this comment, enabling FIPS mode will make all legacy providers unavailable: https://github.com/nodejs/node/blob/d150316a8ecad1a9c20615ae62fcaf4f8d060dcc/test/addons/openssl-providers/test-legacy-provider-option.js#L20

[jstibbards-elastic]

@watson - Any progress on this, or a schedule by which we'll know the schedule? Thanks

[legrego]

@jstibbards-elastic, @kc13greiner and I are your contacts for Kibana FIPS work at this point. Kurt, would you mind setting up some time for the three of us to chat next week to get @jstibbards-elastic caught up?