RPM plugin generates RPM with bad SHA256 digests, won't install on RHEL8 with FIPS enabled

[jelion]

A simple Gradle project that generates a relatively simple RPM sometimes builds RPMs with 'bad' checksums. Such RPMs will not install on a RHEL 8 system with FIPS mode enabled. I have seen this error with 9.1.1, the most recent version I have seen public documentation for. I did notice a version 11.0.0 available in some places; the error occurs with that version too.

Sometimes the RPM will install (on RHEL 8.6 with FIPS compliance checking enabled), other times the RPM will fail to install with an error similar to the following:

[cloud-user@paas-installer ~]$ sudo rpm --install ~/testpackage-1.0.0-1.noarch.rpm
        package testpackage-0:1.0.0-1.noarch does not verify: Payload SHA256 digest: BAD (Expected 747e511743005c7c0d65a279cca436f156e63585e8023cbb41d5326ae4f47604 != f6accca179a01e379434ffb1190fdca887d535d65615a45f19ebeb19778eab3a)

The RPM can be examined with the --checksig switch. A "good" RPM looks like:

[cloud-user@paas-installer foo]$ rpm --checksig -v ~/testpackage-1.0.0-1.noarch.rpm
/home/cloud-user/testpackage-1.0.0-1.noarch.rpm:
    Header SHA256 digest: OK
    Header SHA1 digest: OK
    Payload SHA256 digest: OK

A bad one looks like:

[cloud-user@paas-installer foo]$ rpm --checksig -v ~/testpackage-1.0.0-1.noarch.rpm
/home/cloud-user/testpackage-1.0.0-1.noarch.rpm:
    Header SHA256 digest: OK
    Header SHA1 digest: OK
    Payload SHA256 digest: BAD (Expected 747e511743005c7c0d65a279cca436f156e63585e8023cbb41d5326ae4f47604 != f6accca179a01e379434ffb1190fdca887d535d65615a45f19ebeb19778eab3a)
    MD5 digest: NOTFOUND

Running 'gradle build' on the attached build script, with a single trivial script in a scripts subfolder, produces an RPM that is sometimes good and sometimes bad. They tend to come in cycles; I may get 20 good RPMs in a row, and then 3 bad ones in a row. I have tried building with '--no-daemon'; deleting the output RPM, and even the entire build folder between builds - but have not been able to reliably produce a 'bad' RPM (or a 'good' one). It just seems to be that some are bad, and some are good. (Most are good; it may be telling that I have yet to notice bad RPMs in a completely-clean project initial build.

We are building with Gradle 7.6, on Windows 10 (AMD 64) and Java 8. We used the plugin for 1 1/2 years on RHEL 7 with no issues (FIPS was not enabled on those environments; in performing the --checksig test on a number of releases we found that 29 of 290 RPMs had bad signatures.) Our Linux environment is:

[root@paas-installer ~] uname -a
Linux paas-installer.jkepaas.gov 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 x86_64 x86_64 x86_64 GNU/Linux
[root@paas-installer cloud-user]# fips-mode-setup --check
FIPS mode is enabled.

(The RPM --checksig test should work whether FIPS is enabled or not.)

I have not been able to paste to this ticket, so my build file is pasted below. The test project also has a single file in scripts with trivial content.
build.gradle:

buildscript {
  repositories { maven { url 'https://plugins.gradle.org/m2' }}
  dependencies {
    classpath 'com.netflix.nebula:gradle-ospackage-plugin:9.1.1'
  }
}

apply plugin: 'nebula.ospackage'
import org.redline_rpm.header.RpmType
import org.redline_rpm.header.Architecture
import org.redline_rpm.header.Os

task rpm(type: Rpm) {
  packageName     'testpackage'
  description     'Test Package'
  version         '1.0.0'
  release         '1'

  type            RpmType.BINARY
  arch            Architecture.NOARCH
  os              Os.LINUX

  user            'root'
  permissionGroup 'root'
  dirMode         0755
  fileMode        0755

  directory('/opt/test', 0755)

  from ('scripts') {
    into '/opt/testpackage/bin'
  }
}

build.dependsOn rpm

[Hendrik-H]

I'm seeing the issue as well but so far only on s390x and not x86_64. The payload signing support was added in redline 1.2.10, which was picked up here: a942e96 So the problem should exist since 8.6.1.

[patricklucas]

I've just started to use this plugin and very quickly ran into this on an almost trivial rpm—just a single preinstall script.

[DanielThomas]

@patricklucas do you also experience this as an intermittent failure, or is it reproducible?

produces an RPM that is sometimes good and sometimes bad

We don't use rpms ourselves, so we'll find it difficult to find the time to look into this. But if someone has time to troubleshoot and fix, we'd be happy to merge/release from a PR.

[mricciuti]

Hi @DanielThomas , pinging you as there are others PRs waiting to be checked for quite some time, maybe you would be happy to check/merge those as well? ( eg.: #414 , #417 , #424 , and some others)

[patricklucas]

@DanielThomas I've only encountered it once so far, but at the same time I've only used the plugin for a few days, so statistically-speaking...

If it happens again I'll report back here.