Test suite fails if commit signing is enabled

[VorpalBlade]

Bug description

While trying to make a PR for release-plz I have had lots of issues running the test suite. Apparently the tests suite doesn't work if you have commit signing enabled in your ~/.gitconfig:

failures:

---- tests::non_existing_tag_is_recognized stdout ----
thread 'tests::non_existing_tag_is_recognized' panicked at crates/git_cmd/src/test_fixture.rs:19:64:
called `Result::unwrap()` on an `Err` value: error while running git in directory `"/tmp/.tmpaIjRmR"` with args `["commit", "-m", "add README"]:
- stderr: error: gpg failed to sign the data:
[GNUPG:] KEY_CONSIDERED <PRIVATE KEY ID REMOVED> 2
[GNUPG:] BEGIN_SIGNING H8
gpg: signing failed: Cannot allocate memory
[GNUPG:] FAILURE sign 16810070
gpg: signing failed: Cannot allocate memory

fatal: failed to write commit object
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- tests::existing_tag_is_recognized stdout ----
thread 'tests::existing_tag_is_recognized' panicked at crates/git_cmd/src/test_fixture.rs:19:64:
called `Result::unwrap()` on an `Err` value: error while running git in directory `"/tmp/.tmpG4RFao"` with args `["commit", "-m", "add README"]:
- stderr: error: gpg failed to sign the data:
[GNUPG:] KEY_CONSIDERED <PRIVATE KEY ID REMOVED> 2
[GNUPG:] BEGIN_SIGNING H8
gpg: signing failed: Cannot allocate memory
[GNUPG:] FAILURE sign 16810070
gpg: signing failed: Cannot allocate memory

fatal: failed to write commit object

---- tests::previous_commit_is_retrieved stdout ----
thread 'tests::previous_commit_is_retrieved' panicked at crates/git_cmd/src/test_fixture.rs:19:64:
called `Result::unwrap()` on an `Err` value: error while running git in directory `"/tmp/.tmpRJP7oO"` with args `["commit", "-m", "add README"]:
- stderr: error: gpg failed to sign the data:
[GNUPG:] KEY_CONSIDERED <PRIVATE KEY ID REMOVED> 2
[GNUPG:] BEGIN_SIGNING H8
gpg: signing failed: Cannot allocate memory
[GNUPG:] FAILURE sign 16810070
gpg: signing failed: Cannot allocate memory

fatal: failed to write commit object


failures:
    tests::existing_tag_is_recognized
    tests::non_existing_tag_is_recognized
    tests::previous_commit_is_retrieved

• Would you like to work on a fix? [y/n] No thanks, this seems annoying to fix

To Reproduce

Steps to reproduce the behavior:

1. Set up git commit signing by default:

[commit]
        gpgSign = true

2. Attempt to run test suite

Expected behavior

The test should isolate itself from the system git config. There could be all sorts of settings in there that aren't compatible with what the test suite wants. Possibly many other settings that I don't happen to have.

Screenshots

Environment

• OS: [e.g. Ubuntu 20.04]
• release-plz version: [e.g. 0.1.0]

Additional context

The test also breaks if tag signing is set up:

[tag]
   gpgSign = true

[MarcoIeni]

I have commit signing enabled, and I can't reproduce this :/
I use a mac. I'm trying to reproduce this on a linux container.

docker run rust sleep infinity

then in another terminal:

docker exec -it <container_id> bash
git config --global user.name Marco
git config --global user.email myemail@me.com
gpg --full-generate-key # generate the gpg key with default values.

Then I Tell git about your signing key

This is how the file ~/.gitconfig look after:

[user]
	name = Marco
	email = myemail@me.com
        signingkey = XXXXXXXXXX
[commit]
	gpgsign = true

Then I run the test suite:

export GPG_TTY=$(tty) # I don't know why this is required for the docker container to make gpg work.
git clone https://github.com/MarcoIeni/release-plz
cd release-plz
cargo t --no-default-features

[VorpalBlade]

Hm, I'm on my phone now, but I do remember I have a specific key ID set up to use for signing too. I use Gnome Keyring for providing the ssh/gpg agents, though I don't use Gnome otherwise. X11 session.

I'll take a look later and provide an anonymised git config for you.

[MarcoIeni]

I'm still working on the reproduction, I sent the message to save what I done.

Hopefully I can reproduce the issue in docker container 👍

[MarcoIeni]

I updated my previous message with all the steps to reproduce this in docker. The test suite passes (I added the flag --no-default-features because I don't want to install docker in docker, but tests that are failing for you such as tests::previous_commit_is_retrieved are succeeding).

So yeah, please provide instructions on how to reproduce this in docker 🙏

[VorpalBlade]

It seems the key is to set a password for the gpg key. But do notice I got a completely different error than the one above!

# Because this is what I'm familar with (arch and podman)
# I assume you can do this with arch in docker too.
podman run -ti --rm docker.io/library/archlinux /bin/bash

# The rest of the commands is in the container

pacman -Syu --noconfirm base-devel git rustup # If this asks about which alternative to install for dbus, just go with the default.
rustup install stable
gpg --full-generate-key
# Generate key with defaults. Make sure to set a password!
# Note down key ID

# Adjust with key ID from previous step
cat > ~/.gitconfig <<EOF
[user]
	name = My name
	email = user@example.com
	signingKey = AE745A0883D55A2EFEC1118FCBF98F3123A22DE6
[commit]
	gpgSign = true
EOF

git clone https://github.com/MarcoIeni/release-plz
cd release-plz
RUST_BACKTRACE=1 cargo test --no-default-features

With this I get errors like:

---- tests::inexistent_previous_commit_detected stdout ----
thread 'tests::inexistent_previous_commit_detected' panicked at crates/git_cmd/src/test_fixture.rs:19:64:
called `Result::unwrap()` on an `Err` value: error while running git in directory `"/tmp/.tmpiGnCeW"` with args `["commit", "-m", "add README"]:
- stderr: error: gpg failed to sign the data:
[GNUPG:] KEY_CONSIDERED AE745A0883D55A2EFEC1118FCBF98F3123A22DE6 2
[GNUPG:] BEGIN_SIGNING H10
[GNUPG:] PINENTRY_LAUNCHED 9861 curses 1.2.1 - xterm - - 0/0 0
gpg: signing failed: Inappropriate ioctl for device
[GNUPG:] FAILURE sign 83918950
gpg: signing failed: Inappropriate ioctl for device

fatal: failed to write commit object

Stack backtrace:
   0: anyhow::kind::Adhoc::new
             at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.80/src/kind.rs:73:36
   1: git_cmd::git_in_dir
             at ./src/lib.rs:350:13
   2: git_cmd::test_fixture::<impl git_cmd::Repo>::init
             at ./src/test_fixture.rs:19:9
   3: git_cmd::tests::inexistent_previous_commit_detected
             at ./src/lib.rs:377:20
   4: git_cmd::tests::inexistent_previous_commit_detected::{{closure}}
             at ./src/lib.rs:375:45
   5: core::ops::function::FnOnce::call_once
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:250:5
   6: core::ops::function::FnOnce::call_once
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:250:5
   7: test::__rust_begin_short_backtrace
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/test/src/lib.rs:627:18
   8: test::run_test_in_process::{{closure}}
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/test/src/lib.rs:650:60
   9: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/panic/unwind_safe.rs:272:9
  10: std::panicking::try::do_call
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40
  11: std::panicking::try
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19
  12: std::panic::catch_unwind
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14
  13: test::run_test_in_process
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/test/src/lib.rs:650:27
  14: test::run_test::{{closure}}
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/test/src/lib.rs:573:43
  15: test::run_test::{{closure}}
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/test/src/lib.rs:601:41
  16: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys_common/backtrace.rs:155:18
  17: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/thread/mod.rs:529:17
  18: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/panic/unwind_safe.rs:272:9
  19: std::panicking::try::do_call
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40
  20: std::panicking::try
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19
  21: std::panic::catch_unwind
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14
  22: std::thread::Builder::spawn_unchecked_::{{closure}}
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/thread/mod.rs:528:30
  23: core::ops::function::FnOnce::call_once{{vtable.shim}}
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:250:5
  24: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/alloc/src/boxed.rs:2015:9
  25: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/alloc/src/boxed.rs:2015:9
  26: std::sys::unix::thread::Thread::new::thread_start
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys/unix/thread.rs:108:17
  27: <unknown>
  28: <unknown>
stack backtrace:
   0: rust_begin_unwind
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:645:5
   1: core::panicking::panic_fmt
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/panicking.rs:72:14
   2: core::result::unwrap_failed
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/result.rs:1649:5
   3: core::result::Result<T,E>::unwrap
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/result.rs:1073:23
   4: git_cmd::test_fixture::<impl git_cmd::Repo>::init
             at ./src/test_fixture.rs:19:9
   5: git_cmd::tests::inexistent_previous_commit_detected
             at ./src/lib.rs:377:20
   6: git_cmd::tests::inexistent_previous_commit_detected::{{closure}}
             at ./src/lib.rs:375:45
   7: core::ops::function::FnOnce::call_once
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:250:5
   8: core::ops::function::FnOnce::call_once
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

All of these fail in that way:

test tests::inexistent_previous_commit_detected ... FAILED
test tests::existing_tag_is_recognized ... FAILED
test tests::previous_commit_is_retrieved ... FAILED
test tests::current_commit_is_retrieved ... FAILED
test tests::non_existing_tag_is_recognized ... FAILED
test tests::dirty_project_is_recognized ... FAILED
test tests::clean_project_is_recognized ... FAILED

The exact error message here differs from before gpg: signing failed: Inappropriate ioctl for device instead of gpg: signing failed: Cannot allocate memory.

I will continue to test with trying to set up gnome keyring in docker (somehow!). I suspect this is related to whatever gpg agent/password entry mechanism you use.

Update:

• With the default gpg-agent, just setting export GPG_TTY=$(tty) before running the tests fixes the issue (tests pass, though they end up asking for the password once).
• Still looking into the gnome-keyring bit...

[VorpalBlade]

Unfortunately there seem to be a bunch of roadblocks to running the gnome-keyring part in a docker container. Such as getting a full systemd/dbus inside the container.

Here is a collection of errors I have come across:

secret-tool: Cannot spawn a message bus without a machine-id: Unable to load /var/lib/dbus/machine-id or /etc/machine-id: Failed to open file “/var/lib/dbus/machine-id”: No such file or directory

secret-tool: Cannot autolaunch D-Bus without X11 $DISPLAY

etc

This may need a full VM to exactly reproduce. :(

HOWEVER: I do believe that my above reproducer when GPG_TTY is unset demonstrate that the current behaviour loads the user's .gitconfig (instead of being isolated from the host system. This at the very least makes the test more fragile.

[MarcoIeni]

Yeah, I see.
If anybody wants to edit the tests to use an isolated git config, pr welcome!