Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crate specific overrides (eg. building OpenSSL without errors out of the box) #326

Merged
merged 7 commits into from
May 30, 2024

Conversation

FireFragment
Copy link
Contributor

@FireFragment FireFragment commented May 16, 2024

This PR adds a new system of crate-specific overrides, which allows to automatically adjust build behavior to make some crates working.
This will probably be mainly adding non-rust system dependencies of crates automatically as buildInputs, so they work out-of-the-box for users.

In this PR, there is just one such override (if openssl-sys is in dependencies, automatically provide pkg-config in nativeBuildInputs and openssl in buildInputs), but more can be now added easily in future by editing crate_specific.nix.

Summary of the changes

  • Added the file crate_specific.nix for defining the crate-specific overrides easily
  • Added autoCrateSpecificOverrides option for enabling/disabling this feature
  • Added a crate with a dependency on OpenSSL to test, whether it works

@FireFragment FireFragment changed the title Crate specific overrides and building OpenSSL without errors out of the box Crate specific overrides (eg. building OpenSSL without errors out of the box) May 16, 2024
Copy link
Collaborator

@nmattia nmattia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very cool!

crate_specific.nix Show resolved Hide resolved
@@ -267,6 +267,7 @@ Note that you shouldn't call `overrideAttrs` on a derivation built by Naersk
| `postInstall` | Optional hook to run after the compilation is done; inside this script, `$out/bin` contains compiled Rust binaries. Useful if your application needs e.g. custom environment variables, in which case you can simply run `wrapProgram $out/bin/your-app-name` in here. Default: `false` |
| `usePureFromTOML` | Whether to use the `fromTOML` built-in or not. When set to `false` the python package `remarshal` is used instead (in a derivation) and the JSON output is read with `builtins.fromJSON`. This is a workaround for old versions of Nix. May be used safely from Nix 2.3 onwards where all bugs in `builtins.fromTOML` seem to have been fixed. Default: `true` |
| `mode` | What to do when building the derivation. Either `build`, `check`, `test`, `fmt` or `clippy`. <br/> When set to something other than `build`, no binaries are generated. Default: `"build"` |
| `autoCrateSpecificOverrides` | Whether to automatically apply crate-specific overrides, mainly additional `buildInputs` for dependencies. <br /> For example, if you use the `openssl` crate, `pkgs.pkg-config` and `pkgs.openssl` are automatically added as buildInputs. Default: `true` |
Copy link
Collaborator

@nmattia nmattia May 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think having it enabled by default might create issues for users that already build with openssl and already add dependencies manually. WDYT?

EDIT: to be clear I think it's a good idea to have it enabled by default, just wondering how we can avoid breaking existing setups, esp. for people with custom openssl packages

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One idea I have is to emit a warning if there is already a derivation in buildInputs with the same name. It would not work in 100% of the cases, but it should work most of the time. Or we can just disable it by default.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think having it enabled by default is much nicer!

@nmattia
Copy link
Collaborator

nmattia commented May 21, 2024

@FireFragment looks like there's a cargo issue in the test, do they run successfully for you?

       >   failed to update replaced source registry `[https://github.com/rust-lang/crates.io-index`](https://github.com/rust-lang/crates.io-index%60)
       >
       > Caused by:
       >   failed to parse manifest at `/nix/store/shh4214g8hn6jakwa8vrhbmqpx8fzj67-crates-io-dependencies/bitflags-2.5.0-cf4b9d6a944f767f8e5e0db018570623c85f3d925ac718db4e06d0187adb21c1/Cargo.toml`
       >
       > Caused by:
       >   feature `edition2021` is required
       >
       >   consider adding `cargo-features = ["edition2021"]` to the manifest

https://github.com/nix-community/naersk/actions/runs/9117568700/job/25091230219?pr=326#step:4:3064

There on nixpkgs 22 and lower, there is too old Rust to build OpenSSL
@FireFragment
Copy link
Contributor Author

FireFragment commented May 28, 2024

It looks like that in nixpkgs 22 and lower, the cargo and rust versions are too old to compile the openssl crate, so I just disabled the test on nixpkgs versions lower than 23 here.

Now it passes: https://github.com/FireFragment/naersk/actions/runs/9275182581

On unstable and 23.05 it always worked.

Copy link
Collaborator

@nmattia nmattia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a ton for doing this @FireFragment, much appreciated!

@nmattia nmattia merged commit fa19d8c into nix-community:master May 30, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Have some extra packages per crate
2 participants