Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use RUSTC_WRAPPER if no other wrapper is provided #918

Merged
merged 2 commits into from
Jan 26, 2024

Conversation

LeonMatthesKDAB
Copy link
Contributor

Previously, the rustc_wrapper_fallback was only called if the tool
was not a full path.
With this patch, the RUSTC_WRAPPER is always used, even if the tool provided is an exact path.
Providing a wrapper manually is still possible and overrides the RUSTC_WRAPPER.

If the path to the tool includes spaces it is otherwise impossible to provide a compiler cache like sccache through the CXX or CC environment variables, as the arguments will be split up incorrectly.

See also: corrosion-rs/corrosion#474

Previously, the `rustc_wrapper_fallback` was only called if the tool
 was **not** a full path.
With this patch, the `RUSTC_WRAPPER` is always used, even if the tool
provided is an exact path.
Providing a wrapper manually is still possible and overrides the
`RUSTC_WRAPPER`.

If the path to the tool includes spaces it is otherwise impossible to
provide a compiler cache like sccache through the CXX or CC environment
variables, as the arguments will be split up incorrectly.

See also: corrosion-rs/corrosion#474
@NobodyXu
Copy link
Collaborator

I'd recommend you to add #[allow(dead_code)] for line 2385

This should suppress new warnings generated by the nightly toolchain.
@LeonMatthesKDAB
Copy link
Contributor Author

I'd recommend you to add #[allow(dead_code)] for line 2385

Hi, thank you. That has fixed the CI issues.
Unsure where that issue was coming from in the first place, as I haven't touched that code at all...

Probably just an update in nightly.

@NobodyXu
Copy link
Collaborator

If the path to the tool includes spaces it is otherwise impossible to provide a compiler cache like sccache through the CXX or CC environment variables, as the arguments will be split up incorrectly.

P.S. Looking at the source code, seems like cc actually supports splitting env CC='sccache cc'.

This is just a rant from me, doesn't affect merging of this PR.

Copy link
Collaborator

@NobodyXu NobodyXu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great if you can setup a new CI that tests using RUSTC_WRAPPER for tools with relative and absolute pathas you describe, honestly a lot of stuff in cc are untested, so I'd prefer to use this opportunity to add some test for this.

@LeonMatthesKDAB
Copy link
Contributor Author

LeonMatthesKDAB commented Jan 26, 2024

If the path to the tool includes spaces it is otherwise impossible to provide a compiler cache like sccache through the CXX or CC environment variables, as the arguments will be split up incorrectly.

P.S. Looking at the source code, seems like cc actually supports splitting env CC='sccache cc'.

This is just a rant from me, doesn't affect merging of this PR.

Yes, it does, but only if the path to cc doesn't include spaces.

On windows, our CI finds the path to cc is something like:
C:\Program Files\...\cc
If this is encoded in CC, what cc-rs ends up generating is:
sccache "C:\Program" "Files\...\cc".
Which then means sccache thinks C:\Program is the compiler, which of course doesn't make sense.

For this to work correctly, we'd need #847 .
But the sentiment seems to be that it's not an ideal solution.
The patch here can work with the current behavior, as spaces are supported if CC is a direct file path.
Just the RUSTC_WRAPPER wasn't supported in that case, which would still be useful to support.
It can speed up our CI builds by more than half!

You can follow the corrosion in the corrision issue I mentioned, which hits the problem.

@NobodyXu
Copy link
Collaborator

Thanks I see, this is something I didn't realize could happen in production where path includes spaces.

@NobodyXu
Copy link
Collaborator

NobodyXu commented Jan 26, 2024

It would be great if you could verify it works, then I would have the confidence of approving and merging this PR.

Edit:

Sorry if I sound rude, I just want some confidence it would work as intended (cc has a history of merging PRs later found out to not work as intended and yanked the release, which is why v0.1.84 is yanked).

It would be great if you could try this in your project and veirfy that it work as intended.

@LeonMatthesKDAB
Copy link
Contributor Author

Hi, of course.
We have tried this here: KDAB/cxx-qt#820 (with a backported version of this patch in cc-rs 1.0.79)
And it seems to work.

For some reason our Windows CI doesn't gain much speed when running under ctest, which we haven't fully investigated.
But e.g. our macOS CI has speed up from ~12 min. to ~6 min 🥳

@Be-ing
Copy link
Contributor

Be-ing commented Jan 26, 2024

It would be great if you could verify it works, then I would have the confidence of approving and merging this PR.

This behavior should also be documented, though considering that none of the environment variable usage is documented, I don't think that should hold up this PR. I opened a new issue for that: #920.

Copy link
Collaborator

@NobodyXu NobodyXu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@NobodyXu NobodyXu merged commit 4d36cfa into rust-lang:main Jan 26, 2024
18 checks passed
Be-ing added a commit to Be-ing/cxx-qt that referenced this pull request Feb 13, 2024
1.0.85 includes rust-lang/cc-rs#918
and the macOS deployment target issue has been fixed.
LeonMatthesKDAB pushed a commit to Be-ing/cxx-qt that referenced this pull request Feb 14, 2024
1.0.85 includes rust-lang/cc-rs#918
and the macOS deployment target issue has been fixed.
Be-ing added a commit to Be-ing/cxx-qt that referenced this pull request Feb 21, 2024
1.0.86 includes rust-lang/cc-rs#918
and the macOS deployment target issue has been fixed.
ahayzen-kdab pushed a commit to Be-ing/cxx-qt that referenced this pull request Feb 21, 2024
1.0.86 includes rust-lang/cc-rs#918
and the macOS deployment target issue has been fixed.
ahayzen-kdab pushed a commit to KDAB/cxx-qt that referenced this pull request Feb 21, 2024
1.0.86 includes rust-lang/cc-rs#918
and the macOS deployment target issue has been fixed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants