Environment variable for Cargo Workspace #3946

Ygg01 · 2017-04-24T09:45:58Z

T-cargo notes:

A CARGO_RUSTC_CURRENT_DIR is added as a nightly only environment variable. See #3946 (comment). Seek for feedback.

Hi, while working on using workspace in html5ever, I've ran into issue of needing the CARGO_WORKSPACE directory, and being unable, to find it. What I resorted to is essentially, &Path(cargo_manifest).join("..") which feels hacky.

Could CARGO_WORKSPACE be added as environment variable? I'm not sure what it should be when there is no workspace defined, I assume it should either return Err or default it to CARGO_MANIFEST_DIR.

Sidenote I'm willing to work on this issue, if I could get quick pointers, to what I need to do.

The text was updated successfully, but these errors were encountered:

shepmaster · 2017-04-24T13:56:17Z

when there is no workspace defined

I'd expect the environment variable to not be set at all in that case.

Ygg01 · 2017-04-24T14:23:24Z

What would adding the CARGO_WORKSPACE entail. From cursory look, I see custom_build.rs has reference to Context, that has reference to Workspace, but same doesn't exist for compilation.rs.

Would having CARGO_WORKSPACE only for custom builds be ok?

alexcrichton · 2017-04-24T21:22:15Z

Thanks for the report! I think I may not quite be following what's going on here though? Do you mean accessing the workspace directory from a build script perhaps?

Ygg01 · 2017-04-24T21:31:48Z

@alexcrichton Yes. I was looking for workspace directory in my custom build script. It's related to servo/html5ever#261.

There is a simple workaround of taking CARGO_MANIFEST_DIR and using its parent, but I thought having CARGO_WORKSPACE would be cleaner solution.

alexcrichton · 2017-04-26T15:56:59Z

Oh yeah definitely makes sense to me! Seems reasonable to basically enhance this section

Ygg01 · 2017-05-04T10:46:20Z

So if I understand correctly, if I expose workspace.root_manifest that would be workspace dir of all projects in workspace, correct? Then I can just:

if let Some(workspace_dir) = cx.ws.root_manifest() { 
    cmd.env("CARGO_WORKSPACE_DIR", workspace_dir);
}

alexcrichton · 2017-05-04T14:43:35Z

sounds about right! I think you may not want precisely the root_manifest field but rather ws.root().join("Cargo.toml")

Ygg01 · 2017-05-08T14:30:45Z

@alexcrichton I am total newb, but won't adding Cargo.toml to workspace root, return the location of workspace manifest as opposed to directory of the directory in which workspace manifest is?

alexcrichton · 2017-05-08T16:35:35Z

oh sure yeah, it just depends on the intent of what's being conveyed (the workspace manifest or the directory of the workspace), I'm fine with either.

Ygg01 · 2017-05-08T22:55:09Z

Hm, while writing tests, I've noticed a peculiarity. I assume I'm using this wrong. But I wanted to double check

let p = project("foo")
    .file("Cargo.toml", r#"
        [project]
        name = "foo"
        version = "0.5.0"
        authors = []


        [workspace]
        members = ["a"]
    "#)
    .file("src/lib.rs", "")
    .file("build.rs", r#"
        fn main() {
            //panic!("WILL FAIL");
        }
    "#)
    .file("a/Cargo.toml", r#"
        [project]
        name = "a"
        version = "0.5.0"
        authors = []
        links = "foo"
        build = "build.rs"
    "#)
    .file("a/src/lib.rs", "")
    .file("a/build.rs", r#"
        fn main() {
                panic!("PASSES?");
        }
    "#);
assert_that(p.cargo_process("build").arg("-v"),
                execs().with_status(0));

The panic in a/build.rs never happens, and panic in build.rs always happens, even if workspace doesn't have a build= "build.rs" line.

Idea behind tests was to verify that each member build.rs has appropriate value for environment variables.

alexcrichton · 2017-05-09T15:03:44Z

@Ygg01 oh cargo build on a workspace doesn't execute all build scripts, and build.rs is inferred to be a build script if otherwise not specified.

Ygg01 · 2017-05-09T15:28:09Z

@alexcrichton Is there an alternative way to test env. variables are properly set in each member build script?

alexcrichton · 2017-05-09T18:50:22Z

@Ygg01 I think you'd just cargo build in both directories, right? And then have an assert in the build script the env var is correct?

Ygg01 · 2017-05-10T10:42:03Z

Yes, I think that is correct (one call to cargo build in member directory and one call in workspace directory). Are there any examples of such code? I can only see adding files to ProjectBuilder file.

alexcrichton · 2017-05-10T18:25:57Z

Oh you'll just want to call p.cargo multiple times basically, there's some other tests I believe which run cargo more than once.

withoutboats · 2018-01-23T00:10:12Z

This was assigned to me to summarize why we didn't merge my PR which would have closed it #4787.

We decided to punt on this feature because of the question about what happens when building a crate downloaded from crates.io, which is no longer in a workspace in that form, but might have been originally produced in a workspace and have a build script that expects to have this env var set.

Its also unclear what the motivation for this variable is; my motivation was to find the lockfile, but I concluded that the best way to get the information I was getting from the lockfile was to run cargo metadata rather than actually read from Cargo.lock.

ghost · 2018-01-23T07:31:17Z

Its also unclear what the motivation for this variable is;

One motivation would be to find the absolute path of the resulting binary executable. How I'm currently doing it.
EDIT: added the above.

run cargo metadata rather than

hey that's pretty cool:

$ pwd
/home/xftroxgpx/build/2nonpkgs/rust.stuff/rustlearnage/recompile_self
$ time cargo metadata --format-version 1 | json_reformat | grep workspace_root
    "workspace_root": "/home/xftroxgpx/build/2nonpkgs/rust.stuff/rustlearnage"

real	0m1.054s
user	0m0.847s
sys	0m0.214s

Ygg01 · 2018-01-23T12:27:40Z

@withoutboats my original motivation for this feature was when html5ever, was moving from one project per workspace to multiple. Namely some tests that were specific, became shared and not in the same directory they were left.

However fact that almost no one needed this feature, and it was easily implementable by other means, kinda made me think it's not needed.

I did forgot about it completely.

peterhuene · 2018-08-12T18:42:58Z

I also have another use case for this feature: I'm trying to get the absolute path to the source file being compiled. I embed this as metadata from a procedural macro invocation so that source can be copied during a subsequent cargo run into a particular directory structure to integrate it into a third party, language-agnostic user interface.

When using a workspace, the file!() macro expands to be workspace directory relative (I was expecting it to be CARGO_MANIFEST_DIR relative, though, to be honest). Without a way to get the workspace directory, I have to rely on a hacky method of walking up the file!() path until I hit root, popping sub-directories off of CARGO_MANIFEST_DIR as I go.

Of course, there could easily be a better way to get the source file's absolute path that I'm completely ignorant of, so please let me know if that's the case. Thanks!

mitsuhiko · 2019-01-15T09:33:26Z

I also ran into this problem where file! returns a workspace relative path but I cannot absolutize it which I need for some test related situations.

mitsuhiko · 2019-01-15T10:07:09Z

This is the workaround i have in place now which is pretty ugly: https://github.com/mitsuhiko/insta/blob/b113499249584cb650150d2d01ed96ee66db6b30/src/runtime.rs#L67-L88

This is an alternative to rust-lang#12158's `CARGO_WORKSPACE_DIR` that was implementing the solution to rust-lang#3946 that previously discussed in the cargo team meeting. `CARGO_WORKSPACE_DIR` is a bit awkward to document / describe because its the effective workspace directory of the thing being built. If the thing being built doesn't have a workspace, it falls back to `CARGO_MANIFEST_DIR`. It would also be hard to take into account what the `CARGO_WORKSPACE_DIR` would be for path dependencies into foreign workspaces *and* it wouldn't solve the problem the user is having. What the user really wants is the CWD of rustc when it is invoked. This is much simpler to describe and is accurate when using a path dependency to a foreign package. Because the CWD is a much simpler mechanism to talk about, I figured we could diverge from our prior consensus and make it always present, rather than limiting it to tests. Remaining work for rust-lang#3946: get this stabilized

epage · 2023-11-21T03:23:51Z

Looking back over use cases. Unfortunately, there aren't actually that much in the way of details

build.rs wanting to access
- html and xml named entity tables should be shared servo/html5ever#261
- Environment variable for Cargo Workspace #3946 (comment)
- Cargo.lock (from Add CARGO_WORKSPACE_DIR env variable. #4787)
- Rejection reason: Environment variable for Cargo Workspace #3946 (comment)
- Workaround: cargo metadata
Find a binary (Environment variable for Cargo Workspace #3946 (comment))
- Seems like this is indirectly seeking Need a reliable way to get the target dir from the build script #9661
Resolving file!(), particularly for snapshot testing

When the cargo team discussed this (#3946 (comment)) it was mostly focused on file!().

This is an alternative to rust-lang#12158's `CARGO_WORKSPACE_DIR` that was implementing the solution to rust-lang#3946 that previously discussed in the cargo team meeting. `CARGO_WORKSPACE_DIR` is a bit awkward to document / describe because its the effective workspace directory of the thing being built. If the thing being built doesn't have a workspace, it falls back to `CARGO_MANIFEST_DIR`. It would also be hard to take into account what the `CARGO_WORKSPACE_DIR` would be for path dependencies into foreign workspaces *and* it wouldn't solve the problem the user is having. What the user really wants is the CWD of rustc when it is invoked. This is much simpler to describe and is accurate when using a path dependency to a foreign package. Because the CWD is a much simpler mechanism to talk about, I figured we could diverge from our prior consensus and make it always present, rather than limiting it to tests. Remaining work for rust-lang#3946: get this stabilized

feat: Add `CARGO_RUSTC_CURRENT_DIR` (unstable) ### What does this PR try to resolve? This is an alternative to #12158's `CARGO_WORKSPACE_DIR` that was implementing the solution to #3946 that previously discussed in the cargo team meeting. `CARGO_WORKSPACE_DIR` is a bit awkward to document / describe because its the effective workspace directory of the thing being built. If the thing being built doesn't have a workspace, it falls back to `CARGO_MANIFEST_DIR`. It would also be hard to take into account what the `CARGO_WORKSPACE_DIR` would be for path dependencies into foreign workspaces *and* it wouldn't solve the problem the user is having. What the user really wants is the CWD of rustc when it is invoked. This is much simpler to describe and is accurate when using a path dependency to a foreign package. Because the CWD is a much simpler mechanism to talk about, I figured we could diverge from our prior consensus and make it always present, rather than limiting it to tests. ### How should we test and review this PR? The preparatory refactor commits have explanation for why they were to help ### Additional information Remaining work for #3946: get this stabilized

epage · 2023-11-29T18:55:24Z

#12996 provided an alternative to CARGO_WORKSPACE_DIR called CARGO_RUSTC_CURRENT_DIR which, in theory, should be exactly what file! is directly relative to in all cases, rather than indirectly.

rust-lang/rust was updated with this change in rust-lang/rust#118275 (24 Nov 2023) so within a day or two after that, it should be in a nightly.

Would some of the snapshot testing people be interested in trying this out on nightly and providing feedback to help towards stabilization?

For other uses, we are continuing to punt, see #3946 (comment)

lu-zero · 2024-02-11T13:45:56Z

Another use-case for it is with tools like https://github.com/jamesmunns/toml-cfg since you would like to find the cfg file at the root of the workspace and there is the additional annoyance of having to find the information from a proc_macro.

epage · 2024-02-12T16:48:44Z

@lu-zero so it sounds like that is a case of wanting access to the end-users workspace and not to your own package's workspace.

I think we had expressed concern in the past about being able to change your build with side band information like that. With https://internals.rust-lang.org/t/pre-rfc-mutually-excusive-global-features/19618, I had considered the idea of allowing something like that though.

lu-zero · 2024-02-12T19:20:13Z

For my specific needs, putting a cfg.toml in the root of the workspace is already good. It makes easier to share the same configuration among multiple example-crates that might be radically different (no_std vs std-ish esp targets)

This provides what cargo sets as the `current_dir` for the `rustc` process. While `std::file!` is unspecified in what it is relative to, it is relatively safe, it is generally relative to `rustc`s `current_dir`. This can be useful for snapshot testing. For example, `snapbox` has been using this macro on nightly since assert-rs/snapbox#247, falling back to finding a parent of `CARGO_MANIFEST_DIR`, if present. This has been in use in Cargo since rust-lang#13441. This was added in rust-lang#12996. Relevant points discussed in that issue: - This diverged from the original proposal from the Cargo team of having a `CARGO_WORKSPACE_DIR` that is the "workspace" of the package being built (ie registry packages would map to `CARGO_MANIFEST_DIR`). In looking at the `std::file!` use case, `CARGO_MANIFEST_DIR`, no matter how we defined it, would only sort of work because no sane definition of that maps to `rustc`'s `current_dir`.a This instead focuses on the mechanism currently being used. - Using "current dir" in the name is meant to be consistent with `std::env::current_dir`. - I can go either way on `CARGO_RUSTC` vs `RUSTC`. Existing related variables: - `RUSTC` - `RUSTC_WRAPPER` - `RUSTC_WORKSPACE_WRAPPER` - `RUSTFLAGS` (no `C`) - `CARGO_CACHE_RUSTC_INFO` Note that rust-lang#3946 was overly broad and covered many use cases. One of those was for packages to look up information on their dependents. Issue rust-lang#13484 is being left open to track that. Fixes rust-lang#3946

epage · 2024-03-26T20:43:12Z

For the snapshot testing side of this (ie using std::file!), I've posted #13644 for stabilizing CARGO_RUSTC_CURRENT_DIR.

This provides what cargo sets as the `current_dir` for the `rustc` process. While `std::file!` is unspecified in what it is relative to, it is relatively safe, it is generally relative to `rustc`s `current_dir`. This can be useful for snapshot testing. For example, `snapbox` has been using this macro on nightly since assert-rs/snapbox#247, falling back to finding a parent of `CARGO_MANIFEST_DIR`, if present. This has been in use in Cargo since rust-lang#13441. This was added in rust-lang#12996. Relevant points discussed in that issue: - This diverged from the original proposal from the Cargo team of having a `CARGO_WORKSPACE_DIR` that is the "workspace" of the package being built (ie registry packages would map to `CARGO_MANIFEST_DIR`). In looking at the `std::file!` use case, `CARGO_MANIFEST_DIR`, no matter how we defined it, would only sort of work because no sane definition of that maps to `rustc`'s `current_dir`.a This instead focuses on the mechanism currently being used. - Using "current dir" in the name is meant to be consistent with `std::env::current_dir`. - I can go either way on `CARGO_RUSTC` vs `RUSTC`. Existing related variables: - `RUSTC` - `RUSTC_WRAPPER` - `RUSTC_WORKSPACE_WRAPPER` - `RUSTFLAGS` (no `C`) - `CARGO_CACHE_RUSTC_INFO` Note that rust-lang#3946 was overly broad and covered many use cases. One of those was for packages to look up information on their dependents. Issue rust-lang#13484 is being left open to track that. Fixes rust-lang#3946

Ygg01 mentioned this issue May 19, 2017

Adding Workspace DIR functionality #4073

Closed

carols10cents added A-environment-variables Area: environment variables A-workspaces Area: workspaces C-feature-request Category: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted` labels Oct 2, 2017

alexcrichton mentioned this issue Dec 7, 2017

Add CARGO_WORKSPACE_DIR env variable. #4787

Closed

aturon assigned withoutboats Jan 22, 2018

Ygg01 closed this as completed Jan 23, 2018

Ygg01 reopened this Jan 23, 2018

alexcrichton mentioned this issue Jul 15, 2019

add env variable to retrieve workspace root path #7089

Closed

Arc-blroth mentioned this issue Oct 4, 2023

Fix --target getting passed twice to the Android NDK clang on Windows rust-lang/cc-rs#705

Merged

epage mentioned this issue Nov 17, 2023

feat: Add CARGO_RUSTC_CURRENT_DIR (unstable) #12996

Merged

Teebor-Choka mentioned this issue Dec 19, 2023

Finish the migration of the HOPR node to Rust hoprnet/hoprnet#5814

Merged

31 tasks

weihanglo mentioned this issue Jan 26, 2024

Option to cargo run to make it set the project dir as the current working directory #13348

Open

epage mentioned this issue Feb 14, 2024

feat(snap)!: expect_file! macro instead of _path-variants of asserts assert-rs/snapbox#247

Merged

weihanglo mentioned this issue Feb 23, 2024

Providing path to originally selected manifest file #13484

Open

epage mentioned this issue Mar 1, 2024

Need a reliable way to get the target dir from the build script #9661

Open

epage mentioned this issue Mar 25, 2024

Add env vars specifying the path for the dependent crate when compiling a dependency #13636

Closed

epage linked a pull request Mar 26, 2024 that will close this issue

feat: Stablize CARGO_RUSTC_CURRENT_DIR #13644

Open

Umio-Yasuno mentioned this issue Apr 17, 2024

Cannot find workspace version in crate_package taiki-e/find-crate#23

Open

SK83RJOSH mentioned this issue May 2, 2024

embedded_watcher does not work in multi-crate workspaces bevyengine/bevy#10575

Open

epage added S-needs-team-input Status: Needs input from team on whether/how to proceed. and removed S-accepted Status: Issue or feature is accepted, and has a team member available to help mentor or review E-medium Experience: Medium labels May 15, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Environment variable for Cargo Workspace #3946

Environment variable for Cargo Workspace #3946

Ygg01 commented Apr 24, 2017 •

edited by weihanglo

shepmaster commented Apr 24, 2017

Ygg01 commented Apr 24, 2017

alexcrichton commented Apr 24, 2017

Ygg01 commented Apr 24, 2017

alexcrichton commented Apr 26, 2017

Ygg01 commented May 4, 2017 •

edited

alexcrichton commented May 4, 2017

Ygg01 commented May 8, 2017

alexcrichton commented May 8, 2017

Ygg01 commented May 8, 2017 •

edited

alexcrichton commented May 9, 2017

Ygg01 commented May 9, 2017

alexcrichton commented May 9, 2017

Ygg01 commented May 10, 2017

alexcrichton commented May 10, 2017

withoutboats commented Jan 23, 2018

ghost commented Jan 23, 2018 •

edited by ghost

Ygg01 commented Jan 23, 2018 •

edited

peterhuene commented Aug 12, 2018 •

edited

mitsuhiko commented Jan 15, 2019

mitsuhiko commented Jan 15, 2019

epage commented Nov 21, 2023

epage commented Nov 29, 2023 •

edited

lu-zero commented Feb 11, 2024

epage commented Feb 12, 2024

lu-zero commented Feb 12, 2024

epage commented Mar 26, 2024

Environment variable for Cargo Workspace #3946

Environment variable for Cargo Workspace #3946

Comments

Ygg01 commented Apr 24, 2017 • edited by weihanglo

shepmaster commented Apr 24, 2017

Ygg01 commented Apr 24, 2017

alexcrichton commented Apr 24, 2017

Ygg01 commented Apr 24, 2017

alexcrichton commented Apr 26, 2017

Ygg01 commented May 4, 2017 • edited

alexcrichton commented May 4, 2017

Ygg01 commented May 8, 2017

alexcrichton commented May 8, 2017

Ygg01 commented May 8, 2017 • edited

alexcrichton commented May 9, 2017

Ygg01 commented May 9, 2017

alexcrichton commented May 9, 2017

Ygg01 commented May 10, 2017

alexcrichton commented May 10, 2017

withoutboats commented Jan 23, 2018

ghost commented Jan 23, 2018 • edited by ghost

Ygg01 commented Jan 23, 2018 • edited

peterhuene commented Aug 12, 2018 • edited

mitsuhiko commented Jan 15, 2019

mitsuhiko commented Jan 15, 2019

epage commented Nov 21, 2023

epage commented Nov 29, 2023 • edited

lu-zero commented Feb 11, 2024

epage commented Feb 12, 2024

lu-zero commented Feb 12, 2024

epage commented Mar 26, 2024

Ygg01 commented Apr 24, 2017 •

edited by weihanglo

Ygg01 commented May 4, 2017 •

edited

Ygg01 commented May 8, 2017 •

edited

ghost commented Jan 23, 2018 •

edited by ghost

Ygg01 commented Jan 23, 2018 •

edited

peterhuene commented Aug 12, 2018 •

edited

epage commented Nov 29, 2023 •

edited