Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: always include files from the same workspace as the build target in copy_to_directory() #360

Merged
merged 3 commits into from Oct 10, 2023

Conversation

dgp1130
Copy link
Contributor

@dgp1130 dgp1130 commented Feb 11, 2023

Fixes #359.

This updates the copy_to_directory tool to accept the workspace name of the target it is executing under and automatically include all source files under that workspace (after apply other filters) regardless of include_external_repositories.

I believe all the existing tests should pass, as they do when I manually update copy_to_directory() to use the version of the tool built from HEAD. However they don't do that by default, and adding this workspace name option is a breaking change to the existing contract of the copy_to_directory tool. It needs to be released at the same time as this fix, unfortunately I'm not sure how to do that or if I even can. @gregmagolan, any suggestions on how to move forward here?

@gregmagolan
Copy link
Member

Thanks for the PR @dgp1130 .
The sequencing here is tricky with the pre-built golang binary. The golang changes need to land first and then a release cut for those to be released and then the starlark impl updated to use the new version of the golang binary.

I can help with the sequencing after I find a bit of time to review this code. We've been really busy with Workflows sales in the last month so haven't had as much time as I wanted for OSS work.

@dgp1130
Copy link
Contributor Author

dgp1130 commented Mar 11, 2023

No worries @gregmagolan. Would it be helpful for me to split this into two PRs? One for the Go binary and one with the Starlark changes? That way you could merge them independently.

@alexeagle
Copy link
Member

Hey @dgp1130 sorry this got parked like this. Yeah if you're willing to make the Golang change in a PR, I'll review/merge/release with that, then the starlark will be able to rely on it being there.

Note to @gregmagolan - I have the same problem in container-structure-test now too. I know we've brainstormed some ways to make the golang library "live from HEAD" but it's probably not time to design that yet.

@dgp1130
Copy link
Contributor Author

dgp1130 commented Jul 30, 2023

No problem @alexeagle, I'm sure you've had more important things to worry about than this. I broke up the Go changes into a separate PR and made the new value optional to avoid breaking anything. Once that gets merged and released, we should be able to rebase this and merge it with the changes to the copy_to_directory rule.

It's not very important, but IMHO the new workspace name option in the other PR should be required. Unfortunately doing that would break the copy_to_directory rule since we can't change both atomically. So once this PR merges and the copy_to_directory rule is updated, we could consider a third PR which makes the workspace name option required in the Go tool. I don't know if that tool is used in other places or is considered public API such that a change like that would be considered breaking, but if not it might be worth updating it to make the option required.

… in `copy_to_directory`

Fixes aspect-build#359.

This updates the `copy_to_directory` tool to accept a workspace name representing the workspace of the target it is executing under. Any files in this workspace are automatically included, regardless of the `include_external_repositories` option. This makes it support usage within an external target (such as `@wksp//:dir`).
@dgp1130 dgp1130 force-pushed the external-copy-directory branch 3 times, most recently from f84c925 to 2d313ee Compare October 10, 2023 01:18
…workspace

Refs aspect-build#359.

This should catch regressions where no files are copied when built within an external workspace and not using `include_external_repositories`.
@dgp1130
Copy link
Contributor Author

dgp1130 commented Oct 10, 2023

Since #488 got merged, I rebased this PR so it should be mostly good to go. Unfortunately it seems like the CI setup has changed and it isn't running the new workspace properly. Not sure how this is supposed to work or why the directories seem to be missing, but hopefully someone can help point me in the right direction here?

@alexeagle
Copy link
Member

Somehow you had an extra /workspace segment on your new e2e folder and an extra e2e/workspace entry as well. Added a commit to fix that

Copy link
Member

@alexeagle alexeagle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@alexeagle alexeagle merged commit c6b39ab into aspect-build:main Oct 10, 2023
30 checks passed
@dgp1130 dgp1130 deleted the external-copy-directory branch October 10, 2023 14:17
@dgp1130
Copy link
Contributor Author

dgp1130 commented Oct 10, 2023

Thanks! Glad we're able to land this fix!

alexeagle added a commit that referenced this pull request Oct 10, 2023
… in `copy_to_directory()` (#360)

* fix: always include files from the same workspace as the build target in `copy_to_directory`

Fixes #359.

This updates the `copy_to_directory` tool to accept a workspace name representing the workspace of the target it is executing under. Any files in this workspace are automatically included, regardless of the `include_external_repositories` option. This makes it support usage within an external target (such as `@wksp//:dir`).

* test: add e2e test which uses `copy_to_directory` within an external workspace

Refs #359.

This should catch regressions where no files are copied when built within an external workspace and not using `include_external_repositories`.

* ci: fix stray workspace refs

---------

Co-authored-by: Alex Eagle <alex@aspect.dev>
alexeagle added a commit that referenced this pull request Dec 23, 2023
… in `copy_to_directory()` (#360)

* fix: always include files from the same workspace as the build target in `copy_to_directory`

Fixes #359.

This updates the `copy_to_directory` tool to accept a workspace name representing the workspace of the target it is executing under. Any files in this workspace are automatically included, regardless of the `include_external_repositories` option. This makes it support usage within an external target (such as `@wksp//:dir`).

* test: add e2e test which uses `copy_to_directory` within an external workspace

Refs #359.

This should catch regressions where no files are copied when built within an external workspace and not using `include_external_repositories`.

* ci: fix stray workspace refs

---------

Co-authored-by: Alex Eagle <alex@aspect.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working in review
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

[Bug]: copy_to_directory_bin_action() creates an empty directory when run in an external workspace.
3 participants