Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deps: Upgrade google.golang.org/{genproto, grpc} #13593

Merged
merged 1 commit into from
Jul 28, 2023
Merged

Conversation

abhinav
Copy link
Contributor

@abhinav abhinav commented Jul 26, 2023

Updates to the latest versions of
google.golang.org/genproto and google.golang.org/grpc
in all submodules in the repository.

This is necessary because in a recent change,
genproto split out some of its subpackages into independent submodules.
(googleapis/go-genproto#1015)

As a result of this, some users may see the error:

google.golang.org/genproto/googleapis/rpc/status: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/status in multiple modules:
    google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 (/home/runner/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230410155749-daa745c078e1/googleapis/rpc/status)
    google.golang.org/genproto/googleapis/rpc v0.0.0-20230725213213-b022f6e96895

Because pu/pu is using 20230410155749,
which has googleapis/rpc as a subpackage,
but another dependency references the independent submodule (20230725213213),
so the system doesn't know which module to use for the import path,
google.golang.org/genproto/googleapis/rpc/status.

This is a problem for codegen tests and ProgramTest-based tests
for Pulumi Go programs that do not have a go.mod in the test directory.
This issue was encountered by @thomas11 while attempting to upgrade
dependencies in pulumi-docker (pulumi/pulumi-docker#700).

The grpc upgrade is necessary because the current version of grpc
also pulls the outdated version of genproto.

@abhinav abhinav requested a review from a team July 26, 2023 18:10
@pulumi-bot
Copy link
Contributor

pulumi-bot commented Jul 26, 2023

Changelog

[uncommitted] (2023-07-27)

Miscellaneous

  • [sdk/go] Bump the minimum required versions of google.golang.org/genproto and google.golang.org/grpc.
    #13593

abhinav added a commit that referenced this pull request Jul 26, 2023
The regression test for #13301 needs an intentionally bad go.mod file.
This file was excluded from `make tidy`, allowing it to remain invalid,
but this doesn't protect it from bulk commands like the following
used in #13593

```bash
find . -name go.mod -exec dirname '{}' ';' | while read R; do
  (cd "$R" && ... && go mod tidy)
done
```

In fact, #13593 accidentally tidied this go.mod file
(removing the extraneous dependencies critical to the regression test)
and failed in CI.

To prevent issues like this, rename the go.mod to go.mod.bad,
and rename it back to go.mod in the test environment at test time.

This also lets us revert the `make tidy` exclusion support in tidy.sh.
@abhinav
Copy link
Contributor Author

abhinav commented Jul 26, 2023

bors merge

bors bot added a commit that referenced this pull request Jul 26, 2023
13593: deps: Upgrade google.golang.org/genproto r=abhinav a=abhinav

Updates to the latest version of google.golang.org/genproto
in all submodules in the repository.

This is necessary because in a recent change,
genproto split out some of its subpackages into independent submodules.
(googleapis/go-genproto#1015)

As a result of this, some users may see the error:

```
google.golang.org/genproto/googleapis/rpc/status: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/status in multiple modules:
    google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 (/home/runner/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230410155749-daa745c078e1/googleapis/rpc/status)
    google.golang.org/genproto/googleapis/rpc v0.0.0-20230725213213-b022f6e96895
```

Because pu/pu is using 20230410155749,
which has googleapis/rpc as a subpackage,
but another dependency references the independent submodule (20230725213213),
so the system doesn't know which module to use for the import path,
google.golang.org/genproto/googleapis/rpc/status.

This is specifically a problem for ProgramTest-based tests
for Pulumi Go programs that do not have a go.mod in the test directory.
This issue was encountered by `@thomas11` while attempting to upgrade
dependencies in pulumi-docker (pulumi/pulumi-docker#700).

---

The contents of this commit were generated with the following command:

```
find . -name go.mod -exec dirname '{}' ';' | while read R; do
  (echo "--- $R" && cd "$R" && go get google.golang.org/genproto@latest && go mod tidy) || break
done
```

This failed for tests/integration/go/go-build-target
which had an old `go 1.17` directive. This was changed to `go 1.20`.

This also accidentally tidied up go/regress-13301 which was reverted.
PR #13596 aims to prevent mistakes like this in the future.

Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
abhinav added a commit that referenced this pull request Jul 26, 2023
The regression test for #13301 needs an intentionally bad go.mod file.
This file was excluded from `make tidy`, allowing it to remain invalid,
but this doesn't protect it from bulk commands like the following
used in #13593

```bash
find . -name go.mod -exec dirname '{}' ';' | while read R; do
  (cd "$R" && ... && go mod tidy)
done
```

In fact, #13593 accidentally tidied this go.mod file
(removing the extraneous dependencies critical to the regression test)
and failed in CI.

To prevent issues like this, rename the go.mod to go.mod.bad,
and rename it back to go.mod in the test environment at test time.

This also lets us revert the `make tidy` exclusion support in tidy.sh.
@abhinav
Copy link
Contributor Author

abhinav commented Jul 26, 2023

bors cancel

@bors
Copy link
Contributor

bors bot commented Jul 26, 2023

Canceled.

@abhinav
Copy link
Contributor Author

abhinav commented Jul 26, 2023

bors merge

bors bot added a commit that referenced this pull request Jul 26, 2023
13593: deps: Upgrade google.golang.org/genproto r=abhinav a=abhinav

Updates to the latest version of google.golang.org/genproto
in all submodules in the repository.

This is necessary because in a recent change,
genproto split out some of its subpackages into independent submodules.
(googleapis/go-genproto#1015)

As a result of this, some users may see the error:

```
google.golang.org/genproto/googleapis/rpc/status: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/status in multiple modules:
    google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 (/home/runner/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230410155749-daa745c078e1/googleapis/rpc/status)
    google.golang.org/genproto/googleapis/rpc v0.0.0-20230725213213-b022f6e96895
```

Because pu/pu is using 20230410155749,
which has googleapis/rpc as a subpackage,
but another dependency references the independent submodule (20230725213213),
so the system doesn't know which module to use for the import path,
google.golang.org/genproto/googleapis/rpc/status.

This is specifically a problem for ProgramTest-based tests
for Pulumi Go programs that do not have a go.mod in the test directory.
This issue was encountered by `@thomas11` while attempting to upgrade
dependencies in pulumi-docker (pulumi/pulumi-docker#700).

---

The contents of this commit were generated with the following command:

```
find . -name go.mod -exec dirname '{}' ';' | while read R; do
  (echo "--- $R" && cd "$R" && go get google.golang.org/genproto@latest && go mod tidy) || break
done
```

This failed for tests/integration/go/go-build-target
which had an old `go 1.17` directive. This was changed to `go 1.20`.

This also accidentally tidied up go/regress-13301 which was reverted.
PR #13596 aims to prevent mistakes like this in the future.

13598: Changelog and go.mod updates for v3.76.1 r=pulumi-bot a=pulumi-bot

bors merge

Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
Co-authored-by: github-actions <github-actions@github.com>
@bors
Copy link
Contributor

bors bot commented Jul 26, 2023

Build failed (retrying...):

bors bot added a commit that referenced this pull request Jul 26, 2023
13584: [sdkgen/python] SDKgen on external enums r=dixler a=dixler

<!--- 
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->

Fixes #12695

Python SDKGen was not correctly referencing external enum types which needed an import alias.

## Checklist

- [x] I have run `make tidy` to update any new dependencies
- [x] I have run `make lint` to verify my code passes the lint check
  - [x] I have formatted my code using `gofumpt`

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [x] I have added tests that prove my fix is effective or that my feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


13596: test(regress-13301): Avoid accidental tidying r=abhinav a=abhinav

The regression test for #13301 needs an intentionally bad go.mod file.
This file was excluded from `make tidy`, allowing it to remain invalid,
but this doesn't protect it from bulk commands like the following
used in #13593

```bash
find . -name go.mod -exec dirname '{}' ';' | while read R; do
  (cd "$R" && ... && go mod tidy)
done
```

In fact, #13593 accidentally tidied this go.mod file
(removing the extraneous dependencies critical to the regression test)
and failed in CI.

To prevent issues like this, rename the go.mod to go.mod.bad,
and rename it back to go.mod in the test environment at test time.

This also lets us revert the `make tidy` exclusion support in tidy.sh.


Co-authored-by: Kyle Dixler <kyle@pulumi.com>
Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
@abhinav
Copy link
Contributor Author

abhinav commented Jul 26, 2023

bors cancel

@bors
Copy link
Contributor

bors bot commented Jul 26, 2023

Canceled.

@abhinav
Copy link
Contributor Author

abhinav commented Jul 26, 2023

Something is off about those failures. These tests all pass locally for me.

Copy link
Contributor Author

abhinav commented Jul 26, 2023

Current dependencies on/for this PR:

This comment was auto-generated by Graphite.

@abhinav
Copy link
Contributor Author

abhinav commented Jul 26, 2023

bors merge

bors bot added a commit that referenced this pull request Jul 26, 2023
13593: deps: Upgrade google.golang.org/genproto r=abhinav a=abhinav

Updates to the latest version of google.golang.org/genproto
in all submodules in the repository.

This is necessary because in a recent change,
genproto split out some of its subpackages into independent submodules.
(googleapis/go-genproto#1015)

As a result of this, some users may see the error:

```
google.golang.org/genproto/googleapis/rpc/status: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/status in multiple modules:
    google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 (/home/runner/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230410155749-daa745c078e1/googleapis/rpc/status)
    google.golang.org/genproto/googleapis/rpc v0.0.0-20230725213213-b022f6e96895
```

Because pu/pu is using 20230410155749,
which has googleapis/rpc as a subpackage,
but another dependency references the independent submodule (20230725213213),
so the system doesn't know which module to use for the import path,
google.golang.org/genproto/googleapis/rpc/status.

This is specifically a problem for ProgramTest-based tests
for Pulumi Go programs that do not have a go.mod in the test directory.
This issue was encountered by `@thomas11` while attempting to upgrade
dependencies in pulumi-docker (pulumi/pulumi-docker#700).

---

The contents of this commit were generated with the following command:

```
find . -name go.mod -exec dirname '{}' ';' | while read R; do
  (echo "--- $R" && cd "$R" && go get google.golang.org/genproto@latest && go mod tidy) || break
done
```

This failed for tests/integration/go/go-build-target
which had an old `go 1.17` directive. This was changed to `go 1.20`.

This also accidentally tidied up go/regress-13301 which was reverted.
PR #13596 aims to prevent mistakes like this in the future.

13596: test(regress-13301): Avoid accidental tidying r=abhinav a=abhinav

The regression test for #13301 needs an intentionally bad go.mod file.
This file was excluded from `make tidy`, allowing it to remain invalid,
but this doesn't protect it from bulk commands like the following
used in #13593

```bash
find . -name go.mod -exec dirname '{}' ';' | while read R; do
  (cd "$R" && ... && go mod tidy)
done
```

In fact, #13593 accidentally tidied this go.mod file
(removing the extraneous dependencies critical to the regression test)
and failed in CI.

To prevent issues like this, rename the go.mod to go.mod.bad,
and rename it back to go.mod in the test environment at test time.

This also lets us revert the `make tidy` exclusion support in tidy.sh.


Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
@bors
Copy link
Contributor

bors bot commented Jul 27, 2023

Build failed (retrying...):

bors bot added a commit that referenced this pull request Jul 27, 2023
13593: deps: Upgrade google.golang.org/genproto r=abhinav a=abhinav

Updates to the latest version of google.golang.org/genproto
in all submodules in the repository.

This is necessary because in a recent change,
genproto split out some of its subpackages into independent submodules.
(googleapis/go-genproto#1015)

As a result of this, some users may see the error:

```
google.golang.org/genproto/googleapis/rpc/status: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/status in multiple modules:
    google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 (/home/runner/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230410155749-daa745c078e1/googleapis/rpc/status)
    google.golang.org/genproto/googleapis/rpc v0.0.0-20230725213213-b022f6e96895
```

Because pu/pu is using 20230410155749,
which has googleapis/rpc as a subpackage,
but another dependency references the independent submodule (20230725213213),
so the system doesn't know which module to use for the import path,
google.golang.org/genproto/googleapis/rpc/status.

This is specifically a problem for ProgramTest-based tests
for Pulumi Go programs that do not have a go.mod in the test directory.
This issue was encountered by `@thomas11` while attempting to upgrade
dependencies in pulumi-docker (pulumi/pulumi-docker#700).

---

The contents of this commit were generated with the following command:

```
find . -name go.mod -exec dirname '{}' ';' | while read R; do
  (echo "--- $R" && cd "$R" && go get google.golang.org/genproto@latest && go mod tidy) || break
done
```

This failed for tests/integration/go/go-build-target
which had an old `go 1.17` directive. This was changed to `go 1.20`.

This also accidentally tidied up go/regress-13301 which was reverted.
PR #13596 aims to prevent mistakes like this in the future.

Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
@bors
Copy link
Contributor

bors bot commented Jul 27, 2023

Build failed:

bors bot added a commit that referenced this pull request Jul 27, 2023
13596: test(regress-13301): Avoid accidental tidying r=abhinav a=abhinav

The regression test for #13301 needs an intentionally bad go.mod file.
This file was excluded from `make tidy`, allowing it to remain invalid,
but this doesn't protect it from bulk commands like the following
used in #13593

```bash
find . -name go.mod -exec dirname '{}' ';' | while read R; do
  (cd "$R" && ... && go mod tidy)
done
```

In fact, #13593 accidentally tidied this go.mod file
(removing the extraneous dependencies critical to the regression test)
and failed in CI.

To prevent issues like this, rename the go.mod to go.mod.bad,
and rename it back to go.mod in the test environment at test time.

This also lets us revert the `make tidy` exclusion support in tidy.sh.


Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
@abhinav
Copy link
Contributor Author

abhinav commented Jul 27, 2023

bors retry

bors bot added a commit that referenced this pull request Jul 27, 2023
13593: deps: Upgrade google.golang.org/genproto r=abhinav a=abhinav

Updates to the latest version of google.golang.org/genproto
in all submodules in the repository.

This is necessary because in a recent change,
genproto split out some of its subpackages into independent submodules.
(googleapis/go-genproto#1015)

As a result of this, some users may see the error:

```
google.golang.org/genproto/googleapis/rpc/status: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/status in multiple modules:
    google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 (/home/runner/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230410155749-daa745c078e1/googleapis/rpc/status)
    google.golang.org/genproto/googleapis/rpc v0.0.0-20230725213213-b022f6e96895
```

Because pu/pu is using 20230410155749,
which has googleapis/rpc as a subpackage,
but another dependency references the independent submodule (20230725213213),
so the system doesn't know which module to use for the import path,
google.golang.org/genproto/googleapis/rpc/status.

This is specifically a problem for ProgramTest-based tests
for Pulumi Go programs that do not have a go.mod in the test directory.
This issue was encountered by `@thomas11` while attempting to upgrade
dependencies in pulumi-docker (pulumi/pulumi-docker#700).

---

The contents of this commit were generated with the following command:

```
find . -name go.mod -exec dirname '{}' ';' | while read R; do
  (echo "--- $R" && cd "$R" && go get google.golang.org/genproto@latest && go mod tidy) || break
done
```

This failed for tests/integration/go/go-build-target
which had an old `go 1.17` directive. This was changed to `go 1.20`.

This also accidentally tidied up go/regress-13301 which was reverted.
PR #13596 aims to prevent mistakes like this in the future.

13596: test(regress-13301): Avoid accidental tidying r=abhinav a=abhinav

The regression test for #13301 needs an intentionally bad go.mod file.
This file was excluded from `make tidy`, allowing it to remain invalid,
but this doesn't protect it from bulk commands like the following
used in #13593

```bash
find . -name go.mod -exec dirname '{}' ';' | while read R; do
  (cd "$R" && ... && go mod tidy)
done
```

In fact, #13593 accidentally tidied this go.mod file
(removing the extraneous dependencies critical to the regression test)
and failed in CI.

To prevent issues like this, rename the go.mod to go.mod.bad,
and rename it back to go.mod in the test environment at test time.

This also lets us revert the `make tidy` exclusion support in tidy.sh.


Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
@bors
Copy link
Contributor

bors bot commented Jul 27, 2023

Build failed (retrying...):

bors bot added a commit that referenced this pull request Jul 27, 2023
13593: deps: Upgrade google.golang.org/genproto r=abhinav a=abhinav

Updates to the latest version of google.golang.org/genproto
in all submodules in the repository.

This is necessary because in a recent change,
genproto split out some of its subpackages into independent submodules.
(googleapis/go-genproto#1015)

As a result of this, some users may see the error:

```
google.golang.org/genproto/googleapis/rpc/status: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/status in multiple modules:
    google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 (/home/runner/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230410155749-daa745c078e1/googleapis/rpc/status)
    google.golang.org/genproto/googleapis/rpc v0.0.0-20230725213213-b022f6e96895
```

Because pu/pu is using 20230410155749,
which has googleapis/rpc as a subpackage,
but another dependency references the independent submodule (20230725213213),
so the system doesn't know which module to use for the import path,
google.golang.org/genproto/googleapis/rpc/status.

This is specifically a problem for ProgramTest-based tests
for Pulumi Go programs that do not have a go.mod in the test directory.
This issue was encountered by `@thomas11` while attempting to upgrade
dependencies in pulumi-docker (pulumi/pulumi-docker#700).

---

The contents of this commit were generated with the following command:

```
find . -name go.mod -exec dirname '{}' ';' | while read R; do
  (echo "--- $R" && cd "$R" && go get google.golang.org/genproto@latest && go mod tidy) || break
done
```

This failed for tests/integration/go/go-build-target
which had an old `go 1.17` directive. This was changed to `go 1.20`.

This also accidentally tidied up go/regress-13301 which was reverted.
PR #13596 aims to prevent mistakes like this in the future.

Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
@abhinav
Copy link
Contributor Author

abhinav commented Jul 27, 2023

bors cancel

@bors
Copy link
Contributor

bors bot commented Jul 27, 2023

Canceled.

@abhinav
Copy link
Contributor Author

abhinav commented Jul 27, 2023

The failures suggest that this is still broken:

        internal-dependencies-go/example imports
        	github.com/pulumi/pulumi/sdk/v3/go/pulumi imports
        	github.com/pulumi/pulumi/sdk/v3/go/common/resource/plugin imports
        	google.golang.org/grpc/status imports
        	google.golang.org/genproto/googleapis/rpc/status: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/status in multiple modules:
        	google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 (/home/runner/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230410155749-daa745c078e1/googleapis/rpc/status)
        	google.golang.org/genproto/googleapis/rpc v0.0.0-20230706204954-ccb25ca9f130 (/home/runner/go/pkg/mod/google.golang.org/genproto/googleapis/rpc@v0.0.0-20230706204954-ccb25ca9f130/status)
        internal-dependencies-go/example imports
        	github.com/pulumi/pulumi/sdk/v3/go/pulumi imports
        	github.com/pulumi/pulumi/sdk/v3/go/common/resource/plugin imports
        	google.golang.org/grpc/codes tested by
        	google.golang.org/grpc/codes.test imports
        	google.golang.org/genproto/googleapis/rpc/code: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/code in multiple modules:
        	google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 (/home/runner/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230410155749-daa745c078e1/googleapis/rpc/code)
        	google.golang.org/genproto/googleapis/rpc v0.0.0-20230706204954-ccb25ca9f130 (/home/runner/go/pkg/mod/google.golang.org/genproto/googleapis/rpc@v0.0.0-20230706204954-ccb25ca9f130/code)
        internal-dependencies-go/example imports
        	github.com/pulumi/pulumi/sdk/v3/go/pulumi imports
        	github.com/pulumi/pulumi/sdk/v3/go/common/resource/plugin imports
        	google.golang.org/grpc/status tested by
        	google.golang.org/grpc/status.test imports
        	google.golang.org/genproto/googleapis/rpc/errdetails: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/errdetails in multiple modules:
        	google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 (/home/runner/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230410155749-daa745c078e1/googleapis/rpc/errdetails)
        	google.golang.org/genproto/googleapis/rpc v0.0.0-20230706204954-ccb25ca9f130 (/home/runner/go/pkg/mod/google.golang.org/genproto/googleapis/rpc@v0.0.0-20230706204954-ccb25ca9f130/errdetails)

The tests passed locally when I tried them last. Will dig in more later. Converting to Draft for now.

@abhinav abhinav marked this pull request as draft July 27, 2023 16:20
bors bot added a commit that referenced this pull request Jul 27, 2023
13588: A few fixes for `pulumi logs` r=lukehoban a=lukehoban

While many of the most fundamental limitations of `pulumi logs` are still dependent on moving this code out of pulumi/pulumi and into pulumi/pulumi-aws and other resource providers (#608), this PR implements a few more tactical fixes which improve some common scenarios.

Unfortunately, this whole area of the code is largely untested, in part because we didn't want to add a dependency on AWS in this layer (yet another reason we really want to do #608).  We could choose to further violate this layering and add tests here that use `pulumi-aws` to deploy Pulumi resources and then test them against this AWS operations provider.

In the meantime, I have manually validated fixes for the following issues.
    
Fixes #2665.
Fixes #1926.
Fixes #3947.
Fixes #1828.



13596: test(regress-13301): Avoid accidental tidying r=abhinav a=abhinav

The regression test for #13301 needs an intentionally bad go.mod file.
This file was excluded from `make tidy`, allowing it to remain invalid,
but this doesn't protect it from bulk commands like the following
used in #13593

```bash
find . -name go.mod -exec dirname '{}' ';' | while read R; do
  (cd "$R" && ... && go mod tidy)
done
```

In fact, #13593 accidentally tidied this go.mod file
(removing the extraneous dependencies critical to the regression test)
and failed in CI.

To prevent issues like this, rename the go.mod to go.mod.bad,
and rename it back to go.mod in the test environment at test time.

This also lets us revert the `make tidy` exclusion support in tidy.sh.


Co-authored-by: Luke Hoban <lukehoban@gmail.com>
Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
@abhinav
Copy link
Contributor Author

abhinav commented Jul 27, 2023

bors try

bors bot added a commit that referenced this pull request Jul 27, 2023
@bors
Copy link
Contributor

bors bot commented Jul 27, 2023

try

Build failed:

@abhinav
Copy link
Contributor Author

abhinav commented Jul 27, 2023

I'm really confused. CI fails consistently. The test runs fine on Linux and macOS.

=== RUN   TestGeneratePackage
=== PAUSE TestGeneratePackage
=== CONT  TestGeneratePackage
=== RUN   TestGeneratePackage/internal-dependencies-go
=== PAUSE TestGeneratePackage/internal-dependencies-go
=== CONT  TestGeneratePackage/internal-dependencies-go
    sdk_driver.go:457: Emit Go internal dependencies
=== RUN   TestGeneratePackage/internal-dependencies-go/go/compile
    gen_test.go:110: Found an existing go.mod, leaving as is
    command.go:34: **** Invoke '/usr/lib/go/bin/go mod tidy' in '/Users/abg/src/pulumi/pulumi/pkg/codegen/testing/test/testdata/internal-dependencies-go/go'
    command.go:106: Command completed without output
    command.go:34: **** Invoke '/usr/lib/go/bin/go build -v all' in '/Users/abg/src/pulumi/pulumi/pkg/codegen/testing/test/testdata/internal-dependencies-go/go'
    command.go:106: Command completed without output
=== RUN   TestGeneratePackage/internal-dependencies-go/go/test
    command.go:34: **** Invoke '/usr/lib/go/bin/go test internal-dependencies-go/...' in '/Users/abg/src/pulumi/pulumi/pkg/codegen/testing/test/testdata/internal-dependencies-go/go'
    command.go:106: Command completed without output
--- PASS: TestGeneratePackage (0.00s)
    --- PASS: TestGeneratePackage/internal-dependencies-go (4.09s)
        --- PASS: TestGeneratePackage/internal-dependencies-go/go/compile (3.43s)
        --- PASS: TestGeneratePackage/internal-dependencies-go/go/test (0.65s)
PASS
ok      github.com/pulumi/pulumi/pkg/v3/codegen/go      4.125s

@abhinav
Copy link
Contributor Author

abhinav commented Jul 27, 2023

Okay, so the local checkout had the generated go.mod files sitting around, which is why it's been passing locally

@abhinav
Copy link
Contributor Author

abhinav commented Jul 27, 2023

Okay, so the old version of grpc was pulling in the old version of genproto, which was causing that issue for tests. Upgrading that too.

@abhinav
Copy link
Contributor Author

abhinav commented Jul 27, 2023

bors try

bors bot added a commit that referenced this pull request Jul 27, 2023
@abhinav abhinav marked this pull request as ready for review July 27, 2023 23:21
@abhinav abhinav changed the title deps: Upgrade google.golang.org/genproto deps: Upgrade google.golang.org/{genproto, grpc} Jul 27, 2023
Updates to the latest versions of
google.golang.org/genproto and google.golang.org/grpc
in all submodules in the repository.

This is necessary because in a recent change,
genproto split out some of its subpackages into independent submodules.
(googleapis/go-genproto#1015)

As a result of this, some users may see the error:

```
google.golang.org/genproto/googleapis/rpc/status: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/status in multiple modules:
    google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 (/home/runner/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230410155749-daa745c078e1/googleapis/rpc/status)
    google.golang.org/genproto/googleapis/rpc v0.0.0-20230725213213-b022f6e96895
```

Because pu/pu is using 20230410155749,
which has googleapis/rpc as a subpackage,
but another dependency references the independent submodule (20230725213213),
so the system doesn't know which module to use for the import path,
google.golang.org/genproto/googleapis/rpc/status.

This is a problem for codegen tests and ProgramTest-based tests
for Pulumi Go programs that do not have a go.mod in the test directory.
This issue was encountered by @thomas11 while attempting to upgrade
dependencies in pulumi-docker (pulumi/pulumi-docker#700).

The grpc upgrade is necessary because the current version of grpc
also pulls the outdated version of genproto.
@abhinav
Copy link
Contributor Author

abhinav commented Jul 27, 2023

bors cancel

@abhinav
Copy link
Contributor Author

abhinav commented Jul 27, 2023

bors try

@bors
Copy link
Contributor

bors bot commented Jul 27, 2023

try

Already running a review

@bors
Copy link
Contributor

bors bot commented Jul 27, 2023

try

Build failed:

@abhinav
Copy link
Contributor Author

abhinav commented Jul 28, 2023

bors merge

bors bot added a commit that referenced this pull request Jul 28, 2023
13593: deps: Upgrade google.golang.org/{genproto, grpc} r=abhinav a=abhinav

Updates to the latest versions of
google.golang.org/genproto and google.golang.org/grpc
in all submodules in the repository.

This is necessary because in a recent change,
genproto split out some of its subpackages into independent submodules.
(googleapis/go-genproto#1015)

As a result of this, some users may see the error:

```
google.golang.org/genproto/googleapis/rpc/status: ambiguous import: found package google.golang.org/genproto/googleapis/rpc/status in multiple modules:
    google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 (/home/runner/go/pkg/mod/google.golang.org/genproto@v0.0.0-20230410155749-daa745c078e1/googleapis/rpc/status)
    google.golang.org/genproto/googleapis/rpc v0.0.0-20230725213213-b022f6e96895
```

Because pu/pu is using 20230410155749,
which has googleapis/rpc as a subpackage,
but another dependency references the independent submodule (20230725213213),
so the system doesn't know which module to use for the import path,
google.golang.org/genproto/googleapis/rpc/status.

This is a problem for codegen tests and ProgramTest-based tests
for Pulumi Go programs that do not have a go.mod in the test directory.
This issue was encountered by `@thomas11` while attempting to upgrade
dependencies in pulumi-docker (pulumi/pulumi-docker#700).

The grpc upgrade is necessary because the current version of grpc
also pulls the outdated version of genproto.

Co-authored-by: Abhinav Gupta <abhinav@pulumi.com>
@bors
Copy link
Contributor

bors bot commented Jul 28, 2023

Build failed:

@abhinav
Copy link
Contributor Author

abhinav commented Jul 28, 2023

bors merge

@bors
Copy link
Contributor

bors bot commented Jul 28, 2023

Build succeeded!

The publicly hosted instance of bors-ng is deprecated and will go away soon.

If you want to self-host your own instance, instructions are here.
For more help, visit the forum.

If you want to switch to GitHub's built-in merge queue, visit their help page.

@bors bors bot merged commit 6728c87 into master Jul 28, 2023
52 checks passed
@bors bors bot deleted the abhinav/upgrade-genproto branch July 28, 2023 05:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants