Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vendor: upgrade OpenTelemetry to v1.19.0 / v0.45.0 #9256

Merged
merged 1 commit into from Nov 15, 2023

Conversation

milas
Copy link
Contributor

@milas milas commented Oct 17, 2023

Upgrade OpenTelemetry core libs to v1.19.0 and contrib (for gRPC tracing) to v0.45.0.

The OpenTelemetry internal module structure/dependency graph is rather complex, and recently some parts (e.g. metrics) have graduated to "stable" from "unstable", so this upgrade is important to unblock downstream projects to be able to use newer versions of the OpenTelemetry libraries, as they can cause compatibility issues due to internal/peer dependency changes otherwise.

Hopefully, future updates won't be as problematic, such that projects using containerd as a dependency will be able to use newer versions of the libraries in a compatible fashion.

See also:

@k8s-ci-robot
Copy link

Hi @milas. Thanks for your PR.

I'm waiting for a containerd member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@thaJeztah
Copy link
Member

Oh while I see you here; perhaps you know; I asked this question the other day; another dependency related to OTEL (at least, it was added as part of the initial OTEL bits I think);

Who’s familiar with the grpc-middleware dependency? I noticed they moved towards v2, and wondering if that’s a switch container should make for V2. Also noticed there’s a providers/prometheus module in that repository, and wondering how it relates to the github.com/grpc-ecosystem/go-grpc-prometheus module we’re currently using (admitted, I didn’t spend much time on looking what each of those did)

https://github.com/grpc-ecosystem/go-grpc-middleware/releases

@thaJeztah
Copy link
Member

I see this is linked to moby/buildkit#4318

Note that containerd updates in main are for containerd v2.0 (so we can't consume main until the module is renamed); if we need this update in BuildKit (and if for that to work, containerd needs to be updated), your containerd PR has to be backported and accepted in the 1.7 release branch.

I see it's a rather large diff though, and includes an update to grpc; so for it to be backported to the 1.7 branch we must be sure it's backward compatible with older versions; e.g. both BuildKit v0.12 and BuildKit "master" (v0.13-dev) use the containerd 1.7 branch, so it needs to be compatible with both those versions.

Copy link
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

noticed that we were updating to a vulnerable version of grpc here, so left a quick comment so that we don't merge before that's addressed.

go.mod Outdated Show resolved Hide resolved
@milas
Copy link
Contributor Author

milas commented Oct 19, 2023

@crazy-max Do you have a sense of whether this will be an issue?

I see it's a rather large diff though, and includes an update to grpc; so for it to be backported to the 1.7 branch we must be sure it's backward compatible with older versions; e.g. both BuildKit v0.12 and BuildKit "master" (v0.13-dev) use the containerd 1.7 branch, so it needs to be compatible with both those versions.

@crazy-max
Copy link

@crazy-max Do you have a sense of whether this will be an issue?

I see it's a rather large diff though, and includes an update to grpc; so for it to be backported to the 1.7 branch we must be sure it's backward compatible with older versions; e.g. both BuildKit v0.12 and BuildKit "master" (v0.13-dev) use the containerd 1.7 branch, so it needs to be compatible with both those versions.

I don't think gRPC 1.58.3 will cause an issue on BuildKit. I will push changes to moby/buildkit#4318 to make sure of it.

Maybe we could bump gRPC in a dedicated PR on containerd first?

@crazy-max
Copy link

@milas Opened moby/buildkit#4359 to check gRPC update. Also took a look at otel updates on BuildKit side but encounter schema conflicts linked to otel/sdk: moby/buildkit#4356

@thaJeztah
Copy link
Member

@milas Opened moby/buildkit#4359 to check gRPC update. Also took a look at otel updates on BuildKit side but encounter schema conflicts linked to otel/sdk: moby/buildkit#4356

do we need a draft PR against the release/1.7 branch of containers to verify, or is updating in buildkit enough?

@crazy-max
Copy link

crazy-max commented Oct 20, 2023

@milas Opened moby/buildkit#4359 to check gRPC update. Also took a look at otel updates on BuildKit side but encounter schema conflicts linked to otel/sdk: moby/buildkit#4356

do we need a draft PR against the release/1.7 branch of containers to verify, or is updating in buildkit enough?

I think best would be a gRPC update PR (similar to moby/buildkit#4359) before OTEL one to ensure there is no regression on containerd repo.

milas added a commit to milas/containerd that referenced this pull request Oct 26, 2023
Upgrade to the latest OpenTelemetry libraries; this will unblock
a lot of downstream projects in the ecosystem to upgrade, as some
of the parts here were pre-1.0/unstable.

See also:
 * containerd#9256
milas added a commit to milas/containerd that referenced this pull request Oct 26, 2023
Upgrade to the latest OpenTelemetry libraries; this will unblock
a lot of downstream projects in the ecosystem to upgrade, as some
of the parts here were pre-1.0/unstable.

See also:
 * containerd#9256
milas added a commit to milas/containerd that referenced this pull request Oct 26, 2023
Upgrade to the latest OpenTelemetry libraries; this will unblock
a lot of downstream projects in the ecosystem to upgrade, as some
of the parts here were pre-1.0/unstable.

See also:
 * containerd#9256

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
@thaJeztah
Copy link
Member

🙈 needs a rebase (again) @milas

Upgrade OpenTelemetry core libs to v1.19.0 and contrib (for gRPC
tracing) to v0.45.0.

The OpenTelemetry internal module structure/dependency graph is
rather complex, and recently some parts (e.g. metrics) have
graduated to "stable" from "unstable", so this upgrade is important
to unblock downstream projects to be able to use newer versions of
the OpenTelemetry libraries, as they can cause compatibility issues
due to internal/peer dependency changes otherwise.

Hopefully, future updates won't be as problematic, such that projects
using containerd as a dependency will be able to use newer versions
of the libraries in a compatible fashion.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
@milas
Copy link
Contributor Author

milas commented Nov 15, 2023

Since #9281 was merged, it'd probably be good to get this in so that 1.7.x isn't on a newer version of OTel deps than 2.0.x.

And again, this bump is sorta important because some of the OTel components have now been 'stabilized' so by raising the limit here, we'll have many fewer problems with Go module MVS moving forward (I hope)

Copy link
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM if CI is happy :)

@dmcgowan dmcgowan added this pull request to the merge queue Nov 15, 2023
Merged via the queue into containerd:main with commit 134dc87 Nov 15, 2023
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants