Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[olm] Store previous versions of olm manifests #3906

Merged

Conversation

varshaprasad96
Copy link
Member

Description of the change:
This PR introduces an internal optimization of not going to github
and downloading the olm manifests when 'operator-sdk olm install'
command is used. Previous version of olm manifest is stored as
bindata and packaged along with sdk binary. When user specifies
any of this version, the manifests are fetched locally and
applied to the cluster.

Currently, manifests for only one version of olm (0.15.1) is stored.

Motivation for the change:
Make olm subcommands more robust.

Output:

INFO[0000] Fetching CRDs for version "0.15.1"
Fetching crd.yaml from locally stored bindata in SDK
INFO[0000] Fetching resources for version "0.15.1"
Fetching olm.yaml from locally stored bindata in SDK
I0917 10:47:33.803002   51339 request.go:621] Throttling request took 1.047136271s, request: GET:https://127.0.0.1:32780/apis/networking.k8s.io/v1beta1?timeout=32s
INFO[0007] Creating CRDs and resources
INFO[0007]   Creating CustomResourceDefinition "catalogsources.operators.coreos.com"
INFO[0007]   Creating CustomResourceDefinition "clusterserviceversions.operators.coreos.com"
INFO[0007]   Creating CustomResourceDefinition "installplans.operators.coreos.com"
INFO[0007]   Creating CustomResourceDefinition "operatorgroups.operators.coreos.com"
INFO[0007]   Creating CustomResourceDefinition "subscriptions.operators.coreos.com"
INFO[0007]   Creating Namespace "olm"
INFO[0007]   Creating Namespace "operators"
INFO[0007]   Creating ServiceAccount "olm/olm-operator-serviceaccount"
INFO[0007]   Creating ClusterRole "system:controller:operator-lifecycle-manager"
INFO[0007]   Creating ClusterRoleBinding "olm-operator-binding-olm"
INFO[0007]   Creating Deployment "olm/olm-operator"
INFO[0007]   Creating Deployment "olm/catalog-operator"
INFO[0007]   Creating ClusterRole "aggregate-olm-edit"
INFO[0007]   Creating ClusterRole "aggregate-olm-view"
INFO[0007]   Creating OperatorGroup "operators/global-operators"
INFO[0009]   Creating OperatorGroup "olm/olm-operators"
INFO[0009]   Creating ClusterServiceVersion "olm/packageserver"
INFO[0011]   Creating CatalogSource "olm/operatorhubio-catalog"
INFO[0011] Waiting for deployment/olm-operator rollout to complete
INFO[0011]   Waiting for Deployment "olm/olm-operator" to rollout: 0 of 1 updated replicas are available
INFO[0019]   Deployment "olm/olm-operator" successfully rolled out
INFO[0019] Waiting for deployment/catalog-operator rollout to complete
INFO[0019]   Waiting for Deployment "olm/catalog-operator" to rollout: 0 of 1 updated replicas are available
INFO[0021]   Deployment "olm/catalog-operator" successfully rolled out
INFO[0021] Waiting for deployment/packageserver rollout to complete
INFO[0021]   Waiting for Deployment "olm/packageserver" to rollout: 1 out of 2 new replicas have been updated
INFO[0029]   Waiting for Deployment "olm/packageserver" to rollout: 1 old replicas are pending termination
INFO[0043]   Deployment "olm/packageserver" successfully rolled out
INFO[0044] Successfully installed OLM version "0.15.1"

NAME                                            NAMESPACE    KIND                        STATUS
catalogsources.operators.coreos.com                          CustomResourceDefinition    Installed
clusterserviceversions.operators.coreos.com                  CustomResourceDefinition    Installed
installplans.operators.coreos.com                            CustomResourceDefinition    Installed
operatorgroups.operators.coreos.com                          CustomResourceDefinition    Installed
subscriptions.operators.coreos.com                           CustomResourceDefinition    Installed
olm                                                          Namespace                   Installed
operators                                                    Namespace                   Installed
olm-operator-serviceaccount                     olm          ServiceAccount              Installed
system:controller:operator-lifecycle-manager                 ClusterRole                 Installed
olm-operator-binding-olm                                     ClusterRoleBinding          Installed
olm-operator                                    olm          Deployment                  Installed
catalog-operator                                olm          Deployment                  Installed
aggregate-olm-edit                                           ClusterRole                 Installed
aggregate-olm-view                                           ClusterRole                 Installed
global-operators                                operators    OperatorGroup               Installed
olm-operators                                   olm          OperatorGroup               Installed
packageserver                                   olm          ClusterServiceVersion       Installed
operatorhubio-catalog                           olm          CatalogSource               Installed

Checklist

If the pull request includes user-facing changes, extra documentation is required:

@estroz
Copy link
Member

estroz commented Sep 17, 2020

@varshaprasad96 how much larger is the operator-sdk binary with just this bindata included? You can use

stat --printf="%s\n" "$(command -v operator-sdk)"

to check

@varshaprasad96
Copy link
Member Author

size of the binary after including bindata is ~69.32 MB, and SDK 1.0.0 binary is ~68.34 MB

olmbindata/olm_versions.go Outdated Show resolved Hide resolved
go.mod Outdated Show resolved Hide resolved
go.mod Outdated Show resolved Hide resolved
@varshaprasad96 varshaprasad96 force-pushed the olm-install/bindata branch 2 times, most recently from d462e12 to 9a584e7 Compare September 19, 2020 08:13
Makefile Outdated Show resolved Hide resolved
Copy link
Contributor

@camilamacedo86 camilamacedo86 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What in IHMO shows missing to be done here:

internal/olm/installer/client.go Outdated Show resolved Hide resolved
internal/olm/installer/client.go Outdated Show resolved Hide resolved
internal/olm/installer/client.go Outdated Show resolved Hide resolved
internal/olm/installer/client.go Outdated Show resolved Hide resolved
internal/olm/installer/client.go Outdated Show resolved Hide resolved
internal/olm/installer/client.go Show resolved Hide resolved
hack/generate/olm_bindata.sh Show resolved Hide resolved
hack/generate/olm_bindata.sh Outdated Show resolved Hide resolved
olmbindata/olm_versions.go Outdated Show resolved Hide resolved
Copy link
Member

@estroz estroz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nearly there, mostly nits.

@varshaprasad96 varshaprasad96 force-pushed the olm-install/bindata branch 4 times, most recently from 7eaee58 to d59940c Compare September 22, 2020 18:50
hack/check-license.sh Outdated Show resolved Hide resolved
This PR introduces an internal optimization of not going to github
and downloading the olm manifests when 'operator-sdk olm install'
command is used. Previous version of olm manifest is stored as
bindata and packaged along with sdk binary. When user specifies
any of this version, the manifests are fetched locally and
applied to the cluster.
Copy link
Contributor

@camilamacedo86 camilamacedo86 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work 👍 IMO before merge we need to

After that, it has my /lgtm

Copy link
Member

@estroz estroz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does need a test, but just an integration test (in test/integration) that installs the specific version of OLM stored as bindata. No need to change e2e tests.

Makefile Outdated Show resolved Hide resolved
Makefile Outdated Show resolved Hide resolved
changelog/fragments/store-olm-bindata.yaml Outdated Show resolved Hide resolved
hack/generate/olm_bindata.sh Outdated Show resolved Hide resolved
internal/bindata/olmbindata/versions.go Outdated Show resolved Hide resolved
hack/lib/test_lib.sh Outdated Show resolved Hide resolved
@varshaprasad96 varshaprasad96 force-pushed the olm-install/bindata branch 4 times, most recently from 012b5a6 to dca96e8 Compare September 29, 2020 22:15
Copy link
Member

@estroz estroz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running make bindata needs to be added as a step to release docs, then LGTM

internal/olm/installer/client.go Outdated Show resolved Hide resolved
internal/olm/installer/client.go Outdated Show resolved Hide resolved
Copy link
Member

@estroz estroz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Oct 1, 2020
@varshaprasad96 varshaprasad96 merged commit bfc6bfa into operator-framework:master Oct 1, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants