Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Wait for configuration reload completion to stabilize e2e tests #8124

Merged
merged 1 commit into from
May 8, 2024
Merged

Wait for configuration reload completion to stabilize e2e tests #8124

merged 1 commit into from
May 8, 2024

Conversation

bitoku
Copy link
Contributor

@bitoku bitoku commented May 4, 2024 •
edited by sohankunkerkar

Fixes #8074

What type of PR is this?

/kind ci
/kind flake

What this PR does / why we need it:

This PR makes some "reload config" tests to wait for the completion of reload.
Currently, those tests in reload_config.bats don't wait for PinnedImagesList updated.

  • reload config should update 'pinned_images'
  • reload config should update 'pinned_images' and only 'pause_image' is pinned
  • reload config should update 'pause_image' and it becomes 'pinned_images'

It causes a race condition and test failures.
Specifically, The test sends SIGHUP and runs crictl images.
It expects the results to be updated with the reloaded new config.
However when crictl images runs before PinnedImagesList is updated, the regexForPinnedImages are not updated, and the image which should be pinned won't be regarded as pinned.

cri-o/internal/storage/image.go

Lines 904 to 921 in 7a17235

func (svc *imageService) UpdatePinnedImagesList(pinnedImages []string) {
svc.regexForPinnedImages = CompileRegexpsForPinnedImages(pinnedImages)
}
// FilterPinnedImage checks if the given image needs to be pinned
// and excluded from kubelet's image GC.
func FilterPinnedImage(image string, pinnedImages []*regexp.Regexp) bool {
if len(pinnedImages) == 0 {
return false
}
for _, pinnedImage := range pinnedImages {
if pinnedImage.MatchString(image) {
return true
}
}
return false
}

cri-o/internal/storage/image.go

Lines 307 to 312 in 7a17235

for _, image := range image.Names {
if FilterPinnedImage(image, svc.regexForPinnedImages) {
imagePinned = true
break
}
}

example: https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/cri-o_cri-o/8102/pull-ci-cri-o-cri-o-main-ci-fedora-integration/1786581177407115264/artifacts/fedora-integration/cri-o-gather/artifacts/testout.txt

Which issue(s) this PR fixes:

None

Special notes for your reviewer:

Does this PR introduce a user-facing change?

None

@bitoku bitoku requested a review from mrunalp as a code owner May 4, 2024 14:57
@openshift-ci openshift-ci bot added release-note-none Denotes a PR that doesn't merit a release note. kind/ci Categorizes issue or PR as related to CI dco-signoff: yes Indicates the PR's author has DCO signed all their commits. kind/flake Categorizes issue or PR as related to a flaky test. labels May 4, 2024
@openshift-ci openshift-ci bot requested review from klihub and wgahnagl May 4, 2024 14:57
@openshift-ci openshift-ci bot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label May 4, 2024
@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented May 4, 2024

Hi @bitoku. Thanks for your PR.

I'm waiting for a cri-o member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@bitoku
Copy link
Contributor Author

bitoku commented May 4, 2024

cc: @roman-kiselenko

@roman-kiselenko
Copy link
Member

/ok-to-test

@openshift-ci openshift-ci bot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels May 4, 2024
@bitoku
Copy link
Contributor Author

bitoku commented May 4, 2024

/retest

kwilczynski
kwilczynski reviewed May 4, 2024
View reviewed changes
server/server.go Outdated Show resolved Hide resolved
@kwilczynski kwilczynski changed the title wait for reload completion to stablize e2e Wait for configuration reload completion to stabilize e2e tests May 4, 2024
@kwilczynski
Copy link
Member

/retest

@roman-kiselenko
Copy link
Member

/approve

@bitoku bitoku force-pushed the reload-config-e2e branch 2 times, most recently from 1426b84 to 5c9143c Compare May 5, 2024 10:28
@bitoku
wait for reload completion for stable e2e
c3bfcd3 
Signed-off-by: Ayato Tokubi <atokubi@redhat.com>
@bitoku
Copy link
Contributor Author

bitoku commented May 5, 2024

/retest

@kwilczynski
Copy link
Member

/hold

To see if there is a different way to solve this.

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 5, 2024
@kwilczynski
Copy link
Member

/unhold

@openshift-ci openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 7, 2024
@kwilczynski
Copy link
Member

/approve
/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label May 7, 2024
@haircommander
Copy link
Member

/approve

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented May 7, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bitoku, haircommander, kwilczynski, roman-kiselenko

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 7, 2024
@roman-kiselenko
Copy link
Member

Since the kata tests have passed, I'm going to update #8074 after the merge.

@sohankunkerkar
Copy link
Member

/retest

1 similar comment
@bitoku
Copy link
Contributor Author

bitoku commented May 7, 2024

/retest

@kwilczynski
Copy link
Member

/test ci-e2e-evented-pleg

@kwilczynski
Copy link
Member

/test periodics-images

@openshift-merge-bot openshift-merge-bot bot merged commit 2afe77b into cri-o:main May 8, 2024
69 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kwilczynski kwilczynski kwilczynski left review comments

@roman-kiselenko roman-kiselenko roman-kiselenko left review comments

@sohankunkerkar sohankunkerkar sohankunkerkar left review comments

@mrunalp mrunalp Awaiting requested review from mrunalp mrunalp is a code owner

@klihub klihub Awaiting requested review from klihub

@wgahnagl wgahnagl Awaiting requested review from wgahnagl

Assignees

@kwilczynski kwilczynski

@bitoku bitoku

@roman-kiselenko roman-kiselenko

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. dco-signoff: yes Indicates the PR's author has DCO signed all their commits. kind/ci Categorizes issue or PR as related to CI kind/flake Categorizes issue or PR as related to a flaky test. lgtm Indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note-none Denotes a PR that doesn't merit a release note.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Failing test on Kata runtime (container_min_memory)
5 participants
@bitoku @roman-kiselenko @kwilczynski @haircommander @sohankunkerkar