Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Connectors flaky tests fix #1717

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

valdar
Copy link
Contributor

@valdar valdar commented Apr 21, 2023

Description

Fixed flaky tests.
More in detail:

  • Fix waiting assertions to fail if waiting time expires.
  • LOCK/UNLOCk assertions made more robust.
  • Tests now logs to file and stderr.
  • Changes test feature parallelism, scenario randomization and test suit names.
  • Better error handling of keycloack delete clinets.
  • Removed unecessary assertions, uniformed waiting times, made some usernames unique.
  • Connector update reconcile does not have to save lastVersion anymore.
  • Fixed connectors deployed per namespace count to not fail if a deployment is deleted while conting operatin is in progress.
  • Implemented optimistic locking for Deployments connector updates/patch.

Verification Steps

  1. i=0
  2. while /home/valdar/projects/communityProjects/kas-fleet-manager/bin/gotestsum --junitfile data/results/integraton-tests-connector.xml --format short-verbose -- -p 1 -ldflags -s -v -timeout 15m -count=1 ./internal/connector/test/integration/... ; do ((i=i+1)) && echo $i ; done

Checklist (Definition of Done)

  • All acceptance criteria specified in JIRA have been completed
  • Unit and integration tests added that prove the fix is effective or the feature works (tested against emulated and non-emulated OCM environment)
  • Documentation added for the feature
  • CI and all relevant tests are passing
  • Code Review completed
  • Verified independently by reviewer
  • All PR comments are resolved either by addressing them or creating follow up tasks
  • Required metrics/dashboards/alerts have been added (or PR created).
  • Required Standard Operating Procedure (SOP) is added.
  • JIRA has been created for changes required on the client side

@codecov
Copy link

codecov bot commented Apr 21, 2023

Codecov Report

Merging #1717 (9153ed8) into main (ce2958d) will not change coverage.
The diff coverage is n/a.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #1717   +/-   ##
=======================================
  Coverage   82.26%   82.26%           
=======================================
  Files         161      161           
  Lines       14989    14989           
=======================================
  Hits        12330    12330           
  Misses       2223     2223           
  Partials      436      436           
Flag Coverage Δ
unittests 82.26% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Copy link

@manstis manstis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my concerns about gorm's Save() method changes (in another PR).


if err := dbConn.Save(resource).Error; err != nil {
saveResult := dbConn.Save(resource)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I worry about this.

See https://github.com/bf2fc6cc711aee1a0c2a/kas-fleet-manager/pull/1637/files#diff-aee57c287964506015af5e0e2839d1e78a19635c9ce232369e8be08a91101d89R508-R517

The behaviour of Save has changed in newer versions of gorm.

I believe PR #1717 and #1637 are effectively in a "race condition".

We should agree which should merge first and then check the other is unaffected.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As stated in the more general comment I would vote for merging this one since it fixes tests and current behavior and then reason about how to deal with new gorm version behavior.

@@ -505,7 +523,7 @@ func (k *connectorClusterService) UpdateConnectorDeploymentStatus(ctx context.Co
return services.HandleGoneError("Connector deployment", "id", deploymentStatus.ID)
}

if err := dbConn.Model(&deploymentStatus).Where("id = ? and version <= ?", deploymentStatus.ID, deploymentStatus.Version).Save(&deploymentStatus).Error; err != nil {
if err := dbConn.Where("id = ? and version <= ?", deploymentStatus.ID, deploymentStatus.Version).Save(&deploymentStatus).Error; err != nil {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here I just removed the use of Model() with Save() as is not supported and can lead to undefined behavior (as per documentation first NOTE: https://gorm.io/docs/update.html)

So I believe we can merge this one and then reason about the whole // See https://github.com/go-gorm/gorm/issues/6139 // We can no longer rely on the Save throwing a constraint violation exception when upserting a record that already exists. // By first checking that the record existences we ensure the Save only updates an existing record vs trying to create a duplicate.

test/cucumber/http_request.go Show resolved Hide resolved
@valdar
Copy link
Contributor Author

valdar commented Apr 22, 2023

See my concerns about gorm's Save() method changes (in another PR).

Yep that is a problem for the upgrade for sure. I fear an issue to make that behavior configurable needs to be open to gorm unless there is a solution to implement optimistic locking in scenarios where save() is used.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants