Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Flush EC2 service cache on new rollup #316

Closed
wants to merge 74 commits into from

Conversation

eytan-avisror
Copy link
Contributor

Fixes #309

This issue caused by stale cache in LaunchTemplate versions if multiple upgrades happen in succession:

if strings.EqualFold(templateVersion, LaunchTemplateVersionLatest) {
templateVersion = awsprovider.GetTemplateLatestVersion(r.Cloud.LaunchTemplates, launchTemplateName)
}

In this case r.Cloud.LaunchTemplates template versions is stale, adding the flushing on new rollup admission should fix this problem.

Eytan Avisror and others added 30 commits January 11, 2021 21:33
Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
* Validation step to check Nodes and ASG launch configs

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>

* Validating launch definition after a rolling upgrade

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
* Validation step to check Nodes and ASG launch configs

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Validating launch definition after a rolling upgrade

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Resolve error log message and return statement

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Adding Functional Test (keikoproj#113)

* Adding BDD, workflow and badge

* Changing CI workflow job name

* Adding make manifests

* Clarifying cron time zone comment

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* release 0.13 (keikoproj#115)

* release 0.13

* Update CHANGELOG.md

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* bump version (keikoproj#116)

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Repo selection for CI and BDD workflows & CI step for releases (keikoproj#117)

* CI-BDD not on forks & Step for releases (keikoproj#2)

* Testing CI-BDD not on forks & Step for releases

* Adding step for image with tag git-tag

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Terminate unjoined nodes

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Resolving PR comments

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Set version and update CHANGELOG for version 0.14. (keikoproj#121)

Co-authored-by: Shri Javadekar <shrinand_javadekar@intuit.com>
Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Bump version to 0.15-dev.

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Fix typo in README.md. (keikoproj#125)

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Ignore the terminated instance during upgrade

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Added WARNING prefix in the logging

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Apply suggestions from code review

Co-authored-by: Kevin Downey <kevdowney@gmail.com>
Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Capitalize sprintf to Sprintf

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Upgrade to Go 1.15 (keikoproj#128)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Fix few typos and simplify error returns, remove redundant types (keikoproj#131)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Readiness gates implementation for eager mode (keikoproj#130)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Adding Functional Test (keikoproj#113)

* Adding BDD, workflow and badge

* Changing CI workflow job name

* Adding make manifests

* Clarifying cron time zone comment

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Validation step to check Nodes and ASG launch configs (keikoproj#112)

* Validation step to check Nodes and ASG launch configs

* Validating launch definition after a rolling upgrade

* Resolve error log message and return statement

Co-authored-by: Eytan Avisror <eytan.avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* release 0.13 (keikoproj#115)

* release 0.13

* Update CHANGELOG.md

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* bump version (keikoproj#116)

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Repo selection for CI and BDD workflows & CI step for releases (keikoproj#117)

* CI-BDD not on forks & Step for releases (keikoproj#2)

* Testing CI-BDD not on forks & Step for releases

* Adding step for image with tag git-tag

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Terminate unjoined nodes (keikoproj#120)

* Validation step to check Nodes and ASG launch configs

* Validating launch definition after a rolling upgrade

* Resolve error log message and return statement

* Terminate unjoined nodes

* Resolving PR comments

Co-authored-by: Eytan Avisror <eytan.avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Set version and update CHANGELOG for version 0.14. (keikoproj#121)

Co-authored-by: Shri Javadekar <shrinand_javadekar@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Bump version to 0.15-dev.

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Fix bug when switching to launch templates (keikoproj#136)

* Update rollingupgrade_controller.go

* Update rollingupgrade_controller.go

Signed-off-by: Eytan Avisror <eytan_avisror@intuit.com>

* spacing fixes

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Extract script runner to a separate type; fix work with env. variables (keikoproj#132)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Set version and update CHANGELOG for version v0.15 (keikoproj#137)

Signed-off-by: Shri Javadekar <shrinand@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Bump version to v0.16-dev.

Signed-off-by: Shri Javadekar <shrinand@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Propagate parent env variables to allow to talk with API Server (keikoproj#144)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Bump Golang CI action to fix failed CI run (keikoproj#146)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Simplify (keikoproj#145)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Add Expiration to cache and do not refresh ASG if cache is not expired (keikoproj#143)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>

Co-authored-by: Shri Javadekar <shrinandj@users.noreply.github.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Fix documentation for uniform across AZ Update strategy and fix typos (keikoproj#147)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Move cluster state from package level to a cluster state impl (keikoproj#148)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Simplify work with intstr type. (keikoproj#149)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* If instance is in standby mode already, just return (keikoproj#138)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>

Co-authored-by: Shri Javadekar <shrinandj@users.noreply.github.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Handle terminated instances gracefully. (keikoproj#150)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Template version comparison fix (keikoproj#155)

* get template version

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>

* fix tests

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* release 0.16 (keikoproj#157)

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* bump version to 0.17-dev (keikoproj#158)

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Don't uncordon node on failure to run postDrain script when IgnoreDrainFailures set (keikoproj#151)

* Don't uncordon node on failure to run postDrain script when IgnoreDrainFailures set

Signed-off-by: Adam Malcontenti-Wilson <amalcontenti-wilson@zendesk.com>

* Test node uncordon when postDrain / postDrainWait script fails

Signed-off-by: Adam Malcontenti-Wilson <amalcontenti-wilson@zendesk.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Abort on strategy failure instead of continuing (keikoproj#152)

* Abort on strategy failure instead of continuing

Signed-off-by: Adam Malcontenti-Wilson <amalcontenti-wilson@zendesk.com>

* Remove unformatted error message placeholder

Signed-off-by: Adam Malcontenti-Wilson <amalcontenti-wilson@zendesk.com>

* Explictly specify strategy for tests

Signed-off-by: Adam Malcontenti-Wilson <amalcontenti-wilson@zendesk.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* use NamespacedName (keikoproj#160)

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>

Co-authored-by: Shri Javadekar <shrinandj@users.noreply.github.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Set version and update CHANGELOG for version v0.17 (keikoproj#161)

Signed-off-by: Shri Javadekar <shrinand@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Bump version to v0.18-dev (keikoproj#162)

Signed-off-by: Shri Javadekar <shrinand@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Move constants to types so that they can be reused (keikoproj#167)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Remove separate module for pkg/log (keikoproj#168)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>

Co-authored-by: Shri Javadekar <shrinandj@users.noreply.github.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Bump dependencies. (keikoproj#169)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* use standard fmt.Errorf to format error message; unify error format (keikoproj#171)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Fix namespaced name order (keikoproj#170)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Add instance id to the logs (keikoproj#173)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>

Co-authored-by: Shri Javadekar <shrinandj@users.noreply.github.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Bump golang and busybox (keikoproj#172)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>

Co-authored-by: Shri Javadekar <shrinandj@users.noreply.github.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Expose template list and other execution errors to logs (keikoproj#166)

* Log and return wrapped launchtemplate error

Signed-off-by: Adam Malcontenti-Wilson <amalcontenti-wilson@zendesk.com>

* Expose execution error in logs

Signed-off-by: Adam Malcontenti-Wilson <amalcontenti-wilson@zendesk.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* output can contain other messages from API Server, so be more relaxed (keikoproj#174)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Delete README.md

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* delete all

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* scaffolding

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* add API

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* initial code

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* add more scaffolding

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Add kubernetes API calls

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* aws API calls

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* AWS API calls & Drift detection

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* validate() function

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* modified validate()

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* modified validate()

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* initial rotation logic

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* basic script_runner without any modifications

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Fix all the vet related errors

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

Co-authored-by: Alfredo Garo <44888596+garomonegro@users.noreply.github.com>
Co-authored-by: Eytan Avisror <eytan.avisror@hotmail.com>
Co-authored-by: Shri Javadekar <shrinandj@users.noreply.github.com>
Co-authored-by: Shri Javadekar <shrinand_javadekar@intuit.com>
Co-authored-by: Shri Javadekar <shrinand@gmail.com>
Co-authored-by: Craig Robson <craig@zhatt.com>
Co-authored-by: Kevin Downey <kevdowney@gmail.com>
Co-authored-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Co-authored-by: Shreyas Badiger <7680410+hard-fault@users.noreply.github.com>
Co-authored-by: Adam Malcontenti-Wilson <adman.com@gmail.com>
Co-authored-by: Adam Malcontenti-Wilson <amalcontenti-wilson@zendesk.com>
Co-authored-by: Eytan Avisror <eytan_avisror@intuit.com>
* fix make vet errors.

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Terminate instances and run v2 for first time.

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Addressing review comments

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* addressing more review comments

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Log error message

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* error handling for instance tagging

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>
* Basic script runner

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>

* Update upgrade.go

Signed-off-by: Eytan Avisror <Eytan_Avisror@intuit.com>
* Eager mode implementation

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>
Signed-off-by: xshao <Sheldon_Shao@intuit.com>
* Process the batch rotation in parallel

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* addressing review comments

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>
…gUpgrade CR (keikoproj#195)

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>
* Refine the metrics status

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Refine the metrics status

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Fix test case error

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Use group instead of ASG

Signed-off-by: xshao <Sheldon_Shao@intuit.com>
* Refine the metrics status

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Refine the metrics status

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Fix test case error

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Use group instead of ASG

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Ignore generated code

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Ignore generated code

Signed-off-by: xshao <Sheldon_Shao@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>
* Unit tests

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* minor change in accessing the namespace name

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* move helper functions to a differnt file

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>
* #2285: rollup CR statistic metrics in v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2285: updated metric flags

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2285: updated metric flags

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadla1 <sahil_badla@intuit.com>
* #2285: rollup CR statistic metrics in v2 (keikoproj#218)

* #2285: rollup CR statistic metrics in v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2285: updated metric flags

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2285: updated metric flags

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* log cloud discovery failure

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Create RollingUpgrade Context

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* rollingupgrade context

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

Co-authored-by: Sahil Badla <sahil_badla@intuit.com>
* #2285: rollup CR statistic metrics in v2 (keikoproj#218)

* #2285: rollup CR statistic metrics in v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2285: updated metric flags

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2285: updated metric flags

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* log cloud discovery failure

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Create RollingUpgrade Context

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* rollingupgrade context

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* resolve compile errors due to merge conflict

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

Co-authored-by: Sahil Badla <sahil_badla@intuit.com>
* #2285: rollup CR statistic metrics in v2 (keikoproj#218)

* #2285: rollup CR statistic metrics in v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2285: updated metric flags

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2285: updated metric flags

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* log cloud discovery failure

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Create RollingUpgrade Context

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* rollingupgrade context

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* #2285: rollup CR statistic metrics in v2 (keikoproj#218)

* #2285: rollup CR statistic metrics in v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2285: updated metric flags

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2285: updated metric flags

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* #2285: renamed some methods related to metrics (keikoproj#224)

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* #2286: removed version from metric namespace (keikoproj#227)

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* resolve compile errors due to merge conflict

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* move drain-manager to reconciler

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* initialize RollingUpgrade object

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* use bool instead of count for standby function

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* refactor in-progress and standby code

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* rename instance standby function

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* DrainManager changes in unit test files

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

Co-authored-by: Sahil Badla <sahil_badla@intuit.com>
* Refine the metrics status

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Refine the metrics status

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Fix test case error

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Use group instead of ASG

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Ignore generated code

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Ignore generated code

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Fix the concurrent issue

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Fix the concurrent issue

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Move metrics related functions into RollingUpgradeContext

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Move metrics related functions into RollingUpgradeContext

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Move metrics related functions into upgrade_metrics.go

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Move metrics related functions into metrics.go

Signed-off-by: xshao <Sheldon_Shao@intuit.com>
shreyas-badiger and others added 28 commits July 21, 2021 12:41
…#276)

* upgrade-manager-v2: Fix unit tests (keikoproj#275)

* Delete README.md

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* delete all

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* scaffolding

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* add API

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* initial code

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* add more scaffolding

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Add kubernetes API calls

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* aws API calls

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* AWS API calls & Drift detection

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* initial rotation logic

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Implemented RollingUpgrade object validation. (keikoproj#176)

* Validation step to check Nodes and ASG launch configs

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>

* Validating launch definition after a rolling upgrade

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Fix all the "make vet" errors in Controller V2 branch. (keikoproj#177)

* Validation step to check Nodes and ASG launch configs

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Validating launch definition after a rolling upgrade

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Resolve error log message and return statement

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Adding Functional Test (keikoproj#113)

* Adding BDD, workflow and badge

* Changing CI workflow job name

* Adding make manifests

* Clarifying cron time zone comment

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* release 0.13 (keikoproj#115)

* release 0.13

* Update CHANGELOG.md

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* bump version (keikoproj#116)

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Repo selection for CI and BDD workflows & CI step for releases (keikoproj#117)

* CI-BDD not on forks & Step for releases (keikoproj#2)

* Testing CI-BDD not on forks & Step for releases

* Adding step for image with tag git-tag

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Terminate unjoined nodes

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Resolving PR comments

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Set version and update CHANGELOG for version 0.14. (keikoproj#121)

Co-authored-by: Shri Javadekar <shrinand_javadekar@intuit.com>
Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Bump version to 0.15-dev.

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Fix typo in README.md. (keikoproj#125)

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Ignore the terminated instance during upgrade

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Added WARNING prefix in the logging

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Apply suggestions from code review

Co-authored-by: Kevin Downey <kevdowney@gmail.com>
Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Capitalize sprintf to Sprintf

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Upgrade to Go 1.15 (keikoproj#128)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Fix few typos and simplify error returns, remove redundant types (keikoproj#131)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Readiness gates implementation for eager mode (keikoproj#130)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Adding Functional Test (keikoproj#113)

* Adding BDD, workflow and badge

* Changing CI workflow job name

* Adding make manifests

* Clarifying cron time zone comment

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Validation step to check Nodes and ASG launch configs (keikoproj#112)

* Validation step to check Nodes and ASG launch configs

* Validating launch definition after a rolling upgrade

* Resolve error log message and return statement

Co-authored-by: Eytan Avisror <eytan.avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* release 0.13 (keikoproj#115)

* release 0.13

* Update CHANGELOG.md

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* bump version (keikoproj#116)

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Repo selection for CI and BDD workflows & CI step for releases (keikoproj#117)

* CI-BDD not on forks & Step for releases (keikoproj#2)

* Testing CI-BDD not on forks & Step for releases

* Adding step for image with tag git-tag

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Terminate unjoined nodes (keikoproj#120)

* Validation step to check Nodes and ASG launch configs

* Validating launch definition after a rolling upgrade

* Resolve error log message and return statement

* Terminate unjoined nodes

* Resolving PR comments

Co-authored-by: Eytan Avisror <eytan.avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Set version and update CHANGELOG for version 0.14. (keikoproj#121)

Co-authored-by: Shri Javadekar <shrinand_javadekar@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Bump version to 0.15-dev.

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Fix bug when switching to launch templates (keikoproj#136)

* Update rollingupgrade_controller.go

* Update rollingupgrade_controller.go

Signed-off-by: Eytan Avisror <eytan_avisror@intuit.com>

* spacing fixes

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Extract script runner to a separate type; fix work with env. variables (keikoproj#132)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Set version and update CHANGELOG for version v0.15 (keikoproj#137)

Signed-off-by: Shri Javadekar <shrinand@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Bump version to v0.16-dev.

Signed-off-by: Shri Javadekar <shrinand@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Propagate parent env variables to allow to talk with API Server (keikoproj#144)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Bump Golang CI action to fix failed CI run (keikoproj#146)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Simplify (keikoproj#145)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Add Expiration to cache and do not refresh ASG if cache is not expired (keikoproj#143)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>

Co-authored-by: Shri Javadekar <shrinandj@users.noreply.github.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Fix documentation for uniform across AZ Update strategy and fix typos (keikoproj#147)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Move cluster state from package level to a cluster state impl (keikoproj#148)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Simplify work with intstr type. (keikoproj#149)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* If instance is in standby mode already, just return (keikoproj#138)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>

Co-authored-by: Shri Javadekar <shrinandj@users.noreply.github.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Handle terminated instances gracefully. (keikoproj#150)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Template version comparison fix (keikoproj#155)

* get template version

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>

* fix tests

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* release 0.16 (keikoproj#157)

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* bump version to 0.17-dev (keikoproj#158)

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Don't uncordon node on failure to run postDrain script when IgnoreDrainFailures set (keikoproj#151)

* Don't uncordon node on failure to run postDrain script when IgnoreDrainFailures set

Signed-off-by: Adam Malcontenti-Wilson <amalcontenti-wilson@zendesk.com>

* Test node uncordon when postDrain / postDrainWait script fails

Signed-off-by: Adam Malcontenti-Wilson <amalcontenti-wilson@zendesk.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Abort on strategy failure instead of continuing (keikoproj#152)

* Abort on strategy failure instead of continuing

Signed-off-by: Adam Malcontenti-Wilson <amalcontenti-wilson@zendesk.com>

* Remove unformatted error message placeholder

Signed-off-by: Adam Malcontenti-Wilson <amalcontenti-wilson@zendesk.com>

* Explictly specify strategy for tests

Signed-off-by: Adam Malcontenti-Wilson <amalcontenti-wilson@zendesk.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* use NamespacedName (keikoproj#160)

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>

Co-authored-by: Shri Javadekar <shrinandj@users.noreply.github.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Set version and update CHANGELOG for version v0.17 (keikoproj#161)

Signed-off-by: Shri Javadekar <shrinand@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Bump version to v0.18-dev (keikoproj#162)

Signed-off-by: Shri Javadekar <shrinand@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Move constants to types so that they can be reused (keikoproj#167)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Remove separate module for pkg/log (keikoproj#168)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>

Co-authored-by: Shri Javadekar <shrinandj@users.noreply.github.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Bump dependencies. (keikoproj#169)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* use standard fmt.Errorf to format error message; unify error format (keikoproj#171)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Fix namespaced name order (keikoproj#170)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Add instance id to the logs (keikoproj#173)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>

Co-authored-by: Shri Javadekar <shrinandj@users.noreply.github.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Bump golang and busybox (keikoproj#172)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>

Co-authored-by: Shri Javadekar <shrinandj@users.noreply.github.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Expose template list and other execution errors to logs (keikoproj#166)

* Log and return wrapped launchtemplate error

Signed-off-by: Adam Malcontenti-Wilson <amalcontenti-wilson@zendesk.com>

* Expose execution error in logs

Signed-off-by: Adam Malcontenti-Wilson <amalcontenti-wilson@zendesk.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* output can contain other messages from API Server, so be more relaxed (keikoproj#174)

Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Delete README.md

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* delete all

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* scaffolding

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* add API

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* initial code

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* add more scaffolding

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Add kubernetes API calls

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* aws API calls

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* AWS API calls & Drift detection

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* validate() function

Signed-off-by: shreyas-badiger <shreyasbm@outlook.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* modified validate()

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* modified validate()

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* initial rotation logic

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* basic script_runner without any modifications

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Fix all the vet related errors

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

Co-authored-by: Alfredo Garo <44888596+garomonegro@users.noreply.github.com>
Co-authored-by: Eytan Avisror <eytan.avisror@hotmail.com>
Co-authored-by: Shri Javadekar <shrinandj@users.noreply.github.com>
Co-authored-by: Shri Javadekar <shrinand_javadekar@intuit.com>
Co-authored-by: Shri Javadekar <shrinand@gmail.com>
Co-authored-by: Craig Robson <craig@zhatt.com>
Co-authored-by: Kevin Downey <kevdowney@gmail.com>
Co-authored-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Co-authored-by: Shreyas Badiger <7680410+hard-fault@users.noreply.github.com>
Co-authored-by: Adam Malcontenti-Wilson <adman.com@gmail.com>
Co-authored-by: Adam Malcontenti-Wilson <amalcontenti-wilson@zendesk.com>
Co-authored-by: Eytan Avisror <eytan_avisror@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Controller v2: Implementation of Instance termination (keikoproj#178)

* fix make vet errors.

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Terminate instances and run v2 for first time.

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Addressing review comments

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* addressing more review comments

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Log error message

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* error handling for instance tagging

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Migrate Script Runner (keikoproj#179)

* Basic script runner

Signed-off-by: Eytan Avisror <Eytan_Avisror@hotmail.com>

* Update upgrade.go

Signed-off-by: Eytan Avisror <Eytan_Avisror@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Implemented node drain. (keikoproj#181)

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Eager mode implementation (keikoproj#183)

* Eager mode implementation

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Metrics features (keikoproj#189)

Signed-off-by: xshao <Sheldon_Shao@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Process the batch rotation in parallel (keikoproj#192)

* Process the batch rotation in parallel

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* addressing review comments

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Move the DrainManager within ReplaceBatch(), to access one per RollingUpgrade CR (keikoproj#195)

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Refine metrics implementation to support goroutines (keikoproj#196)

* Refine the metrics status

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Refine the metrics status

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Fix test case error

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Use group instead of ASG

Signed-off-by: xshao <Sheldon_Shao@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Ignore generated code  (keikoproj#201)

* Refine the metrics status

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Refine the metrics status

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Fix test case error

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Use group instead of ASG

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Ignore generated code

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Ignore generated code

Signed-off-by: xshao <Sheldon_Shao@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Fix bug in deleting the entry in syncMap (keikoproj#203)

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Unit tests for controller-v2 (keikoproj#215)

* Unit tests

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* minor change in accessing the namespace name

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* move helper functions to a differnt file

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* #2285: rollup CR statistic metrics in v2 (keikoproj#218)

* #2285: rollup CR statistic metrics in v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2285: updated metric flags

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2285: updated metric flags

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* #2285: renamed some methods related to metrics (keikoproj#224)

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* #2286: removed version from metric namespace (keikoproj#227)

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Create RollingUpgradeContext (keikoproj#234)

* #2285: rollup CR statistic metrics in v2 (keikoproj#218)

* #2285: rollup CR statistic metrics in v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2285: updated metric flags

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2285: updated metric flags

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* log cloud discovery failure

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Create RollingUpgrade Context

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* rollingupgrade context

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

Co-authored-by: Sahil Badla <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Resolve compile errors caused by merge conflict. (keikoproj#235)

* #2285: rollup CR statistic metrics in v2 (keikoproj#218)

* #2285: rollup CR statistic metrics in v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2285: updated metric flags

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2285: updated metric flags

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* log cloud discovery failure

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Create RollingUpgrade Context

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* rollingupgrade context

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* resolve compile errors due to merge conflict

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

Co-authored-by: Sahil Badla <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* upgrade-manager-v2: Move DrainManager back to Reconciler (keikoproj#236)

* #2285: rollup CR statistic metrics in v2 (keikoproj#218)

* #2285: rollup CR statistic metrics in v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2285: updated metric flags

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2285: updated metric flags

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* log cloud discovery failure

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Create RollingUpgrade Context

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* rollingupgrade context

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* #2285: rollup CR statistic metrics in v2 (keikoproj#218)

* #2285: rollup CR statistic metrics in v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2285: updated metric flags

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2285: updated metric flags

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* #2285: renamed some methods related to metrics (keikoproj#224)

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* #2286: removed version from metric namespace (keikoproj#227)

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* resolve compile errors due to merge conflict

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* move drain-manager to reconciler

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* initialize RollingUpgrade object

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* use bool instead of count for standby function

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* refactor in-progress and standby code

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* rename instance standby function

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* DrainManager changes in unit test files

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

Co-authored-by: Sahil Badla <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* V2 controller metrics concurrency fix (keikoproj#231)

* Refine the metrics status

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Refine the metrics status

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Fix test case error

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Use group instead of ASG

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Ignore generated code

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Ignore generated code

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Fix the concurrent issue

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Fix the concurrent issue

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Move metrics related functions into RollingUpgradeContext

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Move metrics related functions into RollingUpgradeContext

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Move metrics related functions into upgrade_metrics.go

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Move metrics related functions into metrics.go

Signed-off-by: xshao <Sheldon_Shao@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* add missing parenthesis (keikoproj#239)

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* metricsMutex should be initialized (keikoproj#240)

Signed-off-by: xshao <Sheldon_Shao@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* upgrade-manager-v2: Load test fixes (keikoproj#245)

* upgrade-manager-v2: Move DrainManager back to Reconciler (keikoproj#236)

* #2285: rollup CR statistic metrics in v2 (keikoproj#218)

* #2285: rollup CR statistic metrics in v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2285: updated metric flags

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2285: updated metric flags

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* log cloud discovery failure

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Create RollingUpgrade Context

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* rollingupgrade context

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* #2285: rollup CR statistic metrics in v2 (keikoproj#218)

* #2285: rollup CR statistic metrics in v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2285: updated metric flags

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2285: updated metric flags

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* #2285: renamed some methods related to metrics (keikoproj#224)

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* #2286: removed version from metric namespace (keikoproj#227)

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* resolve compile errors due to merge conflict

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* move drain-manager to reconciler

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* initialize RollingUpgrade object

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* use bool instead of count for standby function

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* refactor in-progress and standby code

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* rename instance standby function

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* DrainManager changes in unit test files

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

Co-authored-by: Sahil Badla <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* V2 controller metrics concurrency fix (keikoproj#231)

* Refine the metrics status

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Refine the metrics status

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Fix test case error

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Use group instead of ASG

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Ignore generated code

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Ignore generated code

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Fix the concurrent issue

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Fix the concurrent issue

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Move metrics related functions into RollingUpgradeContext

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Move metrics related functions into RollingUpgradeContext

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Move metrics related functions into upgrade_metrics.go

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Move metrics related functions into metrics.go

Signed-off-by: xshao <Sheldon_Shao@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* add missing parenthesis

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* load test fixes

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* handle scaling group not found

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Update upgrade.go

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* log one level up

* remove double logging

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* final push before RC release. (keikoproj#254)

* support IgnoreDrainFailures flag

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* add else condition

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* set min for maxUnavailable

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* calculateMaxUnavailable function

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* add a new coloumn (completePercentage)

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* disable debug logs by default

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Fix metrics collecting issue (keikoproj#249)

* metricsMutex should be initialized

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Use InProcessingNode instead of Stringp[] so that it can have the status of steps

Signed-off-by: xshao <Sheldon_Shao@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Revert "Fix metrics collecting issue (keikoproj#249)" (keikoproj#256)

This reverts commit f5dd1cb.

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Fix metrics calculation issue (keikoproj#258)

* metricsMutex should be initialized

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Use InProcessingNode instead of Stringp[] so that it can have the status of steps

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Make the change backward compatible

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Make the change backward compatible

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Add mutex for InProcessingNode deleting

Signed-off-by: xshao <Sheldon_Shao@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Add a mock for test and update version in Makefile (keikoproj#262)

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* and CR end time (keikoproj#264)

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* upgrade-manager-v2: expose totalProcessing time and other metrics (keikoproj#265)

* and CR end time

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* expose totalProcessing time and other metrics

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* addressing review comments

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* upgrade-manager-v2: remove function duplicate declaration. (keikoproj#266)

* and CR end time

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* expose totalProcessing time and other metrics

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* addressing review comments

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* remove function duplication

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Carry the metrics status in RollingUpgrade CR (keikoproj#267)

* Update metrics status at same time

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Update metrics status when terminating instance

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Add terminated step

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Add terminated step

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Add terminated step

Signed-off-by: xshao <Sheldon_Shao@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* move cloud discovery after nodeInterval / drainInterval wait (keikoproj#270)

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* upgrade-manager-v2: Add nodeEvents handler instead of a watch handler (keikoproj#272)

* upgrade-manager-v2: remove function duplicate declaration. (keikoproj#266)

* and CR end time

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* expose totalProcessing time and other metrics

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* addressing review comments

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* remove function duplication

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Carry the metrics status in RollingUpgrade CR (keikoproj#267)

* Update metrics status at same time

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Update metrics status when terminating instance

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Add terminated step

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Add terminated step

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Add terminated step

Signed-off-by: xshao <Sheldon_Shao@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* move cloud discovery after nodeInterval / drainInterval wait

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Add watch event for cluster nodes instead of API calls

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* upon node deletion, remove it from syncMap as well

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Add nodeEvents handler instead of watch handler

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Ignore Reconciles on nodeEvents

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Add comments

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

Co-authored-by: Sheldon Shao <Sheldon_Shao@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* upgrade-manager-v2: Process next batch while waiting on nodeInterval period. (keikoproj#273)

* upgrade-manager-v2: remove function duplicate declaration. (keikoproj#266)

* and CR end time

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* expose totalProcessing time and other metrics

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* addressing review comments

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* remove function duplication

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Carry the metrics status in RollingUpgrade CR (keikoproj#267)

* Update metrics status at same time

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Update metrics status when terminating instance

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Add terminated step

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Add terminated step

Signed-off-by: xshao <Sheldon_Shao@intuit.com>

* Add terminated step

Signed-off-by: xshao <Sheldon_Shao@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* move cloud discovery after nodeInterval / drainInterval wait

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Add watch event for cluster nodes instead of API calls

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* upon node deletion, remove it from syncMap as well

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Add nodeEvents handler instead of watch handler

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Ignore Reconciles on nodeEvents

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Add comments

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Set nextbatch to standBy while waiting for terminate

* Avoid parallel reconcile operation per ASG

* add default requeue time

Co-authored-by: Sheldon Shao <Sheldon_Shao@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* fix unit tests

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

Co-authored-by: Eytan Avisror <eytan_avisror@intuit.com>
Co-authored-by: Alfredo Garo <44888596+garomonegro@users.noreply.github.com>
Co-authored-by: Eytan Avisror <eytan.avisror@hotmail.com>
Co-authored-by: Shri Javadekar <shrinandj@users.noreply.github.com>
Co-authored-by: Shri Javadekar <shrinand_javadekar@intuit.com>
Co-authored-by: Shri Javadekar <shrinand@gmail.com>
Co-authored-by: Craig Robson <craig@zhatt.com>
Co-authored-by: Kevin Downey <kevdowney@gmail.com>
Co-authored-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Co-authored-by: Shreyas Badiger <7680410+hard-fault@users.noreply.github.com>
Co-authored-by: Adam Malcontenti-Wilson <adman.com@gmail.com>
Co-authored-by: Adam Malcontenti-Wilson <amalcontenti-wilson@zendesk.com>
Co-authored-by: Sheldon Shao <shaoxt@gmail.com>
Co-authored-by: Sahil Badla <sahil_badla@intuit.com>
Co-authored-by: Sheldon Shao <Sheldon_Shao@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* add ci.yaml file

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* test commit to trigger ci build

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* move ci.yaml inside workflows

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* delete ci.yaml file from previous place

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* address lint issues

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Update ci.yaml

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* generate coverage.txt file

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* fix golang lint errors

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Delete delete-me.file

* generate coverage.txt file

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

Co-authored-by: Eytan Avisror <eytan_avisror@intuit.com>
Co-authored-by: Alfredo Garo <44888596+garomonegro@users.noreply.github.com>
Co-authored-by: Eytan Avisror <eytan.avisror@hotmail.com>
Co-authored-by: Shri Javadekar <shrinandj@users.noreply.github.com>
Co-authored-by: Shri Javadekar <shrinand_javadekar@intuit.com>
Co-authored-by: Shri Javadekar <shrinand@gmail.com>
Co-authored-by: Craig Robson <craig@zhatt.com>
Co-authored-by: Kevin Downey <kevdowney@gmail.com>
Co-authored-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
Co-authored-by: Shreyas Badiger <7680410+hard-fault@users.noreply.github.com>
Co-authored-by: Adam Malcontenti-Wilson <adman.com@gmail.com>
Co-authored-by: Adam Malcontenti-Wilson <amalcontenti-wilson@zendesk.com>
Co-authored-by: Sheldon Shao <shaoxt@gmail.com>
Co-authored-by: Sahil Badla <sahil_badla@intuit.com>
Co-authored-by: Sheldon Shao <Sheldon_Shao@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>
* #2286: removed version from metric namespace

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* update bdd for testing

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
* #2286: removed version from metric namespace

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* update bdd for testing

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
* #2286: removed version from metric namespace

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* update bdd for testing

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
* #2286: removed version from metric namespace

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* update bdd for testing

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
* #2286: removed version from metric namespace

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* update bdd for testing

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadla1 <sahil_badla@intuit.com>
* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: added makefile for test-bdd module

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: added makefile for test-bdd module

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: update rolling-update template

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: added makefile for test-bdd module

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: added makefile for test-bdd module

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: update rolling-update template

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
* #2286: removed version from metric namespace

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* update bdd for testing

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
* #2286: removed version from metric namespace

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* update bdd for testing

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
* #2286: removed version from metric namespace

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* update bdd for testing

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
* #2286: removed version from metric namespace

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* update bdd for testing

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
* #2286: removed version from metric namespace

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* update bdd for testing

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: added sh for dockerfile

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>
* fix CI job

* fix Makefile
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>
Signed-off-by: Jonah Back <jonah@jonahback.com>
* Controller v2 (keikoproj#289)

* #2286: removed version from metric namespace

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* update bdd for testing

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Controller v2 (keikoproj#290)

* #2286: removed version from metric namespace

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* update bdd for testing

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Controller v2 (keikoproj#291)

* #2286: removed version from metric namespace

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* update bdd for testing

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Controller v2 (keikoproj#292)

* #2286: removed version from metric namespace

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* update bdd for testing

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: ported bdd project from v1

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: change cron

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: bdd changes for v2

Signed-off-by: sbadla1 <sahil_badla@intuit.com>

* #2122: added sh for dockerfile

Signed-off-by: sbadla1 <sahil_badla@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* check for ASG's launch template version instead latest. (keikoproj#293)

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* controller v2: Fix ci job (keikoproj#295)

* fix CI job

* fix Makefile

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Release v1.0.1 (keikoproj#294)

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* replace launchTemplate latest string with version number (keikoproj#296)

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Release v1.0.2 (keikoproj#297)

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Add ignoreDrainFailure and DrainTimeout as controller arguements

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Sequential AWS API call for setting instances to StandBy

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Set Instances to StandBy in batches

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Set Instances to StandBy in batches

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Set Instances to StandBy in batches

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* Set Instances to StandBy in batches

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* fix: fix panic when using MixedInstancesPolicy (keikoproj#298)

Signed-off-by: Jonah Back <jonah@jonahback.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* fix vet errors

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* remove log messagE

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* change the logic

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* fix vet error

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* fix lint errors

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

* add comment

Signed-off-by: sbadiger <shreyas_badiger@intuit.com>

Co-authored-by: Sahil Badla <sahil_badla@intuit.com>
Co-authored-by: Jonah Back <jonah@jonahback.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>
Signed-off-by: sbadiger <shreyas_badiger@intuit.com>
controller flags for ignoreDrainFailures and drainTimeout (keikoproj#307)
Signed-off-by: Eytan Avisror <Eytan_Avisror@intuit.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

RollingUpgrade not consistent
5 participants