Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds Daprd option --dapr-block-shutdown-duration #7268

Merged
merged 8 commits into from Dec 6, 2023

Conversation

JoshVanL
Copy link
Contributor

@JoshVanL JoshVanL commented Dec 5, 2023

Closes #4313
Docs: dapr/docs#3893

PR adds the --dapr-block-shutdown-duration CLI flag and corresponding dapr.io/block-shutdown-duration Kubernetes annotation which configures Daprd to block the graceful shutdown procedure until either, the block shutdown duration has elapsed or the application has become unhealthy, as according to the normal app health status.

Accepts a Go time duration string.

By default, this option is unset, and therefore there is no effect to the current behaviour of graceful shutdown. When set, Daprd will block the interrupt signal cascading into runtime until the above requirements have been met.

The framework process Cleanup order has been reversed as a stack to mimic t.Cleanup and allow the logline process to function correctly for exec's which don't exit before Cleanup.

Closes dapr#4313
Docs: dapr/docs#3893

PR adds the `--block-shutdown-seconds` CLI flag and corresponding
`dapr.io/block-shutdown-seconds` Kubernetes annotation which configures
Daprd to block the graceful shutdown procedure until _either_, the
block shutdown seconds has elapsed _or_ the application has become
unhealthy, as according to the normal app health status.

By default, this option is unset, and therefore there is no effect to
the current behaviour of graceful shutdown. When set, Daprd will
block the interrupt signal cascading into runtime until the above
requirements have been met.

The framework process `Cleanup` order has been reversed to mimic
`t.Cleanup` and allow the `logline` process to function correctly.

Signed-off-by: joshvanl <me@joshvanl.dev>
@JoshVanL JoshVanL marked this pull request as ready for review December 5, 2023 00:00
@JoshVanL JoshVanL requested review from a team as code owners December 5, 2023 00:00
Copy link

codecov bot commented Dec 5, 2023

Codecov Report

Attention: 6 lines in your changes are missing coverage. Please review.

Comparison is base (743d120) 64.51% compared to head (918e08a) 64.56%.

Files Patch % Lines
pkg/runtime/runtime.go 85.18% 4 Missing ⚠️
cmd/daprd/options/options.go 50.00% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7268      +/-   ##
==========================================
+ Coverage   64.51%   64.56%   +0.05%     
==========================================
  Files         225      225              
  Lines       21070    21103      +33     
==========================================
+ Hits        13594    13626      +32     
+ Misses       6314     6308       -6     
- Partials     1162     1169       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

artursouza
artursouza previously approved these changes Dec 5, 2023
Signed-off-by: joshvanl <me@joshvanl.dev>
Signed-off-by: joshvanl <me@joshvanl.dev>
@@ -109,6 +112,7 @@ func New(args []string) *Options {
flag.StringVar(&opts.UnixDomainSocket, "unix-domain-socket", "", "Path to a unix domain socket dir mount. If specified, Dapr API servers will use Unix Domain Sockets")
flag.IntVar(&opts.DaprHTTPReadBufferSize, "dapr-http-read-buffer-size", runtime.DefaultReadBufferSize, "Increasing max size of read buffer in KB to handle sending multi-KB headers")
flag.IntVar(&opts.DaprGracefulShutdownSeconds, "dapr-graceful-shutdown-seconds", int(runtime.DefaultGracefulShutdownDuration/time.Second), "Graceful shutdown time in seconds")
flag.IntVar(&blockShutdownSeconds, "dapr-block-shutdown-seconds", 0, "If enabled, will block graceful shutdown after terminate signal is received until either the number of seconds has been reached or the app reports unhealthy. Disabled by default")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't use "seconds" or other things with complex units. We should standardize on Go durations which allow more flexibility (includng fractions of seconds), at least for new flags. See #7028

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the other hand, it might be more familiar for K8s users with terminationGracePeriodSeconds.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy to use a Go duration instead for the reasons in the issue. Should we move forward with that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've update to use a Go duration- easy enough to revert the commit if we want to in this PR.

artursouza
artursouza previously approved these changes Dec 5, 2023
@artursouza
Copy link
Member

@JoshVanL Please, look at the failing unit and integration tests.

@JoshVanL JoshVanL changed the title Adds Daprd option --block-shutdown-seconds Adds Daprd option --dapr-block-shutdown-duration Dec 5, 2023
@yaron2 yaron2 merged commit ed34172 into dapr:master Dec 6, 2023
19 of 22 checks passed
@ItalyPaleAle ItalyPaleAle added this to the v1.13 milestone Dec 11, 2023
DeepanshuA pushed a commit to DeepanshuA/dapr that referenced this pull request Dec 11, 2023
* Adds Daprd option `--block-shutdown-seconds`

Closes dapr#4313
Docs: dapr/docs#3893

PR adds the `--block-shutdown-seconds` CLI flag and corresponding
`dapr.io/block-shutdown-seconds` Kubernetes annotation which configures
Daprd to block the graceful shutdown procedure until _either_, the
block shutdown seconds has elapsed _or_ the application has become
unhealthy, as according to the normal app health status.

By default, this option is unset, and therefore there is no effect to
the current behaviour of graceful shutdown. When set, Daprd will
block the interrupt signal cascading into runtime until the above
requirements have been met.

The framework process `Cleanup` order has been reversed to mimic
`t.Cleanup` and allow the `logline` process to function correctly.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Revert if check on killing process exec proc cleanup

Signed-off-by: joshvanl <me@joshvanl.dev>

* Revert error ignore of processes already killed in unix

Signed-off-by: joshvanl <me@joshvanl.dev>

* Skip shutdown/graceful/block/healthy on windows.

* Skip shutdown/block/unhealthy test on windows.

* Linting

Signed-off-by: joshvanl <me@joshvanl.dev>

* Updates `dapr-block-shutdown-seconds` to `dapr-block-shutdown-duration`

Signed-off-by: joshvanl <me@joshvanl.dev>

---------

Signed-off-by: joshvanl <me@joshvanl.dev>
Co-authored-by: Loong Dai <long.dai@intel.com>
JoshVanL added a commit to JoshVanL/dapr that referenced this pull request Dec 14, 2023
* Adds Daprd option `--block-shutdown-seconds`

Closes dapr#4313
Docs: dapr/docs#3893

PR adds the `--block-shutdown-seconds` CLI flag and corresponding
`dapr.io/block-shutdown-seconds` Kubernetes annotation which configures
Daprd to block the graceful shutdown procedure until _either_, the
block shutdown seconds has elapsed _or_ the application has become
unhealthy, as according to the normal app health status.

By default, this option is unset, and therefore there is no effect to
the current behaviour of graceful shutdown. When set, Daprd will
block the interrupt signal cascading into runtime until the above
requirements have been met.

The framework process `Cleanup` order has been reversed to mimic
`t.Cleanup` and allow the `logline` process to function correctly.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Revert if check on killing process exec proc cleanup

Signed-off-by: joshvanl <me@joshvanl.dev>

* Revert error ignore of processes already killed in unix

Signed-off-by: joshvanl <me@joshvanl.dev>

* Skip shutdown/graceful/block/healthy on windows.

* Skip shutdown/block/unhealthy test on windows.

* Linting

Signed-off-by: joshvanl <me@joshvanl.dev>

* Updates `dapr-block-shutdown-seconds` to `dapr-block-shutdown-duration`

Signed-off-by: joshvanl <me@joshvanl.dev>

---------

Signed-off-by: joshvanl <me@joshvanl.dev>
Co-authored-by: Loong Dai <long.dai@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

allow annotation to specify shutdown pause
5 participants