Hot Reloading: SelfHosted Component hot reloading #7239

JoshVanL · 2023-11-26T18:04:38Z

Part of #1172

Adds hot reloading functionality to Daprd, which is gated behind the HotReload preview feature.

If enabled, Daprd in Selfhosted mode will watch for resource updates from watching the resource directory(s). If a resource is detected as changed (spec is different to what is currently loaded), then Dapr will close/create/update the resource.

Supports Component except HTTP middlware which requires further changes to the HTTP server handler machinery. A warning is thrown if hot reloading is enabled and a HTTP middleware is updated.

The hot reloader reconciler is made generic to enable other resource types to be added in future.

When running in standalone mode, the disk loader watches the directories passed by --resources-path for updates to yaml files containing Components.

When an event occurs, the reconciler and differ are responsible for determining whether any resources have been created, updated, or deleted, by comparing the local store specs with that of the remote. If any have changed, the resource is closed (if it exists), and then re-initialized (if it exists). A resource will only be closed if it has been deleted, only initialized if it has been created, and closed & initialized if it has been updated.

We consider a resource to have changed generally if anything apart from its Kubernetes metadata or type meta object meta has changed, and therefore needs some kind of reloading.

Currently, if a reloading component errors and spec.ignoreErrors=false then Daprd will gracefully close, like a component loaded at startup today. It is intended that in future the component will be re-inited on a backoff queue in future in the case of errors, even if spec.ignoreErros=true.

HTTP middleware component hot reloading is not supported as it requires further changes to the HTTP server handler machinery, and I didn't want to grow this PR further.

The differ, loader, and reconciler have been made generic so that they can be reused when other resource types (HTTPEndpoint, Subscriptions etc.) can be added to hot reloading in future.

To use the HotReloading feature, which is currently only available as an alpha feature in Selfhosted mode, users need to add the following Configuration to the target Daprd;

apiVersion: dapr.io/v1alpha1
kind: Configuration
metadata:
  name: hotreloading
spec:
  features:
  - name: HotReload
    enabled: true

codecov · 2023-11-26T23:52:27Z

Codecov Report

Attention: 126 lines in your changes are missing coverage. Please review.

Comparison is base (ace5b64) 64.62% compared to head (ad265de) 64.62%.

Files	Patch %	Lines
pkg/runtime/hotreload/reconciler/component.go	0.00%	22 Missing ⚠️
pkg/runtime/processor/binding/send.go	40.62%	14 Missing and 5 partials ⚠️
pkg/runtime/runtime.go	41.37%	12 Missing and 5 partials ⚠️
pkg/runtime/hotreload/loader/fake/fake.go	15.78%	15 Missing and 1 partial ⚠️
pkg/runtime/hotreload/loader/disk/resource.go	75.00%	10 Missing and 3 partials ⚠️
pkg/runtime/processor/processor.go	20.00%	12 Missing ⚠️
pkg/runtime/hotreload/reconciler/reconciler.go	87.50%	6 Missing and 3 partials ⚠️
pkg/runtime/hotreload/loader/disk/disk.go	79.48%	5 Missing and 3 partials ⚠️
pkg/runtime/processor/binding/binding.go	56.25%	6 Missing and 1 partial ⚠️
pkg/runtime/processor/pubsub/pubsub.go	66.66%	1 Missing ⚠️
... and 2 more

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7239      +/-   ##
==========================================
- Coverage   64.62%   64.62%   -0.01%     
==========================================
  Files         226      233       +7     
  Lines       21230    21520     +290     
==========================================
+ Hits        13720    13907     +187     
- Misses       6329     6413      +84     
- Partials     1181     1200      +19

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

pkg/internal/apis/namevaluepair.go

pkg/runtime/compstore/crypto.go

pkg/apis/components/v1alpha1/register.go

pkg/apis/httpEndpoint/v1alpha1/register.go

pkg/runtime/hotreload/differ/differ.go

tests/integration/framework/process/logline/logline.go

daixiang0 · 2023-11-27T01:50:32Z

@JoshVanL thanks for your great job!

I see you close two PRs before, which make some conversations missing. Could we make changes as much as possible in one place to avoid reviewers starting over every time?

ping @ItalyPaleAle @mukundansundar @cicoyle

JoshVanL · 2023-11-27T08:25:13Z

@JoshVanL thanks for your great job!

I see you close two PRs before, which make some conversations missing. Could we make changes as much as possible in one place to avoid reviewers starting over every time?

ping @ItalyPaleAle @mukundansundar @cicoyle

Thanks @daixiang0 :)

This was done intentionally as the original hot reloading PR has diverged significantly enough (this PR is only selfhosted, doesn't include middleware, doesn't include HTTPEndpoints, has loads of integration tests) that force pushing to the same PR would cause more confusion than it was worth imo.

Branched from dapr#7239 Part of dapr#1172 Adds hot reloading functionality to Daprd, which is gated behind the HotReload preview feature. If enabled, Daprd in Kubernetes mode will watch for Component resource updates from the Operator. If a resource is detected as changed (spec is different to what is currently loaded), then Dapr will close/create/update the resource. Supports Component _except_ HTTP middlware which requires further changes to the HTTP server handler machinery. A warning is thrown if hot reloading is enabled and a HTTP middleware is updated. --- When running in Kubernetes mode, the operator loader streams updates from the Operator API. When an event occurs, the event type will trigger the resource to be created, updated, or deleted. Updated components will be closed, and then re-initialized. We consider a resource to have changed generally if anything apart from its Kubernetes metadata or type meta object meta has changed, and therefore needs some kind of reloading. --- Currently, if a reloading component errors and `spec.ignoreErrors=false` then Daprd will gracefully close, like a component loaded at startup today. It is intended that in future the component will be re-inited on a backoff queue in future in the case of errors, even if `spec.ignoreErros=true`. HTTP middleware component hot reloading is not supported as it requires further changes to the HTTP server handler machinery, and I didn't want to grow this PR further. To use the HotReloading feature, which is currently only available as an alpha feature in Selfhosted mode, users need to add the following Configuration to the target Daprd; ```yaml apiVersion: dapr.io/v1alpha1 kind: Configuration metadata: name: hotreloading spec: features: - name: HotReload enabled: true ``` Signed-off-by: joshvanl <me@joshvanl.dev>

daixiang0 · 2023-12-04T02:04:06Z

@JoshVanL thanks for your great job!
I see you close two PRs before, which make some conversations missing. Could we make changes as much as possible in one place to avoid reviewers starting over every time?
ping @ItalyPaleAle @mukundansundar @cicoyle

Thanks @daixiang0 :)

This was done intentionally as the original hot reloading PR has diverged significantly enough (this PR is only selfhosted, doesn't include middleware, doesn't include HTTPEndpoints, has loads of integration tests) that force pushing to the same PR would cause more confusion than it was worth imo.

I know, enough discuss before PRs would be better :)

Part of dapr#1172 Adds hot reloading functionality to Daprd, which is gated behind the HotReload preview feature. If enabled, Daprd in Selfhosted mode will watch for resource updates from watching the resource directory(s). If a resource is detected as changed (spec is different to what is currently loaded), then Dapr will close/create/update the resource. Supports Component _except_ HTTP middlware which requires further changes to the HTTP server handler machinery. A warning is thrown if hot reloading is enabled and a HTTP middleware is updated. The hot reloader reconciler is made generic to enable other resource types to be added in future. --- When running in standalone mode, the disk loader watches the directories passed by --resources-path for updates to yaml files containing Components. When an event occurs, the reconciler and differ are responsible for determining whether any resources have been created, updated, or deleted, by comparing the local store specs with that of the remote. If any have changed, the resource is closed (if it exists), and then re-initialized (if it exists). A resource will only be closed if it has been deleted, only initialized if it has been created, and closed & initialized if it has been updated. We consider a resource to have changed generally if anything apart from its Kubernetes metadata or type meta object meta has changed, and therefore needs some kind of reloading. --- Currently, if a reloading component errors and `spec.ignoreErrors=false` then Daprd will gracefully close, like a component loaded at startup today. It is intended that in future the component will be re-inited on a backoff queue in future in the case of errors, even if `spec.ignoreErros=true`. HTTP middleware component hot reloading is not supported as it requires further changes to the HTTP server handler machinery, and I didn't want to grow this PR further. To use the HotReloading feature, which is currently only available as an alpha feature in Selfhosted mode, users need to add the following Configuration to the target Daprd; ```yaml apiVersion: dapr.io/v1alpha1 kind: Configuration metadata: name: hotreloading spec: features: - name: HotReload enabled: true ``` Signed-off-by: joshvanl <me@joshvanl.dev>

Signed-off-by: joshvanl <me@joshvanl.dev>

each event Signed-off-by: joshvanl <me@joshvanl.dev>

Signed-off-by: joshvanl <me@joshvanl.dev>

Branched from dapr#7239 Part of dapr#1172 Adds hot reloading functionality to Daprd, which is gated behind the HotReload preview feature. If enabled, Daprd in Kubernetes mode will watch for Component resource updates from the Operator. If a resource is detected as changed (spec is different to what is currently loaded), then Dapr will close/create/update the resource. Supports Component _except_ HTTP middlware which requires further changes to the HTTP server handler machinery. A warning is thrown if hot reloading is enabled and a HTTP middleware is updated. --- When running in Kubernetes mode, the operator loader streams updates from the Operator API. When an event occurs, the event type will trigger the resource to be created, updated, or deleted. Updated components will be closed, and then re-initialized. We consider a resource to have changed generally if anything apart from its Kubernetes metadata or type meta object meta has changed, and therefore needs some kind of reloading. --- Currently, if a reloading component errors and `spec.ignoreErrors=false` then Daprd will gracefully close, like a component loaded at startup today. It is intended that in future the component will be re-inited on a backoff queue in future in the case of errors, even if `spec.ignoreErros=true`. HTTP middleware component hot reloading is not supported as it requires further changes to the HTTP server handler machinery, and I didn't want to grow this PR further. To use the HotReloading feature, which is currently only available as an alpha feature in Selfhosted mode, users need to add the following Configuration to the target Daprd; ```yaml apiVersion: dapr.io/v1alpha1 kind: Configuration metadata: name: hotreloading spec: features: - name: HotReload enabled: true ``` Signed-off-by: joshvanl <me@joshvanl.dev>

mukundansundar

first pass review

mukundansundar · 2023-12-06T13:56:31Z

pkg/internal/apis/namevaluepair.go

@@ -42,3 +48,11 @@ func (g GenericNameValueResource) GetSecretStore() string {
 func (g GenericNameValueResource) NameValuePairs() []common.NameValuePair {
 	return g.Pairs
 }
+
+func (g GenericNameValueResource) LogName() string {


should this instead implement the Stringer interface?

This is consistent with other API types https://github.com/dapr/dapr/blob/master/pkg/apis/components/v1alpha1/types.go#L45

mukundansundar · 2023-12-06T14:13:57Z

pkg/runtime/hotreload/differ/differ.go

+type Resource interface {
+	componentsapi.Component
+	meta.Resource
+}


can this type constraint be modified such that it only depends on meta.Resource?
And additionally why is GetSecretStore() part of meta.Resource?

can this type constraint be modified such that it only depends on meta.Resource?

No, this is needed for the generics constraint which we want. This will include componentsapi.Component | httpendapi.HTTPEndpoint ... in future.

And additionally why is GetSecretStore() part of meta.Resource?

It's a method on resources API. This PR didn't introduce this https://github.com/dapr/dapr/blob/master/pkg/runtime/meta/resource.go#L26C1-L26C25

mukundansundar · 2023-12-06T14:14:30Z

pkg/runtime/hotreload/differ/differ.go

+	Created []T
+}
+
+type LocalRemoteResources[T Resource] struct {


nitpick: have name as Resources

Which one? [T Resource] is singular as the type is singular, LocalRemoteResources is plural because is contains multiple Resources.

mukundansundar · 2023-12-06T14:19:54Z

pkg/runtime/hotreload/differ/differ_test.go

+	componentsapi "github.com/dapr/dapr/pkg/apis/components/v1alpha1"
+)
+
+func Test_toComparableObj(t *testing.T) {


nitpick: Please remove _ from all tests and have it as camelCase.

This is done intentionally- toComparableObj is the target function being tested which starts with a lower case letter. It is not valid to have a lower case after Test in func Testx. Changing toComparableObj to ToComparableObj would mismatch the function name. Using a _ which is valid after Test maintains the func name.

pkg/runtime/hotreload/differ/differ.go

mukundansundar · 2023-12-06T14:51:11Z

pkg/runtime/hotreload/loader/disk/disk.go

+
+	fs, err := fswatcher.New(fswatcher.Options{
+		Targets:  opts.Dirs,
+		Interval: ptr.Of(time.Millisecond * 200),


I believe this should be made configurable in the next version before hot reloading is made stable.
This can ideally be in seconds and can be upto minutes also based on the necessity.

Maybe- the use of this interval because of the way inotify works and file events on "single" write/delete events. It is meant to dedupe those specific inotify events, not to arbitrarily delay reconciles. I'm not sure what the use case would be for delaying resource updates.

https://ahmet.im/blog/kubernetes-inotify/
https://www.extrema.is/blog/2022/03/04/vim-and-inotify

pkg/runtime/hotreload/loader/disk/disk.go

pkg/runtime/hotreload/loader/disk/generic.go

mukundansundar · 2023-12-06T15:01:57Z

pkg/runtime/hotreload/loader/store/store_test.go

+	"github.com/dapr/dapr/pkg/runtime/compstore"
+)
+
+func Test_component(t *testing.T) {


camelCase for tests also ...

Same as above ^

Signed-off-by: joshvanl <me@joshvanl.dev>

Branched from dapr#7239 Part of dapr#1172 Adds hot reloading functionality to Daprd, which is gated behind the HotReload preview feature. If enabled, Daprd in Kubernetes mode will watch for Component resource updates from the Operator. If a resource is detected as changed (spec is different to what is currently loaded), then Dapr will close/create/update the resource. Supports Component _except_ HTTP middlware which requires further changes to the HTTP server handler machinery. A warning is thrown if hot reloading is enabled and a HTTP middleware is updated. --- When running in Kubernetes mode, the operator loader streams updates from the Operator API. When an event occurs, the event type will trigger the resource to be created, updated, or deleted. Updated components will be closed, and then re-initialized. We consider a resource to have changed generally if anything apart from its Kubernetes metadata or type meta object meta has changed, and therefore needs some kind of reloading. --- Currently, if a reloading component errors and `spec.ignoreErrors=false` then Daprd will gracefully close, like a component loaded at startup today. It is intended that in future the component will be re-inited on a backoff queue in future in the case of errors, even if `spec.ignoreErros=true`. HTTP middleware component hot reloading is not supported as it requires further changes to the HTTP server handler machinery, and I didn't want to grow this PR further. To use the HotReloading feature, which is currently only available as an alpha feature in Selfhosted mode, users need to add the following Configuration to the target Daprd; ```yaml apiVersion: dapr.io/v1alpha1 kind: Configuration metadata: name: hotreloading spec: features: - name: HotReload enabled: true ``` Signed-off-by: joshvanl <me@joshvanl.dev>

Signed-off-by: joshvanl <me@joshvanl.dev>

artursouza · 2023-12-14T17:13:26Z

dapr/proto/operator/v1/operator.proto

+// ResourceEventType is the type of event to a resource.
+enum ResourceEventType {
+  // UNKNOWN indicates that the event type is unknown.
+  UNKNOWN = 0;


What are the scenarios that this can happen?

It's common/best practice to have the zero enum value to be UNKNOWN https://stackoverflow.com/questions/10392952/how-to-design-for-a-future-additional-enum-value-in-protocol-buffers

Signed-off-by: joshvanl <me@joshvanl.dev>

Branched from dapr#7239 Part of dapr#1172 Adds hot reloading functionality to Daprd, which is gated behind the HotReload preview feature. If enabled, Daprd in Kubernetes mode will watch for Component resource updates from the Operator. If a resource is detected as changed (spec is different to what is currently loaded), then Dapr will close/create/update the resource. Supports Component _except_ HTTP middlware which requires further changes to the HTTP server handler machinery. A warning is thrown if hot reloading is enabled and a HTTP middleware is updated. --- When running in Kubernetes mode, the operator loader streams updates from the Operator API. When an event occurs, the event type will trigger the resource to be created, updated, or deleted. Updated components will be closed, and then re-initialized. We consider a resource to have changed generally if anything apart from its Kubernetes metadata or type meta object meta has changed, and therefore needs some kind of reloading. --- Currently, if a reloading component errors and `spec.ignoreErrors=false` then Daprd will gracefully close, like a component loaded at startup today. It is intended that in future the component will be re-inited on a backoff queue in future in the case of errors, even if `spec.ignoreErros=true`. HTTP middleware component hot reloading is not supported as it requires further changes to the HTTP server handler machinery, and I didn't want to grow this PR further. To use the HotReloading feature, which is currently only available as an alpha feature in Selfhosted mode, users need to add the following Configuration to the target Daprd; ```yaml apiVersion: dapr.io/v1alpha1 kind: Configuration metadata: name: hotreloading spec: features: - name: HotReload enabled: true ``` Signed-off-by: joshvanl <me@joshvanl.dev>

* Hot Reloading: SelfHosted Componenthot reloading Part of #1172 Adds hot reloading functionality to Daprd, which is gated behind the HotReload preview feature. If enabled, Daprd in Selfhosted mode will watch for resource updates from watching the resource directory(s). If a resource is detected as changed (spec is different to what is currently loaded), then Dapr will close/create/update the resource. Supports Component _except_ HTTP middlware which requires further changes to the HTTP server handler machinery. A warning is thrown if hot reloading is enabled and a HTTP middleware is updated. The hot reloader reconciler is made generic to enable other resource types to be added in future. --- When running in standalone mode, the disk loader watches the directories passed by --resources-path for updates to yaml files containing Components. When an event occurs, the reconciler and differ are responsible for determining whether any resources have been created, updated, or deleted, by comparing the local store specs with that of the remote. If any have changed, the resource is closed (if it exists), and then re-initialized (if it exists). A resource will only be closed if it has been deleted, only initialized if it has been created, and closed & initialized if it has been updated. We consider a resource to have changed generally if anything apart from its Kubernetes metadata or type meta object meta has changed, and therefore needs some kind of reloading. --- Currently, if a reloading component errors and `spec.ignoreErrors=false` then Daprd will gracefully close, like a component loaded at startup today. It is intended that in future the component will be re-inited on a backoff queue in future in the case of errors, even if `spec.ignoreErros=true`. HTTP middleware component hot reloading is not supported as it requires further changes to the HTTP server handler machinery, and I didn't want to grow this PR further. To use the HotReloading feature, which is currently only available as an alpha feature in Selfhosted mode, users need to add the following Configuration to the target Daprd; ```yaml apiVersion: dapr.io/v1alpha1 kind: Configuration metadata: name: hotreloading spec: features: - name: HotReload enabled: true ``` Signed-off-by: joshvanl <me@joshvanl.dev> * Adds app ready checks for subscribed input bindings and pubsub Signed-off-by: joshvanl <me@joshvanl.dev> * Linting Signed-off-by: joshvanl <me@joshvanl.dev> * Linting Signed-off-by: joshvanl <me@joshvanl.dev> * Updates reconciler to wait for processor queue to be empty after each event Signed-off-by: joshvanl <me@joshvanl.dev> * Revert processor pending resource queue to 0 channel Signed-off-by: joshvanl <me@joshvanl.dev> * Review comments Signed-off-by: joshvanl <me@joshvanl.dev> * Linitng Signed-off-by: joshvanl <me@joshvanl.dev> * Linting Signed-off-by: joshvanl <me@joshvanl.dev> * Adds review comments Signed-off-by: joshvanl <me@joshvanl.dev> * Hot Reloading: SelfHosted Componenthot reloading Part of #1172 Adds hot reloading functionality to Daprd, which is gated behind the HotReload preview feature. If enabled, Daprd in Selfhosted mode will watch for resource updates from watching the resource directory(s). If a resource is detected as changed (spec is different to what is currently loaded), then Dapr will close/create/update the resource. Supports Component _except_ HTTP middlware which requires further changes to the HTTP server handler machinery. A warning is thrown if hot reloading is enabled and a HTTP middleware is updated. The hot reloader reconciler is made generic to enable other resource types to be added in future. --- When running in standalone mode, the disk loader watches the directories passed by --resources-path for updates to yaml files containing Components. When an event occurs, the reconciler and differ are responsible for determining whether any resources have been created, updated, or deleted, by comparing the local store specs with that of the remote. If any have changed, the resource is closed (if it exists), and then re-initialized (if it exists). A resource will only be closed if it has been deleted, only initialized if it has been created, and closed & initialized if it has been updated. We consider a resource to have changed generally if anything apart from its Kubernetes metadata or type meta object meta has changed, and therefore needs some kind of reloading. --- Currently, if a reloading component errors and `spec.ignoreErrors=false` then Daprd will gracefully close, like a component loaded at startup today. It is intended that in future the component will be re-inited on a backoff queue in future in the case of errors, even if `spec.ignoreErros=true`. HTTP middleware component hot reloading is not supported as it requires further changes to the HTTP server handler machinery, and I didn't want to grow this PR further. To use the HotReloading feature, which is currently only available as an alpha feature in Selfhosted mode, users need to add the following Configuration to the target Daprd; ```yaml apiVersion: dapr.io/v1alpha1 kind: Configuration metadata: name: hotreloading spec: features: - name: HotReload enabled: true ``` Signed-off-by: joshvanl <me@joshvanl.dev> * Adds app ready checks for subscribed input bindings and pubsub Signed-off-by: joshvanl <me@joshvanl.dev> * Hot Reloading: Operator Componenthot reloading Branched from #7239 Part of #1172 Adds hot reloading functionality to Daprd, which is gated behind the HotReload preview feature. If enabled, Daprd in Kubernetes mode will watch for Component resource updates from the Operator. If a resource is detected as changed (spec is different to what is currently loaded), then Dapr will close/create/update the resource. Supports Component _except_ HTTP middlware which requires further changes to the HTTP server handler machinery. A warning is thrown if hot reloading is enabled and a HTTP middleware is updated. --- When running in Kubernetes mode, the operator loader streams updates from the Operator API. When an event occurs, the event type will trigger the resource to be created, updated, or deleted. Updated components will be closed, and then re-initialized. We consider a resource to have changed generally if anything apart from its Kubernetes metadata or type meta object meta has changed, and therefore needs some kind of reloading. --- Currently, if a reloading component errors and `spec.ignoreErrors=false` then Daprd will gracefully close, like a component loaded at startup today. It is intended that in future the component will be re-inited on a backoff queue in future in the case of errors, even if `spec.ignoreErros=true`. HTTP middleware component hot reloading is not supported as it requires further changes to the HTTP server handler machinery, and I didn't want to grow this PR further. To use the HotReloading feature, which is currently only available as an alpha feature in Selfhosted mode, users need to add the following Configuration to the target Daprd; ```yaml apiVersion: dapr.io/v1alpha1 kind: Configuration metadata: name: hotreloading spec: features: - name: HotReload enabled: true ``` Signed-off-by: joshvanl <me@joshvanl.dev> * Linting Signed-off-by: joshvanl <me@joshvanl.dev> * Adds hotreloading e2e config Signed-off-by: joshvanl <me@joshvanl.dev> * Use correct URL endpoints for hotreloading e2e save Signed-off-by: joshvanl <me@joshvanl.dev> * Revert integration test case timeout to 180 seconds Signed-off-by: joshvanl <me@joshvanl.dev> * Fix hotreloading_test.go Signed-off-by: joshvanl <me@joshvanl.dev> * Use correct external URL target Signed-off-by: joshvanl <me@joshvanl.dev> * Lint code Signed-off-by: joshvanl <me@joshvanl.dev> * Hot Reloading: SelfHosted Componenthot reloading Part of #1172 Adds hot reloading functionality to Daprd, which is gated behind the HotReload preview feature. If enabled, Daprd in Selfhosted mode will watch for resource updates from watching the resource directory(s). If a resource is detected as changed (spec is different to what is currently loaded), then Dapr will close/create/update the resource. Supports Component _except_ HTTP middlware which requires further changes to the HTTP server handler machinery. A warning is thrown if hot reloading is enabled and a HTTP middleware is updated. The hot reloader reconciler is made generic to enable other resource types to be added in future. --- When running in standalone mode, the disk loader watches the directories passed by --resources-path for updates to yaml files containing Components. When an event occurs, the reconciler and differ are responsible for determining whether any resources have been created, updated, or deleted, by comparing the local store specs with that of the remote. If any have changed, the resource is closed (if it exists), and then re-initialized (if it exists). A resource will only be closed if it has been deleted, only initialized if it has been created, and closed & initialized if it has been updated. We consider a resource to have changed generally if anything apart from its Kubernetes metadata or type meta object meta has changed, and therefore needs some kind of reloading. --- Currently, if a reloading component errors and `spec.ignoreErrors=false` then Daprd will gracefully close, like a component loaded at startup today. It is intended that in future the component will be re-inited on a backoff queue in future in the case of errors, even if `spec.ignoreErros=true`. HTTP middleware component hot reloading is not supported as it requires further changes to the HTTP server handler machinery, and I didn't want to grow this PR further. To use the HotReloading feature, which is currently only available as an alpha feature in Selfhosted mode, users need to add the following Configuration to the target Daprd; ```yaml apiVersion: dapr.io/v1alpha1 kind: Configuration metadata: name: hotreloading spec: features: - name: HotReload enabled: true ``` Signed-off-by: joshvanl <me@joshvanl.dev> * Adds app ready checks for subscribed input bindings and pubsub Signed-off-by: joshvanl <me@joshvanl.dev> * Linitng Signed-off-by: joshvanl <me@joshvanl.dev> * Linting Signed-off-by: joshvanl <me@joshvanl.dev> * Linting Signed-off-by: joshvanl <me@joshvanl.dev> * Linting Signed-off-by: joshvanl <me@joshvanl.dev> * Linting Signed-off-by: joshvanl <me@joshvanl.dev> * Review comments Signed-off-by: joshvanl <me@joshvanl.dev> * Review comments Signed-off-by: joshvanl <me@joshvanl.dev> * Set eventually in crypto test to 5 secs Signed-off-by: joshvanl <me@joshvanl.dev> --------- Signed-off-by: joshvanl <me@joshvanl.dev> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>

berndverst · 2024-02-10T00:56:01Z

This fails if the component namespace is defined, but not set to default. It even prevents the component from being loaded at all now.

JoshVanL · 2024-02-10T01:06:25Z

@berndverst is this not expected behaviour? daprd in self-hosted mode does still have the concept of namespacing (though this isn't documented well and not clear to the user). The namespace of daprd is set via the environment variable NAMESPACE which itself is set by Kubernetes so is transparent to the user but still clear through context. daprd will default the namespace to default when the env var is not set.

If it is the case that daprd would previously consume Components through file in seflhosted mode, even if the namespace set did not match the NAMESPACE env, then this can be considered a breaking change. Please let me know if this is the case and if it is a breaking change we do not want to accept and I'll add a fix.

JoshVanL requested review from a team as code owners November 26, 2023 18:04

JoshVanL mentioned this pull request Nov 26, 2023

Hot reloading of components #1172

Open

JoshVanL changed the title ~~Hot Reloading: SelfHosted Componenthot reloading~~ Hot Reloading: SelfHosted Component hot reloading Nov 26, 2023

JoshVanL added the autoupdate DaprBot will keep the Pull Request up to date with master branch label Nov 26, 2023