Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hot Reloading: SelfHosted Component hot reloading #7239

Merged
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
9d21f45
Hot Reloading: SelfHosted Componenthot reloading
JoshVanL Nov 18, 2023
7cdbbc2
Adds app ready checks for subscribed input bindings and pubsub
JoshVanL Nov 26, 2023
0157b7f
Linting
JoshVanL Nov 26, 2023
a208b8a
Linting
JoshVanL Nov 26, 2023
9f9d71d
Linting
JoshVanL Nov 26, 2023
d0dd3a3
Linting
JoshVanL Nov 26, 2023
c1d7d1b
Linting
JoshVanL Nov 26, 2023
cf97305
Updates reconciler to wait for processor queue to be empty after
JoshVanL Nov 26, 2023
ce16f37
Revert processor pending resource queue to 0 channel
JoshVanL Nov 26, 2023
0132a49
Speed up hotreloading integration tests
JoshVanL Nov 26, 2023
bd83f7f
Review comments
JoshVanL Nov 27, 2023
43e053a
Move `tempDir` hot reload state test `TempDir()` to outer test scope to
JoshVanL Nov 27, 2023
e12fed3
Linitng
JoshVanL Dec 4, 2023
cf453bc
Linting
JoshVanL Dec 6, 2023
8607abb
Adds review comments
JoshVanL Dec 6, 2023
4c0384f
Merge branch 'master' into hot-reloading-components-selfhosted-tested
dapr-bot Dec 11, 2023
1e8c130
Merge branch 'master' into hot-reloading-components-selfhosted-tested
dapr-bot Dec 12, 2023
3e89b02
Merge branch 'master' into hot-reloading-components-selfhosted-tested
dapr-bot Dec 13, 2023
4bded36
Fix order of close and check bool in runtime_test.go
JoshVanL Dec 15, 2023
3520e01
Merge branch 'master' into hot-reloading-components-selfhosted-tested
dapr-bot Dec 16, 2023
b5a301d
Merge branch 'master' into hot-reloading-components-selfhosted-tested
dapr-bot Dec 19, 2023
ce276d6
Merge branch 'master' into hot-reloading-components-selfhosted-tested
dapr-bot Dec 19, 2023
f399ece
Merge branch 'master' into hot-reloading-components-selfhosted-tested
dapr-bot Dec 20, 2023
d9536e0
Merge branch 'master' into hot-reloading-components-selfhosted-tested
dapr-bot Dec 20, 2023
3ce38c1
Merge branch 'master' into hot-reloading-components-selfhosted-tested
dapr-bot Dec 20, 2023
5917eca
Merge branch 'master' into hot-reloading-components-selfhosted-tested
dapr-bot Dec 20, 2023
022a3df
Merge branch 'master' into hot-reloading-components-selfhosted-tested
dapr-bot Dec 20, 2023
48ccdc0
Merge branch 'master' into hot-reloading-components-selfhosted-tested
dapr-bot Dec 21, 2023
ad265de
Updates state features in integration tests
JoshVanL Dec 22, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
20 changes: 19 additions & 1 deletion dapr/proto/operator/v1/operator.proto
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,21 @@ service Operator {
rpc HTTPEndpointUpdate (HTTPEndpointUpdateRequest) returns (stream HTTPEndpointUpdateEvent) {}
}

// ResourceEventType is the type of event to a resource.
enum ResourceEventType {
// UNKNOWN indicates that the event type is unknown.
UNKNOWN = 0;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are the scenarios that this can happen?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


// CREATED indicates that the resource has been created.
CREATED = 1;

// UPDATED indicates that the resource has been updated.
UPDATED = 2;

// DELETED indicates that the resource has been deleted.
DELETED = 3;
}

// ListComponentsRequest is the request to get components for a sidecar in namespace.
message ListComponentsRequest {
string namespace = 1;
Expand All @@ -55,6 +70,9 @@ message ComponentUpdateRequest {
// ComponentUpdateEvent includes the updated component event.
message ComponentUpdateEvent {
bytes component = 1;

// type is the type of event.
ResourceEventType type = 2;
}

// ListComponentResponse includes the list of available components.
Expand Down Expand Up @@ -134,4 +152,4 @@ message HTTPEndpointUpdateRequest {
// HTTPEndpointsUpdateEvent includes the updated http endpoint event.
message HTTPEndpointUpdateEvent {
bytes http_endpoints = 1;
}
}
4 changes: 2 additions & 2 deletions pkg/apis/components/v1alpha1/register.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ import (
// SchemeGroupVersion is group version used to register these objects.
var SchemeGroupVersion = schema.GroupVersion{Group: components.GroupName, Version: "v1alpha1"}

// Kind takes an unqualified kind and returns back a Group qualified GroupKind.
func Kind(kind string) schema.GroupKind {
// GroupKind takes an unqualified kind and returns back a Group qualified GroupKind.
func GroupKind(kind string) schema.GroupKind {
return SchemeGroupVersion.WithKind(kind).GroupKind()
}

Expand Down
28 changes: 28 additions & 0 deletions pkg/apis/components/v1alpha1/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,15 @@ import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"

"github.com/dapr/dapr/pkg/apis/common"
"github.com/dapr/dapr/pkg/apis/components"
"github.com/dapr/dapr/utils"
)

const (
Kind = "Component"
Version = "v1alpha1"
)

//+genclient
//+genclient:noStatus
//+kubebuilder:object:root=true
Expand All @@ -41,6 +47,16 @@ func (Component) Kind() string {
return "Component"
}

// GetName returns the component name.
func (c Component) GetName() string {
return c.Name
}

// GetNamespace returns the component namespace.
func (c Component) GetNamespace() string {
return c.Namespace
}

// LogName returns the name of the component that can be used in logging.
func (c Component) LogName() string {
return utils.ComponentLogName(c.ObjectMeta.Name, c.Spec.Type, c.Spec.Version)
Expand All @@ -56,6 +72,18 @@ func (c Component) NameValuePairs() []common.NameValuePair {
return c.Spec.Metadata
}

// EmptyMetaDeepCopy returns a new instance of the component type with the
// TypeMeta's Kind and APIVersion fields set.
func (c Component) EmptyMetaDeepCopy() metav1.Object {
n := c.DeepCopy()
n.TypeMeta = metav1.TypeMeta{
Kind: Kind,
APIVersion: components.GroupName + "/" + Version,
}
n.ObjectMeta = metav1.ObjectMeta{Name: c.Name}
return n
}

// ComponentSpec is the spec for a component.
type ComponentSpec struct {
Type string `json:"type"`
Expand Down
5 changes: 3 additions & 2 deletions pkg/apis/httpEndpoint/v1alpha1/register.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,9 @@ import (
// SchemeGroupVersion is group version used to register these objects.
var SchemeGroupVersion = schema.GroupVersion{Group: httpendpoint.GroupName, Version: "v1alpha1"}

// Kind takes an unqualified kind and returns back a Group qualified GroupKind.
func Kind(kind string) schema.GroupKind {
// GroupKind takes an unqualified kind and returns back a Group
// qualified GroupKind.
func GroupKind(kind string) schema.GroupKind {
return SchemeGroupVersion.WithKind(kind).GroupKind()
}

Expand Down
33 changes: 33 additions & 0 deletions pkg/apis/httpEndpoint/v1alpha1/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,12 @@ import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"

"github.com/dapr/dapr/pkg/apis/common"
httpendpoint "github.com/dapr/dapr/pkg/apis/httpEndpoint"
)

const (
Kind = "HTTPEndpoint"
Version = "v1alpha1"
)

//+genclient
Expand All @@ -43,11 +49,26 @@ func (HTTPEndpoint) Kind() string {
return kind
}

// GetName returns the component name.
func (h HTTPEndpoint) GetName() string {
return h.Name
}

// GetNamespace returns the component namespace.
func (h HTTPEndpoint) GetNamespace() string {
return h.Namespace
}

// GetSecretStore returns the name of the secret store.
func (h HTTPEndpoint) GetSecretStore() string {
return h.Auth.SecretStore
}

// LogName returns the name of the component that can be used in logging.
func (h HTTPEndpoint) LogName() string {
return h.Name + " (" + h.Spec.BaseURL + ")"
}

// NameValuePairs returns the component's headers as name/value pairs
func (h HTTPEndpoint) NameValuePairs() []common.NameValuePair {
return h.Spec.Headers
Expand Down Expand Up @@ -83,6 +104,18 @@ func (h HTTPEndpoint) HasTLSPrivateKey() bool {
return h.Spec.ClientTLS != nil && h.Spec.ClientTLS.PrivateKey != nil && h.Spec.ClientTLS.PrivateKey.Value != nil
}

// EmptyMetaDeepCopy returns a new instance of the component type with the
// TypeMeta's Kind and APIVersion fields set.
func (h HTTPEndpoint) EmptyMetaDeepCopy() metav1.Object {
n := h.DeepCopy()
n.TypeMeta = metav1.TypeMeta{
Kind: Kind,
APIVersion: httpendpoint.GroupName + "/" + Version,
}
n.ObjectMeta = metav1.ObjectMeta{Name: h.Name}
return n
}

// HTTPEndpointSpec describes an access specification for allowing external service invocations.
type HTTPEndpointSpec struct {
BaseURL string `json:"baseUrl" validate:"required"`
Expand Down
2 changes: 2 additions & 0 deletions pkg/config/configuration.go
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@ const (
// Enables support for setting TTL on Actor state keys. Remove this flag in
// Dapr 1.12.
ActorStateTTL Feature = "ActorStateTTL"
// Enables support for hot reloading of Daprd Components and HTTPEndpoints.
HotReload Feature = "HotReload"
)

// end feature flags section
Expand Down
16 changes: 15 additions & 1 deletion pkg/internal/apis/namevaluepair.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,13 @@ limitations under the License.

package apis

import "github.com/dapr/dapr/pkg/apis/common"
import (
"fmt"

metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"

"github.com/dapr/dapr/pkg/apis/common"
)

type GenericNameValueResource struct {
Name string
Expand Down Expand Up @@ -42,3 +48,11 @@ func (g GenericNameValueResource) GetSecretStore() string {
func (g GenericNameValueResource) NameValuePairs() []common.NameValuePair {
return g.Pairs
}

func (g GenericNameValueResource) LogName() string {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this instead implement the Stringer interface?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return fmt.Sprintf("%s (%s)", g.Name, g.ResourceKind)
}

func (g GenericNameValueResource) EmptyMetaDeepCopy() metav1.Object {
return &metav1.ObjectMeta{Name: g.Name}
}