Internal and remote actor invocation optimization #7231

ItalyPaleAle · 2023-11-22T23:49:42Z

This PR changes the data structures used for actor invocation (direct messaging, reminders, timers), optimizing them for when internal and remote actors are invoked.

When invoking internal actor such as workflow actors, with this PR we do not need to create a "virtual app channel" anymore, which added significant overhead due to data being copied and re-encoded various time, and complexity with routing. When an actor is internal (and local), it is invoked directly.
Currently, the "standard" data structure used in the actors package is pkg/messaging/v1.InvokeMethodRequest, which is optimized for invocations on the app channel. This changes the main data structure to be pkg/proto/internals/v1.InternalInvokeRequest, which is the same that is used for local and remote actor invocation. This avoids unnecessary conversions to/from the messaging data structure, which is meant for a different purpose and optimized for streaming.
Lastly, internal actors have been improved by allowing the actor runtime itself manage each instance of an active actor. This way, implementation of internal actors can save state locally (in the same object) and don't need to maintain a map of states. It simplifies the development and maintenance of internal actors, as well as improving performance by using more efficient state maps and avoid locking (based on haxmap)

All told these changes should improve performance, especially when invoking internal actors, and reduce allocations.

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

codecov · 2023-11-23T00:00:00Z

Codecov Report

Attention: 366 lines in your changes are missing coverage. Please review.

Comparison is base (aff656e) 62.30% compared to head (bb897da) 62.25%.

Files	Patch %	Lines
...runtime/wfengine/backends/actors/workflow_actor.go	0.00%	123 Missing ⚠️
pkg/actors/actors.go	49.03%	86 Missing and 20 partials ⚠️
...runtime/wfengine/backends/actors/activity_actor.go	0.00%	80 Missing ⚠️
pkg/runtime/wfengine/backends/actors/backend.go	0.00%	25 Missing ⚠️
pkg/actors/actors_mock.go	0.00%	20 Missing ⚠️
pkg/api/grpc/daprinternal.go	22.22%	7 Missing ⚠️
pkg/api/http/http.go	80.00%	0 Missing and 2 partials ⚠️
pkg/actors/internal_actor.go	87.50%	1 Missing ⚠️
pkg/messaging/v1/invoke_method_request.go	66.66%	1 Missing ⚠️
pkg/messaging/v1/invoke_method_response.go	75.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7231      +/-   ##
==========================================
- Coverage   62.30%   62.25%   -0.05%     
==========================================
  Files         240      240              
  Lines       22159    22135      -24     
==========================================
- Hits        13807    13781      -26     
- Misses       7193     7200       +7     
+ Partials     1159     1154       -5

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

…ctor-messaging Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

…ctor-messaging

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

ItalyPaleAle · 2023-12-15T21:41:38Z

/ok-to-test

dapr-bot · 2023-12-15T21:42:03Z

Dapr E2E test

🔗 Link to Action run

Commit ref: f8d0517

✅ Build succeeded for linux/amd64

Image tag: dapre2e7fb98d6332l
Test image tag: dapre2e7fb98d6332l

✅ Infrastructure deployed

Cluster	Resource group name	Azure region
Linux	`Dapr-E2E-dapre2e7fb98d6332l`	westus3
Windows	`Dapr-E2E-dapre2e7fb98d6332w`	westus3
Linux/arm64	`Dapr-E2E-dapre2e7fb98d6332la`	eastus

✅ Build succeeded for windows/amd64

Image tag: dapre2e7fb98d6332w
Test image tag: dapre2e7fb98d6332w

✅ Tests succeeded on windows/amd64

Image tag: dapre2e7fb98d6332w
Test image tag: dapre2e7fb98d6332w

✅ Tests succeeded on linux/amd64

Image tag: dapre2e7fb98d6332l
Test image tag: dapre2e7fb98d6332l

…ctor-messaging Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

dapr-bot · 2024-01-16T07:44:56Z

Dapr SDK Python test

🔗 Link to Action run

Commit ref: 853c705

✅ Python SDK tests passed

dapr-bot · 2024-01-16T07:44:56Z

Dapr SDK Java test

🔗 Link to Action run

Commit ref: 853c705

❌ Java SDK tests failed

Please check the logs for details on the error.

dapr-bot · 2024-01-16T07:44:58Z

Dapr SDK Go test

🔗 Link to Action run

Commit ref: 853c705

✅ Go SDK tests passed

dapr-bot · 2024-01-16T07:45:18Z

Dapr SDK JS test

🔗 Link to Action run

Commit ref: 853c705

✅ JS SDK tests passed

…ctor-messaging Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

pkg/api/grpc/daprinternal.go

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

daixiang0 · 2024-01-17T01:58:24Z

Please fix conflicts.

Codes looks good to me.

…leAle/dapr; branch 'master' of https://github.com/dapr/dapr into internal-actor-messaging Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

ItalyPaleAle · 2024-01-17T03:20:02Z

Please fix conflicts.

Codes looks good to me.

Fixed

daixiang0

LGTM

…ctor-messaging Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* WIP Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Bunch of fixes in tests Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * More test fixes Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * More test fixes Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * More tweaks Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * More test fixes Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Fixes and improvements to code legibility Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Use a lock around internalActors property Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * 💄 Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Fixed last test Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Changed per review feedback: use a single struct for reminders and timers Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * WIP: use factory for internal actors Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Improve APIs for internal actors Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Updated comment Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> --------- Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> Co-authored-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> Co-authored-by: Loong Dai <long.dai@intel.com>

This reverts commit e45de2c.

* WIP Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Bunch of fixes in tests Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * More test fixes Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * More test fixes Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * More tweaks Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * More test fixes Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Fixes and improvements to code legibility Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Use a lock around internalActors property Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * 💄 Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Fixed last test Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Changed per review feedback: use a single struct for reminders and timers Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * WIP: use factory for internal actors Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Improve APIs for internal actors Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Updated comment Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> --------- Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> Co-authored-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> Co-authored-by: Loong Dai <long.dai@intel.com> Signed-off-by: Elena Kolevska <elena@kolevska.com>

This reverts commit e45de2c. Signed-off-by: Elena Kolevska <elena@kolevska.com>

* WIP Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Bunch of fixes in tests Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * More test fixes Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * More test fixes Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * More tweaks Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * More test fixes Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Fixes and improvements to code legibility Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Use a lock around internalActors property Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * 💄 Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Fixed last test Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Changed per review feedback: use a single struct for reminders and timers Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * WIP: use factory for internal actors Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Improve APIs for internal actors Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Updated comment Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> --------- Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> Co-authored-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> Co-authored-by: Loong Dai <long.dai@intel.com>

ItalyPaleAle added 6 commits November 22, 2023 20:17

WIP

4196160

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Bunch of fixes in tests

d7c9ea9

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

More test fixes

6f86ee7

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

More test fixes

8caf5d2

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

More tweaks

356b61d

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

More test fixes

fa3f445

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

ItalyPaleAle added 7 commits November 22, 2023 16:00

Merge branch 'master' into internal-actor-messaging

03b81d8

Merge branch 'master' of https://github.com/dapr/dapr into internal-a…

af0cefc

…ctor-messaging Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Merge branch 'master' of https://github.com/dapr/dapr into internal-a…

aba3836

…ctor-messaging

Fixes and improvements to code legibility

cfa984b

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Use a lock around internalActors property

ca3a82f

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

💄

96056ee

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Fixed last test

f8d0517

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

ItalyPaleAle changed the title ~~WIP: Internal and remote actor invocation optimization~~ Internal and remote actor invocation optimization Dec 15, 2023

ItalyPaleAle marked this pull request as ready for review December 15, 2023 02:31

ItalyPaleAle requested review from a team as code owners December 15, 2023 02:31

ItalyPaleAle added the autoupdate DaprBot will keep the Pull Request up to date with master branch label Dec 15, 2023

dapr-bot added 3 commits December 16, 2023 14:25

Merge branch 'master' into internal-actor-messaging

e493c62

Merge branch 'master' into internal-actor-messaging

435c8ea

Merge branch 'master' into internal-actor-messaging

94fae02

ItalyPaleAle mentioned this pull request Dec 19, 2023

Workflow to Stable - Phase1 #7156

Closed

13 tasks

ItalyPaleAle added this to the v1.13 milestone Dec 19, 2023

ItalyPaleAle and others added 4 commits December 20, 2023 05:07

Merge branch 'master' of https://github.com/dapr/dapr into internal-a…

7444140

…ctor-messaging Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Merge branch 'master' into internal-actor-messaging

7538d73

Merge branch 'master' into internal-actor-messaging

8a2da40

Merge branch 'master' into internal-actor-messaging

c6a43b9

mukundansundar and others added 3 commits January 15, 2024 23:48

Merge branch 'master' into internal-actor-messaging

97b34b9

Merge branch 'master' of https://github.com/dapr/dapr into internal-a…

3289e4d

…ctor-messaging Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Merge branch 'master' into internal-actor-messaging

24f38f5

yaron2 reviewed Jan 16, 2024

View reviewed changes

pkg/api/grpc/daprinternal.go Show resolved Hide resolved

ItalyPaleAle and others added 2 commits January 16, 2024 18:08

Updated comment

a9d0f6b

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Merge branch 'master' into internal-actor-messaging

e72d853

Merge branch 'internal-actor-messaging' of https://github.com/ItalyPa…

59f89e6

…leAle/dapr; branch 'master' of https://github.com/dapr/dapr into internal-actor-messaging Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

daixiang0 previously approved these changes Jan 17, 2024

View reviewed changes

daixiang0 and others added 3 commits January 17, 2024 14:14

Merge branch 'master' into internal-actor-messaging

f1be3dd

Merge branch 'master' into internal-actor-messaging

58ec752

Merge branch 'master' of https://github.com/dapr/dapr into internal-a…

4f02def

…ctor-messaging Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

ItalyPaleAle dismissed daixiang0’s stale review via 4f02def January 17, 2024 16:06

Merge branch 'master' into internal-actor-messaging

1894652

yaron2 approved these changes Jan 17, 2024

View reviewed changes

dapr-bot added 2 commits January 17, 2024 17:41

Merge branch 'master' into internal-actor-messaging

ee4744e

Merge branch 'master' into internal-actor-messaging

bb897da

yaron2 merged commit db53617 into dapr:master Jan 18, 2024
20 of 22 checks passed

elena-kolevska added a commit to elena-kolevska/dapr that referenced this pull request Jan 24, 2024

Revert "Internal and remote actor invocation optimization (dapr#7231)"

e9208fe

This reverts commit e45de2c.

elena-kolevska added a commit to elena-kolevska/dapr that referenced this pull request Jan 25, 2024

Revert "Internal and remote actor invocation optimization (dapr#7231)"

33582e0

This reverts commit e45de2c. Signed-off-by: Elena Kolevska <elena@kolevska.com>

ItalyPaleAle deleted the internal-actor-messaging branch February 26, 2024 19:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Internal and remote actor invocation optimization #7231

Internal and remote actor invocation optimization #7231

ItalyPaleAle commented Nov 22, 2023 •

edited

codecov bot commented Nov 23, 2023 •

edited

ItalyPaleAle commented Dec 15, 2023

dapr-bot commented Dec 15, 2023 •

edited

dapr-bot commented Jan 16, 2024 •

edited

dapr-bot commented Jan 16, 2024 •

edited

dapr-bot commented Jan 16, 2024 •

edited

dapr-bot commented Jan 16, 2024 •

edited

daixiang0 commented Jan 17, 2024

ItalyPaleAle commented Jan 17, 2024

daixiang0 left a comment

Internal and remote actor invocation optimization #7231

Internal and remote actor invocation optimization #7231

Conversation

ItalyPaleAle commented Nov 22, 2023 • edited

codecov bot commented Nov 23, 2023 • edited

Codecov Report

ItalyPaleAle commented Dec 15, 2023

dapr-bot commented Dec 15, 2023 • edited

Dapr E2E test

✅ Build succeeded for linux/amd64

✅ Infrastructure deployed

✅ Build succeeded for windows/amd64

✅ Tests succeeded on windows/amd64

✅ Tests succeeded on linux/amd64

dapr-bot commented Jan 16, 2024 • edited

Dapr SDK Python test

✅ Python SDK tests passed

dapr-bot commented Jan 16, 2024 • edited

Dapr SDK Java test

❌ Java SDK tests failed

dapr-bot commented Jan 16, 2024 • edited

Dapr SDK Go test

✅ Go SDK tests passed

dapr-bot commented Jan 16, 2024 • edited

Dapr SDK JS test

✅ JS SDK tests passed

daixiang0 commented Jan 17, 2024

ItalyPaleAle commented Jan 17, 2024

daixiang0 left a comment

Choose a reason for hiding this comment

ItalyPaleAle commented Nov 22, 2023 •

edited

codecov bot commented Nov 23, 2023 •

edited

dapr-bot commented Dec 15, 2023 •

edited

dapr-bot commented Jan 16, 2024 •

edited

dapr-bot commented Jan 16, 2024 •

edited

dapr-bot commented Jan 16, 2024 •

edited

dapr-bot commented Jan 16, 2024 •

edited