Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Defining Workflow metrics #7152

Merged
merged 47 commits into from Jan 11, 2024
Merged

Conversation

prateek041
Copy link
Contributor

Description

This pull request adds workflow metrics.

Issue reference

Will fix #7109

Checklist

Please make sure you've completed the relevant tasks for this PR, out of the following list:

Signed-off-by: prateek041 <prateeksingh9741@gmail.com>
@prateek041
Copy link
Contributor Author

I have mentioned a few [Questions] to solidify my understanding of the current code. The idea is to have a proper understanding of workflow engine and workflows and eventually write docs for future contributors.

cc @shubham1172

pkg/runtime/wfengine/workflow.go Show resolved Hide resolved
pkg/runtime/wfengine/workflow.go Outdated Show resolved Hide resolved
pkg/runtime/wfengine/workflow.go Outdated Show resolved Hide resolved
pkg/runtime/wfengine/workflow.go Outdated Show resolved Hide resolved
pkg/runtime/wfengine/workflow.go Outdated Show resolved Hide resolved
pkg/runtime/wfengine/workflow.go Outdated Show resolved Hide resolved
@shivamkm07
Copy link
Contributor

shivamkm07 commented Nov 8, 2023

I have mentioned a few [Questions] to solidify my understanding of the current code. The idea is to have a proper understanding of workflow engine and workflows and eventually write docs for future contributors.

cc @shubham1172

Replied over the Questions.
Feel free to ping me on Discord (same Id as github) and we can chat if you have any more queries!
And yes, Thanks for the contribution. Really appreciated!

@prateek041
Copy link
Contributor Author

Thank you for being patient with me @shivamkm07.

Sure, for any more conceptual doubts, I will ping you on discord.

- Operations
- Reminders
- Execution

Signed-off-by: prateek041 <prateeksingh9741@gmail.com>
Signed-off-by: prateek041 <prateeksingh9741@gmail.com>
- Create Workflow
- Get Workflow
- Purge Workflow
- Add Events

Signed-off-by: prateek041 <prateeksingh9741@gmail.com>
Signed-off-by: prateek041 <prateeksingh9741@gmail.com>
executions.

Signed-off-by: prateek041 <prateeksingh9741@gmail.com>
@prateek041
Copy link
Contributor Author

@shivamkm07 please review.

Also, there are many needless comments which I added for extra context, will remove them when the PR is ready.

pkg/diagnostics/metrics.go Outdated Show resolved Hide resolved
pkg/diagnostics/workflow_metrics.go Outdated Show resolved Hide resolved
pkg/diagnostics/workflow_metrics.go Outdated Show resolved Hide resolved
pkg/diagnostics/workflow_metrics.go Outdated Show resolved Hide resolved
pkg/diagnostics/workflow_metrics.go Outdated Show resolved Hide resolved
pkg/diagnostics/workflow_metrics.go Outdated Show resolved Hide resolved
pkg/runtime/wfengine/activity.go Outdated Show resolved Hide resolved
pkg/runtime/wfengine/activity.go Outdated Show resolved Hide resolved
pkg/runtime/wfengine/backend.go Outdated Show resolved Hide resolved
pkg/runtime/wfengine/backend.go Outdated Show resolved Hide resolved
Signed-off-by: prateek041 <prateeksingh9741@gmail.com>
Signed-off-by: prateek041 <prateeksingh9741@gmail.com>
Signed-off-by: prateek041 <prateeksingh9741@gmail.com>
@prateek041 prateek041 marked this pull request as ready for review December 5, 2023 20:19
@prateek041 prateek041 requested review from a team as code owners December 5, 2023 20:19
Signed-off-by: prateek041 <prateeksingh9741@gmail.com>
go.mod Outdated Show resolved Hide resolved
@RyanLettieri
Copy link
Contributor

Can we include testing for these new metrics under this PR?

Copy link
Contributor

@DeepanshuA DeepanshuA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, for these comments at this time of this PR; but had just noticed these points.

diagUtils.NewMeasureView(w.workflowOperationCount, []tag.Key{appIDKey, componentKey, namespaceKey, operationKey, statusKey}, view.Count()),
diagUtils.NewMeasureView(w.workflowOperationLatency, []tag.Key{appIDKey, componentKey, namespaceKey, operationKey, statusKey}, defaultLatencyDistribution),
diagUtils.NewMeasureView(w.workflowExecutionCount, []tag.Key{appIDKey, componentKey, namespaceKey, executionTypeKey, statusKey}, view.Count()),
diagUtils.NewMeasureView(w.workflowExecutionLatency, []tag.Key{appIDKey, componentKey, namespaceKey, executionTypeKey, statusKey}, defaultLatencyDistribution))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should have workflowName and activityName as well in this metric; else a common plotting won't make much sense - as different activities mayhave different expected latencies.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

"The number of successful/failed/recoverable workflow/activity executions.",
stats.UnitDimensionless),
workflowExecutionLatency: stats.Float64(
"runtime/workflow/execution/latency",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should it be runtime/workflow/execution/latency and runtime/workflow/activity/execution/latency - differentiation would be there - and cardinality can also be reduced by removing executionTypeKey.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similarly, for workflow/execution/count metric as well

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good to me.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

shivamkm07 and others added 4 commits January 5, 2024 13:41
Co-authored-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com>
Signed-off-by: Shivam Kumar <shivamkm07@gmail.com>
Signed-off-by: Shivam Kumar <shivamkm07@gmail.com>
Signed-off-by: Shivam Kumar <shivamkm07@gmail.com>
Copy link

codecov bot commented Jan 8, 2024

Codecov Report

Attention: 45 lines in your changes are missing coverage. Please review.

Comparison is base (16c0dd5) 64.42% compared to head (0907987) 64.48%.

❗ Current head 0907987 differs from pull request most recent head 32aab9d. Consider uploading reports for the commit 32aab9d to get more accurate results

Files Patch % Lines
pkg/runtime/wfengine/workflow.go 43.75% 19 Missing and 8 partials ⚠️
pkg/runtime/wfengine/activity.go 66.66% 5 Missing and 2 partials ⚠️
pkg/diagnostics/workflow_monitoring.go 86.36% 3 Missing and 3 partials ⚠️
pkg/runtime/wfengine/backend.go 81.25% 3 Missing ⚠️
pkg/diagnostics/metrics.go 0.00% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7152      +/-   ##
==========================================
+ Coverage   64.42%   64.48%   +0.05%     
==========================================
  Files         237      238       +1     
  Lines       21704    21814     +110     
==========================================
+ Hits        13983    14066      +83     
- Misses       6521     6536      +15     
- Partials     1200     1212      +12     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mukundansundar
Copy link
Contributor

/ok-to-test

@mukundansundar
Copy link
Contributor

/test-version-skew

@dapr-bot
Copy link
Collaborator

dapr-bot commented Jan 11, 2024

Dapr E2E test

🔗 Link to Action run

Commit ref: 0907987

✅ Build succeeded for linux/amd64

  • Image tag: dapre2e6564fef23bl
  • Test image tag: dapre2e6564fef23bl

✅ Infrastructure deployed

Cluster Resource group name Azure region
Linux Dapr-E2E-dapre2e6564fef23bl westus3
Windows Dapr-E2E-dapre2e6564fef23bw westus3
Linux/arm64 Dapr-E2E-dapre2e6564fef23bla eastus

✅ Build succeeded for windows/amd64

  • Image tag: dapre2e6564fef23bw
  • Test image tag: dapre2e6564fef23bw

❌ Tests failed on windows/amd64

Please check the logs for details on the error.

❌ Tests failed on linux/amd64

Please check the logs for details on the error.

@dapr-bot
Copy link
Collaborator

dapr-bot commented Jan 11, 2024

Dapr Version Skew e2e test (dapr-sidecar-master - 1.12.3)

🔗 Link to Action run

Commit ref: 0907987

✅ Version Skew tests passed

@dapr-bot
Copy link
Collaborator

dapr-bot commented Jan 11, 2024

Dapr Version Skew e2e test (control-plane-master - 1.12.3)

🔗 Link to Action run

Commit ref: 0907987

✅ Version Skew tests passed

@dapr-bot
Copy link
Collaborator

dapr-bot commented Jan 11, 2024

Dapr Version Skew integration test (dapr-sidecar-master - ull)

🔗 Link to Action run

Commit ref: 0907987

❌ Version Skew tests failed

Please check the logs for details on the error.

@dapr-bot
Copy link
Collaborator

dapr-bot commented Jan 11, 2024

Dapr Version Skew integration test (control-plane-master - 1.12.3)

🔗 Link to Action run

Commit ref: 0907987

❌ Version Skew tests failed

Please check the logs for details on the error.

* dapr_runtime_workflow_operation_count: The number of successful/failed workflow operation requests.
* dapr_runtime_workflow_operation_latency: The latencies of responses for workflow operation requests.
* dapr_runtime_workflow_execution_count: The number of successful/failed/recoverable workflow/activity executions.
* dapr_runtime_workflow_execution_latency: The total time taken to run a workflow/activity to completion.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: change this metric name

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Signed-off-by: Shivam Kumar <shivamkm07@gmail.com>
mukundansundar
mukundansundar previously approved these changes Jan 11, 2024
Signed-off-by: Shivam Kumar <shivamkm07@gmail.com>
@mukundansundar mukundansundar merged commit ee813d1 into dapr:master Jan 11, 2024
17 of 20 checks passed
@dapr-bot

This comment has been minimized.

@dapr-bot
Copy link
Collaborator

❌ Version Skew tests failed

Please check the logs for details on the error.

@JoshVanL JoshVanL added this to the v1.13 milestone Feb 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Workflow] Adding metrics for Dapr Workflow
9 participants