Configure alternate backend for Dapr workflow

[ASHIQUEMD]

Description

Allow users to configure alternate backend for Dapr Workflow apart from Actor.

Currently supported workflow backends

1. Actor
2. Sqlite

This PR refactors wfengine.go file to be independent of Actor backend.

SQLite backend component example yaml

apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: sqlitebackend
spec:
  type: workflowbackend.sqlite
  version: v1
  metadata:
    - name: filePath
      value: "file::memory:"
    - name: orchestrationLockTimeout
      value: "130000000000ms"
    - name: activityLockTimeout
      value: "130000000000ms"

Actor backend component example yaml.

apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: actorbackend
spec:
  type: workflowbackend.actor
  version: v1

Note: If there is no backend component defined, actor backend will be initialized by default, so its not required for user to specify the actor backend component.

Distinction between the workflow engine component and workflow backend component in context of workflow:

workflow.dapr (Workflow Engine component):

This type represents the workflow engine component.
It is initialized by default without the need for a separate component YAML file.

workflowbackend.sqlite or workflowbackend.actor (Workflow Backend component):

This type represents the workflow backend component.
Initialization is based on the component YAML file provided by the user.
Users can configure and tailor the backend according to their specific requirements.

It's crucial to note that these two types are mutually exclusive and operate independently of each other. The growth and introduction of new components within each category will be handled independently

Issue reference

Please reference the issue this PR will close: #7127

Checklist

Please make sure you've completed the relevant tasks for this PR, out of the following list:

• Code compiles correctly
• Created/updated tests
• Unit tests passing
• End-to-end tests passing
• Extended the documentation / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[3924]
• Specification has been updated / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[3924]
• Provided sample for the feature / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[3924]

[cgillum]

I haven't had a chance to do a full review yet, but from what I've skimmed so far, the approach you're taking matches my expectations, which is great to see. :)

There are a few minor things I was able to spot while looking quickly at the code.

[dapr-bot]

Dapr Version Skew integration test (dapr-sidecar-master - 1.12.3)

🔗 Link to Action run

Commit ref: 312d92d

❌ Version Skew tests failed

Please check the logs for details on the error.

[dapr-bot]

Dapr Version Skew e2e test (control-plane-master - 1.12.3)

🔗 Link to Action run

Commit ref: 312d92d

✅ Version Skew tests passed

[dapr-bot]

Dapr Version Skew integration test (control-plane-master - 1.12.3)

🔗 Link to Action run

Commit ref: 312d92d

❌ Version Skew tests failed

Please check the logs for details on the error.

[ASHIQUEMD]

/ok-to-test

[dapr-bot]

Dapr E2E test

🔗 Link to Action run

Commit ref: 3a3fb33

✅ Build succeeded for linux/amd64

• Image tag: dapre2e15479fc0e5l
• Test image tag: dapre2e15479fc0e5l

✅ Infrastructure deployed

Cluster	Resource group name	Azure region
Linux	Dapr-E2E-dapre2e15479fc0e5l	westus3
Windows	Dapr-E2E-dapre2e15479fc0e5w	westus3
Linux/arm64	Dapr-E2E-dapre2e15479fc0e5la	eastus

✅ Build succeeded for windows/amd64

• Image tag: dapre2e15479fc0e5w
• Test image tag: dapre2e15479fc0e5w

❌ Tests failed on windows/amd64

Please check the logs for details on the error.

❌ Tests failed on linux/amd64

Please check the logs for details on the error.

[ASHIQUEMD]

/ok-to-test

[dapr-bot]

Dapr E2E test

🔗 Link to Action run

Commit ref: da5607f

✅ Build succeeded for linux/amd64

• Image tag: dapre2e72f344586fl
• Test image tag: dapre2e72f344586fl

✅ Infrastructure deployed

Cluster	Resource group name	Azure region
Linux	Dapr-E2E-dapre2e72f344586fl	westus3
Windows	Dapr-E2E-dapre2e72f344586fw	westus3
Linux/arm64	Dapr-E2E-dapre2e72f344586fla	eastus

✅ Build succeeded for windows/amd64

• Image tag: dapre2e72f344586fw
• Test image tag: dapre2e72f344586fw

❌ Tests failed on windows/amd64

Please check the logs for details on the error.

❌ Tests failed on linux/amd64

Please check the logs for details on the error.

[ASHIQUEMD]

/ok-to-perf

[ASHIQUEMD]

/test-version-skew

[dapr-bot]

Dapr perf test

🔗 Link to Action run

Commit ref: da5607f

✅ Infrastructure deployed

• Resource group name: Dapr-Perf-daprprf3a2dc0339a
• Azure region: westus3

✅ Build succeeded

• Image tag: daprprf3a2dc0339a
• Test image tag: daprprf3a2dc0339a

✅ Perf tests succeeded

• Image tag: daprprf3a2dc0339a
• Test image tag: daprprf3a2dc0339a

[dapr-bot]

Dapr Version Skew e2e test (dapr-sidecar-master - 1.12.3)

🔗 Link to Action run

Commit ref: da5607f

✅ Version Skew tests passed

[dapr-bot]

Dapr Version Skew e2e test (control-plane-master - 1.12.3)

🔗 Link to Action run

Commit ref: da5607f

✅ Version Skew tests passed

[dapr-bot]

Dapr Version Skew integration test (dapr-sidecar-master - 1.12.3)

🔗 Link to Action run

Commit ref: da5607f

✅ Version Skew tests passed

[dapr-bot]

Dapr Version Skew integration test (control-plane-master - 1.12.3)

🔗 Link to Action run

Commit ref: da5607f

✅ Version Skew tests passed

[ASHIQUEMD]

/ok-to-test

[dapr-bot]

Dapr E2E test

🔗 Link to Action run

Commit ref: da5607f

✅ Build succeeded for linux/amd64

• Image tag: dapre2edb29d39ea5l
• Test image tag: dapre2edb29d39ea5l

✅ Infrastructure deployed

Cluster	Resource group name	Azure region
Linux	Dapr-E2E-dapre2edb29d39ea5l	westus3
Windows	Dapr-E2E-dapre2edb29d39ea5w	westus3
Linux/arm64	Dapr-E2E-dapre2edb29d39ea5la	eastus

✅ Build succeeded for windows/amd64

• Image tag: dapre2edb29d39ea5w
• Test image tag: dapre2edb29d39ea5w

❌ Tests failed on windows/amd64

Please check the logs for details on the error.

❌ Tests failed on linux/amd64

Please check the logs for details on the error.

[mukundansundar]

lgtm

[JoshVanL]

@mukundansundar @ASHIQUEMD this PR fails integration tests as we now always have a workflow component returned on the component list from the metadata endpoint. Tests need updating to reflect this.

[JoshVanL]

@mukundansundar @ASHIQUEMD Please can you also consider hot reloading, and how hot reloading works for this component type? It doesn't look like there are any tests covering this.

[ItalyPaleAle]

@mukundansundar @ASHIQUEMD Please can you also consider hot reloading, and how hot reloading works for this component type? It doesn't look like there are any tests covering this.

Just like for the actor state store, we shouldn't support hot reloading here as it could cause serious consistency issues.

[JoshVanL]

@ItalyPaleAle I think this is fine for now but do think long term this should be supported. We need to include code that blocks hot reloading of workflowbackend with tests and update the docs to say that it is not supported.