Request for example with multiple versioned processes

[tokarenko]

Description

Please provide example on how to build single Kogito application serving multiple processes with multiple versions.

Implementation ideas

I suppose that the processes should be configured as Maven modules.

[radtriste]

@tokarenko Thanks for sharing.

The purpose of Kogito is cloud-oriented so the different processes/versions should be handled by different applications.
That way you can target the correct process by targeting the correct application.
Also, to use different versions of the same process (so still different application running), I suggest to use a service mesh like https://istio.io/. That way you can target the needed version of your process.

Would that fit ?

[ricardozanini]

To add on top of the excellent @radtriste reply.

Ideally, you should focus on only one workflow per Kogito application. So you have a straightforward interface, a single instance of a workflow running on your architecture. For example, you can move faster and have a clean CI/CD process this way.

Versioning a process, I'd recommend a service mesh/routing operator to have multiple versions of the same application running at the same time, as @radtriste mentioned.

Without a mesh, you can try to do something similar to a Canary deployment, as exemplified here: https://blog.gurock.com/implement-ab-testing-using-kubernetes/.

Or even use NGINX ingress to have different request loads to different versions of the same services. There are many ways of implementing this.

Long story short, I'd recommend leaving the application versioning to the Kubernetes platform.

[tokarenko]

Thank you Tristan and Ricardo for your quick and valuable replies that I will use for guidance. I would be grateful if you could answer my further questions.

Question 1: How to provide for persistence with versioned processes?

From the examples I understood that process version control should be done through the "Version" property in BPMN diagram and Maven GAV is not relevant. Is ID for persistence generated from BPMN digram name and version? Should I guard persistence storage and Data Index against collisions of process names and versions when deploying multiple Kogito applications? I assume that all Kogito applications use shared infrastructure of Data Index, persistence storage, Kafka, etc.

Question 2: What use may be of Data Index in routing and how to do process monitoring with versioned processes?

Cristiano Nicolai mentioned in Process versioning protocol that Data Index is also a sort of Gateway API:

Q: Could the index service GraphQL just have a mutation that joins each of the versions data into a single table response?
A: Data Index is also a sort of Gateway API in that sense, it could act as a routing layer to redirect a mutation to start a new instance to the newest version of the process. That also means that the Data Index would keep track of the available deployments. And as for data, that's already done by Data Index, it collects data from multiple runtimes, different processes and versions as well, so it should be transparent in that sense.

Question 2.1: What use may be of Data Index in routing with service mesh or Canary deployment?

Question 2.2: How to do process monitoring and analysis for the whole versioned process? Should I write mutations on Data Index GraphQL to join each of the versions data into a single table response?

Question 3: How to undeploy versioned process from shared infrastructure?

Assuming that Kogito applications use shared infrastructure of Data Index, persistence storage, Kafka, etc. How to undeploy and clear the infrastructure from versioned process data?

[radtriste]

cc @cristianonicolai

[cristianonicolai]

@tokarenko let me try to answer some of these.

Question 1: How to provide for persistence with versioned processes?

From the examples I understood that process version control should be done through the "Version" property in BPMN diagram and Maven GAV is not relevant. Is ID for persistence generated from BPMN digram name and version? Should I guard persistence storage and Data Index against collisions of process names and versions when deploying multiple Kogito applications? I assume that all Kogito applications use shared infrastructure of Data Index, persistence storage, Kafka, etc.

In order to isolate processes or runtimes in the database, you can use two strategies. These are defined via kogito.workflow.version-strategy property that was recently introduced. In there, you can define either workflow ( default ) or project. When using workflow, it matches what you described, it uses the version attribute from the BPMN diagram to know which version is running. That means that updating the version is a manual process, and you can fine-tune that per process in the runtime. When using project, the workflow version is ignored and Kogito will use the Maven project version instead for all processes. That means that once you bump the version of the project, that will isolate the deployment from older versions. With these strategies, you can use a shared database table for all running instances and isolation is done by the runtime itself.

Data Index should be able to handle data from multiple runtimes and different versions. In there, you will see the aggregated data from all.

Question 2: What use may be of Data Index in routing and how to do process monitoring with versioned processes?

Cristiano Nicolai mentioned in Process versioning protocol that Data Index is also a sort of Gateway API:

Q: Could the index service GraphQL just have a mutation that joins each of the versions data into a single table response?
A: Data Index is also a sort of Gateway API in that sense, it could act as a routing layer to redirect a mutation to start a new instance to the newest version of the process. That also means that the Data Index would keep track of the available deployments. And as for data, that's already done by Data Index, it collects data from multiple runtimes, different processes and versions as well, so it should be transparent in that sense.

Question 2.1: What use may be of Data Index in routing with service mesh or Canary deployment?

I mentioned that we would like to expand the Gateway API in that sense, but I think this is probably more advanced that it should do. What I mean is that the GAteway API, and Data Index, in general knows how to contact to specific runtime for a given process instance. This is done via kogito.service.url in the runtime. Whether that url is pointing to a mesh or a deployment via canary it probably outside the concern of Data Index, it just needs to know how to find that specific service.

Question 2.2: How to do process monitoring and analysis for the whole versioned process? Should I write mutations on Data Index GraphQL to join each of the versions data into a single table response?

In Data Index, you will have data from all process instances on different versions. I don't think you need a mutation, you will be able just to query all process instances, and these will have a version attribute that will identify the running version.

Question 3: How to undeploy versioned process from shared infrastructure?

Assuming that Kogito applications use shared infrastructure of Data Index, persistence storage, Kafka, etc. How to undeploy and clear the infrastructure from versioned process data?

At this point I dont think there is much that needs to be done for that. Regarding the runtime itself, it only keeps data of active process instances, if you remove a runtime that has all process instances completed, you shouldn't have any data in db any longer anyway. As for Data Index, it will keep data for as long as you want, so data regarding executed instances will still be there, but Data Index wouldn't redirect any request to that runtime if all process instances are completed.