Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support acknowledgement configuration for inbound message queue connectors #2134

Closed
chillleader opened this issue Mar 7, 2024 · 2 comments · Fixed by #2333
Closed

Support acknowledgement configuration for inbound message queue connectors #2134

chillleader opened this issue Mar 7, 2024 · 2 comments · Fixed by #2333
Assignees
@chillleader
Labels
component:qa Task containing all details related to QA kind:enhancement New feature or request scope:kafka scope:rabbitmq scope:sqs

Comments

@chillleader
Copy link
Member

Is your feature request related to a problem? Please describe.

Inbound message queue connectors such as RabbitMQ Connector currently acknowledge all messages that were consumed without errors. This behavior doesn't take into account the fact that successfully consumed events not always lead to message correlation in Zeebe. For instance, if the activation condition did not match, the message would not be correlated, but the message would be consumed and removed from the queue. Therefore, in certain cases a different approach is needed, when a connector only acknowledges messages that lead to message correlation.

Message queue connectors should allow configuring whether messages are acknowledged always or only after successful correlation.

Describe the solution you'd like

  • A new dropdown property is added to the element templates, containing 2 choices: "Acknowledge all" and "Acknowledge only correlated events" (exact naming to be reviewed).
  • The following inbound connectors should support this property: Kafka, RabbitMQ, SQS.

Additional context

This issue is part of the following epic: https://github.com/camunda/product-hub/issues/2157
@chillleader chillleader self-assigned this Mar 7, 2024
@chillleader
Copy link
Member Author

We can provide this as part of the CorrelationResult.ActivationConditionNotMet

@chillleader chillleader added the component:qa Task containing all details related to QA label Apr 18, 2024
@chillleader chillleader mentioned this issue Apr 22, 2024
Merged
@chillleader chillleader mentioned this issue May 21, 2024
Merged
@chillleader
Copy link
Member Author

A new element template property will be made available for RabbitMQ, Kafka, SQS inbound connectors ("Consume unmatched events" checkbox). It affects how the connector implementation treats incoming messages with unmatched activation condition.

The following behavior is expected:

  • RMQ:
    • When checkbox is inactive, messages with unmatched activation condition are requeued
    • When active, such messages are consumed and removed from the queue
  • Kafka:
    • When inactive, consumer stops consuming messages and doesn't commit offsets (i.e. doesn't mark such message as processed)
    • When active, consumer commits the offests and keeps processing messages
  • SQS:
    • When inactive, message is requeued (not deleted from the queue) after visibility timeout
    • When active, message is removed from the queue

@chillleader chillleader mentioned this issue May 23, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@chillleader chillleader

Labels
component:qa Task containing all details related to QA kind:enhancement New feature or request scope:kafka scope:rabbitmq scope:sqs
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat(inbound)!: message acknowledgement config camunda/connectors
1 participant
@chillleader