[Workflows] Events of the same name are not buffered

[olitomlinson]

cc @cgillum

Let's say a workflow implements a simple monitor pattern that awaits on the same external event name ("my-external-event") indefinitely.

public override async Task<bool> RunAsync(WorkflowContext context, List<string> input)
{
    var externalEvent = await context.WaitForExternalEventAsync<string>("my-external-event");

    input.Add(externalEvent);
    context.SetCustomStatus(string.Concat(input.ToArray()));
    context.ContinueAsNew(input);
    return true;
}

Actual behaviour

If several events (named "my-external-event") are raised in quick succession or in parrallel to the awaiting workflow, only the first event is consumed by the awaiting workflow, all the other events are dropped, and are therefor not allocated on subsequent generations of the monitor loop.

Expected behaviour

If several events (named "my-external-event") are raised in quick succession or in parallel to the awaiting workflow, all the events will be buffered (up to some configurable user-defined max limit) and each iteration of the monitor loop will consume one of the buffered messages (via context.WaitForExternalEventAsync()), ideally in a FIFO order)

Note : preserveUnprocessedEvents: true does not produce the expected behaviour, despite it sounding like it should.

If set to true, re-adds any unprocessed external events into the new execution history when the workflow instance restarts. If false, any unprocessed external events will be discarded when the workflow instance restarts.

[cgillum]

The behavior you're looking for is supposed to be handled by preserveUnprocessedEvents: true. What are you observing when this is set?

[olitomlinson]

@cgillum preserveUnprocessedEvents: true has no impact it seems.

[olitomlinson]

Potentially related to dapr/dotnet-sdk#1129

[cgillum]

@nyemade-uversky let's add this to the tracking board for workflows. This functionality is important for developers leveraging the monitor pattern.

[olitomlinson]

@nyemade-uversky bump

[mukundansundar]

/assign

[shivamkm07]

/assign

[shivamkm07]

This issue should be resolved with microsoft/durabletask-dotnet#194. It's also verified in dapr/dotnet-sdk#1155 that multiple events of the same name being created in parallel is working as expected. Can this issue be closed then @olitomlinson ?

[olitomlinson]

Can confirm this issue is now fixed. Thanks @shivamkm07