Support setting custom Attributes on PubSub Messages

[AndreasBergmeier6176]

It seems from #735 that currently the sdk has sole ownership of the pubsub.Message and lacks any means to set non-CloudEvents Attributes on PubSub Message when sending.
There should be a way of sending custom Attributes (e.g. content-encoding: zstd).

[embano1]

Briefly skimmed the code and it looks like it should work with SetExtension which IIUC is called through the stack based on the extension attributes defined on a CloudEvent when calling Client.Send(). However, we do prefix even extension attributes with ce- which means content-encoding: zstd becomes ce-content-encoding: zstd.

Questions:

@AndreasBergmeier6176 is my understanding of propagating CloudEvent extension attributes to pubsub.Message attributes correct?

@duglin I thought the SDK should not prefix (custom) extension attributes (I should be reading the code/SPEC but asking the expert here :) )

[AndreasBergmeier6176]

So after reading https://github.com/cloudevents/spec/blob/main/cloudevents/documented-extensions.md and looking at SetExtension here are my thought:

1. I think using SetExtension is wrong here, since Extensions have a somewhat narrow spec (e.g. limited allowed key characters). Not everything you want to express as PubSub Message Attributes falls inside of the Extension spec.
2. When we talk about PubSub, the more practical limit for Attribute key characters is documented here. One could implement these limitations in the SDK but to me it is sufficient when it fails when sending.
3. To me the less cumbersome approach to setting arbitrary custom Attributes would be to introduce a kind of Interceptor concept for PubSub Messages. Just before a PubSub Message is to be sent, it can be first passed to an (optional) Interceptor (maybe as simple as func(*pubsub.Message) *pubsub.Message). In the Interceptor implementation, custom processing (e.g. setting Attributes outside of CloudEvents scope) can take place. This also makes it no longer necessary for SDK to bend over backwards even more to map CloudEvents concepts to PubSub and further complicate the API surface.
4. Alternatively the less invasive way to be able to modify the PubSub Message could be to change WritePubSubMessage from being a function to being a variable. This way a proxy function could be introduced e.g. v2.WritePubSubMessage = myproxy.
5. For a cleaner PubSub specific way, it would also be possible to create new Context values, where PubSubMessageAttributes can be directly attached (e.g. via WithPubSubMessageAttributes) to context and will be added in WritePubSubMessage. Took a stab at implementing this here: Add WithCustomAttributes for PubSub #976

[duglin]

I thought the SDK should not prefix (custom) extension attributes (I should be reading the code/SPEC but asking the expert here :) )

all CE context attributes, including custom/extension attributes, are serialized like normal CE context attributes... so for HTTP they're prefixed with ce-. However, I believe in this case @AndreasBergmeier6176 wants to be able to set non-CE attributes as well, right?

If so, the question for me then becomes: should the SDK user be expected to know how the attribute will be serialized and add the prefix, or should the SDK do it for them? Not all bindings add prefixes so it seems risky to for the user to do it themselves. Would it be possible to allow for people to add attributes and tell the SDK whether each is a CE one or not? E.g. pass in two lists instead of just one? Then the SDK, based on the binding/transport, can prefix the CE attributes if needed.

[duglin]

Oh, sorry, this is just for Google pub/sub... is that limited to just http?

[AndreasBergmeier6176]

Then the SDK, based on the binding/transport, can prefix the CE attributes if needed.

I think from a clarity sense I would prefer to always know the serialization beforehand.
To this end it might make sense to also introduce different types, say:

• Attributes, current type which is specific to CloudEvents - thus in the PubSub case get a ce- prefix
• ExternalAttributes, new type which are attributes not covered by the CloudEvent spec - thus in the PubSub case get no implicit prefix.

[AndreasBergmeier6176]

Oh, sorry, this is just for Google pub/sub... is that limited to just http?

Can you elaborate what you mean by limited that limited to just http?

[duglin]

Then the SDK, based on the binding/transport, can prefix the CE attributes if needed.

I think from a clarity sense I would prefer to always know the serialization beforehand. To this end it might make sense to also introduce different types, say:

* `Attributes`, current type which is specific to CloudEvents - thus in the PubSub case get a `ce-` prefix

* `ExternalAttributes`, new type which are attributes not covered by the CloudEvent spec - thus in the PubSub case get no implicit prefix.

Moving to the PR but I think this gets to the concern I have...