Headers duplicated when sending multiple events and using WithCustomHeader context decorator

[dan-j]

These functions enable users to customise the base set of headers to send to a CE consumer:

sdk-go/v2/protocol/http/headers.go

Lines 49 to 55 in a9224ad

	func HeaderFrom(ctx context.Context) http.Header {
	return binding.GetOrDefaultFromCtx(ctx, headerKey, make(http.Header)).(http.Header)
	}
	func WithCustomHeader(ctx context.Context, header http.Header) context.Context {
	return context.WithValue(ctx, headerKey, header)
	}

Then it's used in Protocol.makeRequest(context.Context) to create our new HTTP request:

sdk-go/v2/protocol/http/protocol.go

Lines 199 to 203 in a9224ad

	req := &http.Request{
	Method: http.MethodPost,
	Header: HeaderFrom(ctx),
	}

Then when writing the event to the request, httpRequestWriter.SetAttribute(Attribute, interface{}) appends the attribute to the request.Header:

sdk-go/v2/protocol/http/write_request.go

Line 121 in a9224ad

b.Header[mapping] = append(b.Header[mapping], s)

Sending multiple events with the same underlying ctx is causing these headers to be duplicated on each invocation.

Can we change newRequest to clone the headers? i.e.

req := &http.Request{
	Method: http.MethodPost,
	Header: HeaderFrom(ctx).Clone(),
}

[embano1]

Thx for flagging this @dan-j !

I briefly skimmed the code, and I'm not yet 100% following. Sending multiple events should invoke Send, i.e., makeRequest every single time (per event), thus creating a new http.Request for each call:

// Request implements binding.Requester
func (p *Protocol) Request(ctx context.Context, m binding.Message, transformers ...binding.Transformer) (binding.Message, error) {
	if ctx == nil {
		return nil, fmt.Errorf("nil Context")
	} else if m == nil {
		return nil, fmt.Errorf("nil Message")
	}

	var err error
	defer func() { _ = m.Finish(err) }()

	req := p.makeRequest(ctx)

	if p.Client == nil || req == nil || req.URL == nil {
		return nil, fmt.Errorf("not initialized: %#v", p)
	}

	if err = WriteRequest(ctx, m, req, transformers...); err != nil {
		return nil, err
	}

	return p.do(ctx, req)
}

Can you elaborate more on what "sending multiple events" means in this issue? Do we also have an issue with retrying the same event btw?

[dan-j]

Hey @embano1, yeah a new request is made, but if you look at the second snippet, is initialises Request.Header with a call to HeaderFrom(ctx). This passes the same value on each invocation. Since http.Header is just a map[string][]string, it's passing the same map each time, and modifications to that map are retained.

I believe cloning the headers each time should fix this issue. I don't mind providing a PR, just it was late Friday evening when I finally spotted the bug (it's affecting us in production) so we just made a local fix on our own codebase for now.

FYI, a temporary workaround is to override the context value with a cloned copy before calling Send(). i.e:

a.sender.Send(cehttp.WithCustomHeader(ctx, cehttp.HeaderFrom(ctx).Clone()), event)