Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pass eventOccurredTS value to the custom_args of the sendgrid/twilio body #1955

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

miguelpdiaz8
Copy link
Contributor

In the DestinationAction payload we receive eventOccurredTS, which we use to stats event delivery latency which is for some reason called eventDeliveryTS. Regardless of confusing name - this metric helps to understand how much time it took from the event creation till the end of event processing by Destination Action.

It would be helpful to pass that eventOccurredTS value to the custom_args of the sendgrid/twilio body, so it will be returned to a webhook call after email/sms is delivered and we can add that [twilio|sendgrid]_eventDeliveryDuration metric to engage-events-ingest service (supplying all the tags that twilio_total and sendgrid_total have) to track how much time it takes for email/sms to be delivered from the moment of placing it to centrifuge to the moment of delivering it to the user.

Testing

Updated unit tests

@joe-ayoub-segment
Copy link
Contributor

hi @miguelpdiaz8 please let me know when someone on your team has reviewed, and when you are happy for me to review+deploy.

@miguelpdiaz8
Copy link
Contributor Author

miguelpdiaz8 commented Apr 3, 2024

hi @miguelpdiaz8 please let me know when someone on your team has reviewed, and when you are happy for me to review+deploy.

Hi @joe-ayoub-segment I just shared it with the team, and you can review it too, thanks

@@ -121,6 +122,7 @@ export abstract class TwilioMessageSender<TPayload extends TwilioPayloadBase> ex
const customArgs: Record<string, string | undefined> = {
...this.payload.customArgs,
space_id: this.settings.spaceId,
event_occurred_ts: this.payload?.eventOccurredTS,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You missed assigning adding it to Tags - which is used by Event Streams:

body.append('Tags', JSON.stringify(tags))
if (webhookUrlWithParams) {
body.append('StatusCallback', webhookUrlWithParams)
}

Even though we do pass that "webhook" (i.e. callback Url) - I don't believe it is used anymore. So most important is add it to tags. However you need to make sure you're not exceeding tag limits - I think it was up to 10 tags max.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I had the tags change but changed it back, I was waiting to ask you about the tags limit knowing it exists and it currently has 10 tags already, do you know which one I could remove safely, or if that limit can be changed? Thanks Philipp

Copy link
Member

@pmunin pmunin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Twilio Tags missing

@joe-ayoub-segment
Copy link
Contributor

hi @miguelpdiaz8 @pmunin @cogwizzle - just a reminder to let me know when I can review / deploy

@miguelpdiaz8 miguelpdiaz8 requested a review from pmunin April 8, 2024 16:25
@cogwizzle
Copy link
Contributor

Looks good. I'll defer to @pmunin as he has requested additional changes.

@rahul8590
Copy link

Can you confirm if adding a new field within custom_args impacts the processing of current event payload with [engage-events-ingest service? In theory, it shouldn't. But in the past, we have seen payloads get dropped (4xx response) due to bad format.

@miguelpdiaz8
Copy link
Contributor Author

miguelpdiaz8 commented Apr 15, 2024

Can you confirm if adding a new field within custom_args impacts the processing of current event payload with [engage-events-ingest service? In theory, it shouldn't. But in the past, we have seen payloads get dropped (4xx response) due to bad format.

So you are saying there is no a hard limit of 10 custom args? @rahul8590

@joe-ayoub-segment
Copy link
Contributor

hi @miguelpdiaz8 @pmunin @rahul8590 @cogwizzle - friendly reminder to let me know when this is ready for deploy.

@joe-ayoub-segment
Copy link
Contributor

hi @miguelpdiaz8 - please let me know when this PR is ready for review / and then to be deployed.
If you need more time please convert it to a draft PR :)

@miguelpdiaz8 miguelpdiaz8 marked this pull request as draft April 23, 2024 18:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants