Try out Datadog CI Visibility for CPython

[hugovk]

The short story

The CPython CI is growing quickly, we'd like to try out some monitoring tools.

Long version

The CPython GitHub Actions CI is getting more complex. We're testing more operating systems and architectures, and things like free-threaded, JIT, address and thread sanitisation, fuzzing, docs. For example:

• On the one hand, automated testing is good, it prevents bugs and regressions in a fast-moving project, and means we can better support the hardware used by our users (e.g. aarch64-apple-darwin was just added to tier 1 in PEP 11).

• On the other, the CI is getting more complex and taking longer to run. We're on the GitHub Free plan which limits us to 5 concurrent macOS jobs. During busy periods this has caused builds to wait ~100 minutes before starting, and to mitigate we've begun reducing some testing (CI: Only test free-threading with faster macOS M1 cpython#116814).

Monitoring could help us see where we're using CI resources and help manage it. Hopefully it would help us see how long different jobs take, how long workflows take within jobs, and so on.

Datadog have a tool called CI Visibility: https://www.datadoghq.com/blog/datadog-github-actions-ci-visibility/ I've not tried it before, but it looks useful.

Datadog also have an Open Source Partner Program to offer free accounts, and I see the PSF is already a partner, using it to monitor PyPI: https://opensource.datadoghq.com/

@ewdurbin Is CI Visibility available for free under the current PSF account and would we be able to try it out for https://github.com/python/cpython? (Config docs)

If not, what are the next steps to set up a demo? Thank you!

[ewdurbin]

I've configured our datadog account to start collecting these. I'll just need to create a team in our DataDog account to view them and invite folks who will be monitoring.

Let me know who to invite.

[hugovk]

Thank you!

Please could you add me? My email's in the voter file.

I've asked other core devs to comment here if they'd like adding too.

[ewdurbin]

I’d prefer to do it in one batch. So let me know when the list is complete.

[gvanrossum]

Is this where I should express interest?

[hugovk]

This is fine, I'll collect a list and send it to Ee tomorrow or the day after 👍

[ned-deily]

Me, too, please!

[pradyunsg]

Me too please.

[willingc]

Me too please.

[hugovk]

Thanks all, I've emailed the list to Ee.

[ewdurbin]

Invite sent.

[willingc]

@ewdurbin Thanks Ee. Login worked just fine.

[erlend-aasland]

Too late to get an invite?

[hugovk]

@erlend-aasland You should have received an invite now.