Engineering and Ops Release Testing Process

This document is meant to outline our current thoughts on our release process. It is meant or provide general guidelines and not impose rigid restrictions on communications or anyone working preferences. Please edit and update as we learn better interaction patterns within our teams.

Release Testing

Our team has a preference for "context-free" communications. With respect to our release process this means that communication between our teams provides enough information so that follow-up questions are ideally not required. Additionally, the engineering work-stream finds full screen recordings to be very helpful (with the console open) when attempting to reproduce issues.

Steps

1. Engineer who is the “release owner” will create a PR for the release using the release script. This PR in GitHub will act as the primary location for communications.
2. The release owner will create a release thread in Discord and tag operations alert them that the release is ready for their testing.
3. Operations will conduct its initial review and test the release.
4. Once completed, Operations will comment with all issues found on the release PR in GitHub.
   1. Each issue should be very clearly marked as a blocker or non-blocking
   2. For non-blocking issues, operations should create a follow up ticket since the work most likely won't be addressed during the release process
   3. Below is an example of what this thread might look like
5. Once Operations has finished their initial review and documented all issues found, Operations tags the release owner.
6. Release owner is responsible for either directly resolving the blocking issues identified or delegating the work out to team members as needed. The release owner is responsible for this coordination and should follow up to make sure all issues are resolved in a timely manner to not slow the release from moving forward.
7. A release owner may change mid-release and delegate the role to a new release owner which will take over. They may become release owner when back online, or wait for the substition release owner, as long as operations are aware of the current owner at all times.
8. Once all blocking issues have been resolved the release owner should comment on the PR with enough context to guide their next round of testing and ping operations in the discord release thread as well.
9. Rinse and repeat until no more blockers and release is pushed to production

Ephemeral Env Testing

Follow same process as above from #3 down, engineer should alert Operations in #operations-public to request feature testing. Engineers who request testing assistance should actively indicate the priority of this testing to operations.