Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DRIVERS-1975: Add showExpandedEvents flag and new change stream events #1220

Conversation

baileympearson
Copy link
Contributor

@baileympearson baileympearson commented May 18, 2022

This PR adds support for a new change stream flag, showExpandedEvents. This flag enables a set of new change stream events and new fields on existing change stream documents.

Technical Design: https://docs.google.com/document/d/1BWElA2_AUH5Wupjnr1igYMZflQTRmuszoNh2T6NAqwY/edit#
Syntax Update Document: https://docs.google.com/document/d/1DHlvXSbrLVR_jnnf96AbUqoyakL_oIp9KEI4Gw3R6II/edit#
Node Driver POC - mongodb/node-mongodb-native#3254

Please complete the following before merging:

  • Bump spec version and last modified date.
  • Update changelog.
  • Make sure there are generated JSON files from the YAML test files.
  • Test changes in at least one language driver.
  • Test these changes against all server versions and topologies (including standalone, replica set, sharded clusters, and serverless).

@baileympearson baileympearson changed the title Add new change stream events to spec DRIVERS-1975: Add showExpandedEvents flag and new change stream events May 19, 2022
Comment on lines +153 to 155
| "refineCollectionShardKey"
| "reshardCollection";

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

refineCollectionShardKey and reshardCollection events are behind a (currently unsupported) feature flag. I included them here for completeness, but notice that there are no spec tests testing for these new events. If there are no objections, I can file a follow up ticket to add tests for these events once they're available.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha. Sounds like the feature flag won't be supported until 6.1.0. I agree that we should file a follow-up ticket and add tests once the feature flag can be enabled and drivers can actually test.

@baileympearson baileympearson marked this pull request as ready for review May 19, 2022 17:57
@baileympearson baileympearson requested a review from a team as a code owner May 19, 2022 17:57
@baileympearson baileympearson requested review from benjirewis and removed request for a team May 19, 2022 17:57
Copy link
Contributor

@benjirewis benjirewis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking great 🧑‍🔧 ! Some initial comments; I'll leave another round of review for the actual tests.

source/change-streams/change-streams.rst Outdated Show resolved Hide resolved
source/change-streams/change-streams.rst Show resolved Hide resolved
source/change-streams/change-streams.rst Outdated Show resolved Hide resolved
source/change-streams/change-streams.rst Outdated Show resolved Hide resolved
source/change-streams/change-streams.rst Outdated Show resolved Hide resolved
source/change-streams/change-streams.rst Outdated Show resolved Hide resolved
source/change-streams/change-streams.rst Outdated Show resolved Hide resolved
Copy link
Contributor

@abr-egn abr-egn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall, and passes in the Rust driver!

@@ -208,7 +229,27 @@ If an aggregate command with a ``$changeStream`` stage completes successfully, t
* The wall time from the mongod that the change event originated from.
* Populated for server versions 6.0 and above.
*/
wallTime: Optional<Datetime>;
wallTime: DateTime;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this changed to non-optional? AFAIK this isn't populated <6.0.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, that was unintentional. I changed it back to optional.

*/
updateDescription: Optional<UpdateDescription>;
updateDescription: Optional<UpdateDescription | Document>;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a breaking change for the Rust driver (going from a struct to an enum in a public API); from the syntax doc, it looks like the underlying change is the addition of a specialFields field?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually this change was unintentional. Originally I wrote the spec update to include the showRawUpdateDescription and showSystemEvents flags but these flags are internal and not intended to be exposed so drivers don't have to support them.

The specialFields and rawUpdateDescription fields on the UpdateDescription are guarded behind showRawUpdateDescription, and so don't need to be added.

arguments:
pipeline: []
showExpandedEvents: true
showSystemEvents: true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since showSystemEvents isn't mentioned in the spec yet, maybe this test should wait for a follow-up?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Related to my comment above about showSystemEvents and showRawUpdateDescription - this change is unnecessary. Thanks for catching this.

Unlike other languages, Node loads the spec tests directly into JS objects (basically just dictionaries) so we don't run into serialization errors like other languages might if extra, unsupported options are present!

Copy link
Contributor

@abr-egn abr-egn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Contributor

@benjirewis benjirewis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Great work 🧑‍🔧

@baileympearson baileympearson merged commit 9652564 into mongodb:master May 31, 2022
@baileympearson baileympearson deleted the DRIVERS-1975-c2c-change-stream-events branch May 31, 2022 18:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants