Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GODRIVER-2348 Make CSOT feature-gated behavior the default #1515

Open
wants to merge 70 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 61 commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
76fe99a
GODRIVER-2762 Use minimum RTT for CSOT maxTimeMS calculation instead …
prestonvasquez Dec 21, 2023
c96883a
GODRIVER-2762 Clean up rtt monitor code
prestonvasquez Dec 21, 2023
88e9f0d
GODRIVER-2762 Add command-execution.yml
prestonvasquez Dec 21, 2023
85812f6
GODRIVER-2762 Fix typo
prestonvasquez Dec 21, 2023
1a0517c
Merge branch 'master' into GODRIVER-2762
prestonvasquez Dec 29, 2023
49298ee
GODRIVER-2762 PR requests
prestonvasquez Dec 29, 2023
04aeb6d
GODRIVER-2762 Remove reference to RTT90 in max time calc
prestonvasquez Jan 5, 2024
2d61660
GODRIVER-2348 connectTimeoutMS to cover all blocking operations durin…
prestonvasquez Jan 11, 2024
0ba0d69
GODRIVER-2762 Fix rounding, cleanup deadline exceeded error
prestonvasquez Jan 11, 2024
bccb529
Merge branch 'master' into GODRIVER-2762
prestonvasquez Jan 12, 2024
f7d8111
GODRIVER-2348 merge master
prestonvasquez Jan 13, 2024
698e4e0
GODRIVER-2762 Use args in wait event
prestonvasquez Jan 13, 2024
8b10b54
Merge branch 'GODRIVER-2762' of github.com:prestonvasquez/mongo-go-dr…
prestonvasquez Jan 13, 2024
ce0c7c2
Update x/mongo/driver/topology/rtt_monitor.go
prestonvasquez Jan 13, 2024
6095151
Merge branch 'master' into GODRIVER-2762
prestonvasquez Jan 13, 2024
a2f0c9a
GODRIVER-2348 Merge 2762
prestonvasquez Jan 16, 2024
d10ced0
GODRIVER-2348 merge master
prestonvasquez Jan 16, 2024
9cb8028
Merge branch 'master' into GODRIVER-2348
prestonvasquez Jan 18, 2024
4669d9b
GODRIVER-2466 Implement assertEventCount
prestonvasquez Jan 19, 2024
7aa63f0
GODRIVER-2466 Fix typo
prestonvasquez Jan 19, 2024
45c6725
Merge remote-tracking branch 'origin/GODRIVER-2466' into GODRIVER-2348
prestonvasquez Jan 19, 2024
998b725
GODRIVER-2466 Remove duplicate failpoint warning
prestonvasquez Jan 19, 2024
15c6d42
Merge remote-tracking branch 'origin/GODRIVER-2466' into GODRIVER-2348
prestonvasquez Jan 19, 2024
9af952f
GODRIVER-2348 Update connectTimeoutMS to cover all blocking operation…
prestonvasquez Jan 19, 2024
2d53d6a
GODRIVER-2348 Deprecate socketTimeoutMS
prestonvasquez Jan 23, 2024
d2b8c50
Merge branch 'master' into GODRIVER-2348
prestonvasquez Jan 23, 2024
9c17094
Merge branch 'master' into GODRIVER-2348
prestonvasquez Jan 23, 2024
d9d61f4
GODRIVER-2348 Remove rw connection timeouts
prestonvasquez Jan 25, 2024
51d6c1b
Merge branch 'GODRIVER-2348' of github.com:prestonvasquez/mongo-go-dr…
prestonvasquez Jan 25, 2024
25368d6
Merge branch 'master' into GODRIVER-2348
prestonvasquez Jan 26, 2024
c32af34
GODRIVER-2348 Deprecate wtimeout
prestonvasquez Jan 30, 2024
54abbc5
GODRIVER-2348 Deprecate maxTimeMS
prestonvasquez Feb 1, 2024
a4c7211
Merge branch 'master' into GODRIVER-2348
prestonvasquez Feb 1, 2024
49427b3
GODRIVER-2348 Deprecate maxCommitTimeMS
prestonvasquez Feb 1, 2024
4a55e76
GODRIVER-2348 Cont. Deprecating MaxTimeMS
prestonvasquez Feb 2, 2024
9293b84
GODRIVER-2348 Implement server selection logic
prestonvasquez Feb 3, 2024
023a1d1
Merge branch 'master' into GODRIVER-2348
prestonvasquez Feb 6, 2024
f6a4f97
Merge branch 'master' into GODRIVER-2348
prestonvasquez Feb 7, 2024
2501fe8
GODRIVER-2348 Cont. implementing Server Selection req
prestonvasquez Feb 8, 2024
a56844e
GODRIVER-2348 Remove unused topology struct
prestonvasquez Feb 8, 2024
7b64cee
GODRIVER-2348 Rename maxTimeMS for cursor to maxAwaitTimeMS
prestonvasquez Feb 9, 2024
11d1f3b
GODRIVER-2348 Add retryability-legacy-timeouts back to UST
prestonvasquez Feb 9, 2024
6892b1f
GODRIVER-2348 Resolve connstring merge conflicts
prestonvasquez Feb 9, 2024
15d80e0
GODRIVER-2348 Implement change stream CSOT conditions
prestonvasquez Feb 10, 2024
99b4389
GODRIVER-2348 Remove CSOT-specific context opts
prestonvasquez Feb 13, 2024
9f4a8e5
GODRIVER-2348 Remove debug tools
prestonvasquez Feb 13, 2024
e69bc0a
GODRIVER-2348 Extend test coverage for csot
prestonvasquez Feb 13, 2024
16f1027
GODRIVER-2348 Resolve race condition
prestonvasquez Feb 14, 2024
818f7d1
GODRIVER-2348 Resolve merge conflict
prestonvasquez Feb 14, 2024
27ae440
GODRIVER-2348 Refine csot logic
prestonvasquez Feb 20, 2024
2c94bd9
GODRIVER-2348 Fix linting errors
prestonvasquez Feb 21, 2024
f65fb19
GODRIVER-2348 Remove lingering comments
prestonvasquez Feb 21, 2024
75e44e2
Merge branch 'master' into GODRIVER-2348
prestonvasquez Feb 21, 2024
4424192
GODRIVER-2348 Fix network error pool test
prestonvasquez Feb 21, 2024
9a5df59
Merge branch 'master' into GODRIVER-2348
prestonvasquez Feb 26, 2024
3043a64
GODRIVER-2348 Resolve PR conversations
prestonvasquez Feb 27, 2024
b2e8844
Update internal/csot/csot.go
prestonvasquez Feb 27, 2024
884f2ba
Merge branch 'master' into GODRIVER-2348
prestonvasquez Mar 8, 2024
db69938
GODRIVER-2348 Resolve merge conflict
prestonvasquez Mar 18, 2024
058b343
Merge branch 'GODRIVER-2348' of github.com:prestonvasquez/mongo-go-dr…
prestonvasquez Mar 18, 2024
00fb9da
GODRIVER-2348 Do not set transactional wtimeout
prestonvasquez Mar 19, 2024
b191ec7
GODRIVER-2348 Merge master
prestonvasquez May 1, 2024
755204f
Update mongo/client.go
prestonvasquez May 1, 2024
8e26315
Update x/mongo/driver/topology/topology_test.go
prestonvasquez May 1, 2024
2aff850
GODRIVER-2348 Resolve various PR issues
prestonvasquez May 8, 2024
731aca7
Merge branch 'GODRIVER-2348' of github.com:prestonvasquez/mongo-go-dr…
prestonvasquez May 8, 2024
3b40fa4
GODRIVER-2348 Resolve build failure
prestonvasquez May 9, 2024
1620f92
GODRIVER-2348 Resolve merge conflicts
prestonvasquez May 9, 2024
a719b0a
GODRIVER-2348 Merge master
prestonvasquez May 22, 2024
c58c276
GODRIVER-2348 Use context listener to cancel HB check
prestonvasquez May 30, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
78 changes: 63 additions & 15 deletions internal/csot/csot.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,26 +11,74 @@ import (
"time"
)

type timeoutKey struct{}
type clientLevel struct{}

// MakeTimeoutContext returns a new context with Client-Side Operation Timeout (CSOT) feature-gated behavior
// and a Timeout set to the passed in Duration. Setting a Timeout on a single operation is not supported in
// public API.
//
// TODO(GODRIVER-2348) We may be able to remove this function once CSOT feature-gated behavior becomes the
// TODO default behavior.
func MakeTimeoutContext(ctx context.Context, to time.Duration) (context.Context, context.CancelFunc) {
// Only use the passed in Duration as a timeout on the Context if it
// is non-zero.
cancelFunc := func() {}
if to != 0 {
ctx, cancelFunc = context.WithTimeout(ctx, to)
func isClientLevel(ctx context.Context) bool {
val := ctx.Value(clientLevel{})
if val == nil {
return false
}
return context.WithValue(ctx, timeoutKey{}, true), cancelFunc

return val.(bool)
}

// IsTimeoutContext checks if the provided context has been assigned a deadline
// or has unlimited retries.
func IsTimeoutContext(ctx context.Context) bool {
return ctx.Value(timeoutKey{}) != nil
_, ok := ctx.Deadline()

return ok || isClientLevel(ctx)
}

// WithTimeout will set the given timeout on the context, if no deadline has
// already been set.
//
// This function assumes that the timeout field is static, given that the
// timeout should be sourced from the client. Therefore, once a timeout function
// parameter has been applied to the context, it will remain for the lifetime
// of the context.
func WithTimeout(parent context.Context, timeout *time.Duration) (context.Context, context.CancelFunc) {
prestonvasquez marked this conversation as resolved.
Show resolved Hide resolved
cancel := func() {}

if timeout == nil || IsTimeoutContext(parent) {
// In the following conditions, do nothing:
// 1. The parent already has a deadline
// 2. The parent does not have a deadline, but a client-level timeout has
// been applied.
// 3. The parent does not have a deadline, there is not client-level
// timeout, and the timeout parameter DNE.
return parent, cancel
}

// If a client-level timeout has not been applied, then apply it.
parent = context.WithValue(parent, clientLevel{}, true)

dur := *timeout

if dur == 0 {
// If the parent does not have a deadline and the timeout is zero, then
// do nothing.
return parent, cancel
}

// If the parent does not have a dealine and the timeout is non-zero, then
// apply the timeout.
return context.WithTimeout(parent, dur)
}

// WithServerSelectionTimeout creates a context with a timeout that is the
// minimum of serverSelectionTimeoutMS and context deadline. The usage of
// non-positive values for serverSelectionTimeoutMS are an anti-pattern and are
// not considered in this calculation.
func WithServerSelectionTimeout(
matthewdale marked this conversation as resolved.
Show resolved Hide resolved
parent context.Context,
serverSelectionTimeout time.Duration,
) (context.Context, context.CancelFunc) {
if serverSelectionTimeout <= 0 {
return parent, func() {}
}

return context.WithTimeout(parent, serverSelectionTimeout)
}

// ZeroRTTMonitor implements the RTTMonitor interface and is used internally for testing. It returns 0 for all
Expand Down
249 changes: 249 additions & 0 deletions internal/csot/csot_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,249 @@
// Copyright (C) MongoDB, Inc. 2024-present.
//
// Licensed under the Apache License, Version 2.0 (the "License"); you may
// not use this file except in compliance with the License. You may obtain
// a copy of the License at http://www.apache.org/licenses/LICENSE-2.0

package csot

import (
"context"
"testing"
"time"

"go.mongodb.org/mongo-driver/internal/assert"
"go.mongodb.org/mongo-driver/internal/ptrutil"
)

func newTestContext(t *testing.T, timeout time.Duration) context.Context {
ctx, cancel := context.WithTimeout(context.Background(), timeout)
t.Cleanup(cancel)

return ctx
}

func TestWithServerSelectionTimeout(t *testing.T) {
t.Parallel()

tests := []struct {
name string
parent context.Context
serverSelectionTimeout time.Duration
wantTimeout time.Duration
wantOk bool
}{
{
name: "no context deadine and ssto is zero",
parent: context.Background(),
serverSelectionTimeout: 0,
wantTimeout: 0,
wantOk: false,
},
{
name: "no context deadline and ssto is positive",
parent: context.Background(),
serverSelectionTimeout: 1,
wantTimeout: 1,
wantOk: true,
},
{
name: "no context deadline and ssto is negative",
parent: context.Background(),
serverSelectionTimeout: -1,
wantTimeout: 0,
wantOk: false,
},
{
name: "context deadline is zero and ssto is positive",
parent: newTestContext(t, 0),
serverSelectionTimeout: 1,
wantTimeout: 1,
wantOk: true,
},
{
name: "context deadline is zero and ssto is negative",
parent: newTestContext(t, 0),
serverSelectionTimeout: -1,
wantTimeout: 0,
wantOk: true,
},
{
name: "context deadline is negative and ssto is zero",
parent: newTestContext(t, -1),
serverSelectionTimeout: 0,
wantTimeout: -1,
wantOk: true,
},
{
name: "context deadline is negative and ssto is positive",
parent: newTestContext(t, -1),
serverSelectionTimeout: 1,
wantTimeout: 1,
wantOk: true,
},
{
name: "context deadline is negative and ssto is negative",
parent: newTestContext(t, -1),
serverSelectionTimeout: -1,
wantTimeout: -1,
wantOk: true,
},
{
name: "context deadline is positive and ssto is zero",
parent: newTestContext(t, 1),
serverSelectionTimeout: 0,
wantTimeout: 1,
wantOk: true,
},
{
name: "context deadline is positive and equal to ssto",
parent: newTestContext(t, 1),
serverSelectionTimeout: 1,
wantTimeout: 1,
wantOk: true,
},
{
name: "context deadline is positive lt ssto",
parent: newTestContext(t, 1),
serverSelectionTimeout: 2,
wantTimeout: 2,
wantOk: true,
},
{
name: "context deadline is positive gt ssto",
parent: newTestContext(t, 2),
serverSelectionTimeout: 1,
wantTimeout: 2,
wantOk: true,
},
{
name: "context deadline is positive and ssto is negative",
parent: newTestContext(t, -1),
serverSelectionTimeout: -1,
wantTimeout: 1,
wantOk: true,
},
}

for _, test := range tests {
test := test // Capture the range variable

t.Run(test.name, func(t *testing.T) {
t.Parallel()

ctx, cancel := WithServerSelectionTimeout(test.parent, test.serverSelectionTimeout)
t.Cleanup(cancel)

deadline, gotOk := ctx.Deadline()
assert.Equal(t, test.wantOk, gotOk)

if gotOk {
delta := time.Until(deadline) - test.wantTimeout
tolerance := 10 * time.Millisecond

assert.True(t, delta > -1*tolerance, "expected delta=%d > %d", delta, -1*tolerance)
assert.True(t, delta <= tolerance, "expected delta=%d <= %d", delta, tolerance)
}
})
}
}

func TestWithTimeout(t *testing.T) {
t.Parallel()

tests := []struct {
name string
parent context.Context
timeout *time.Duration
wantTimeout time.Duration
wantDeadline bool
wantValues []interface{}
}{
{
name: "deadline set with non-zero timeout",
parent: newTestContext(t, 1),
timeout: ptrutil.Ptr(time.Duration(2)),
wantTimeout: 1,
wantDeadline: true,
wantValues: []interface{}{},
},
{
name: "deadline set with zero timeout",
parent: newTestContext(t, 1),
timeout: ptrutil.Ptr(time.Duration(0)),
wantTimeout: 1,
wantDeadline: true,
wantValues: []interface{}{},
},
{
name: "deadline set with nil timeout",
parent: newTestContext(t, 1),
timeout: nil,
wantTimeout: 1,
wantDeadline: true,
wantValues: []interface{}{},
},
{
name: "deadline unset with non-zero timeout",
parent: context.Background(),
timeout: ptrutil.Ptr(time.Duration(1)),
wantTimeout: 1,
wantDeadline: true,
wantValues: []interface{}{},
},
{
name: "deadline unset with zero timeout",
parent: context.Background(),
timeout: ptrutil.Ptr(time.Duration(0)),
wantTimeout: 0,
wantDeadline: false,
wantValues: []interface{}{clientLevel{}},
},
{
name: "deadline unset with nil timeout",
parent: context.Background(),
timeout: nil,
wantTimeout: 0,
wantDeadline: false,
wantValues: []interface{}{},
},
{
// If "clientLevel" has been set, but a new timeout is applied
// to the context, then the constructed context should retain the old
// timeout. To simplify the code, we assume the first timeout is static.
name: "deadline unset with non-zero timeout at clientLevel",
parent: context.WithValue(context.Background(), clientLevel{}, true),
timeout: ptrutil.Ptr(time.Duration(1)),
wantTimeout: 0,
wantDeadline: false,
wantValues: []interface{}{},
},
}

for _, test := range tests {
test := test // Capture the range variable

t.Run(test.name, func(t *testing.T) {
t.Parallel()

ctx, cancel := WithTimeout(test.parent, test.timeout)
t.Cleanup(cancel)

deadline, gotDeadline := ctx.Deadline()
assert.Equal(t, test.wantDeadline, gotDeadline)

if gotDeadline {
delta := time.Until(deadline) - test.wantTimeout
tolerance := 10 * time.Millisecond

assert.True(t, delta > -1*tolerance, "expected delta=%d > %d", delta, -1*tolerance)
assert.True(t, delta <= tolerance, "expected delta=%d <= %d", delta, tolerance)
}

for _, wantValue := range test.wantValues {
assert.NotNil(t, ctx.Value(wantValue), "expected context to have value %v", wantValue)
}
})
}

}
2 changes: 0 additions & 2 deletions internal/docexamples/examples.go
Original file line number Diff line number Diff line change
Expand Up @@ -1971,7 +1971,6 @@ func WithTransactionExample(ctx context.Context) error {

// Prereq: Create collections.
wcMajority := writeconcern.Majority()
wcMajority.WTimeout = 1 * time.Second
wcMajorityCollectionOpts := options.Collection().SetWriteConcern(wcMajority)
fooColl := client.Database("mydb1").Collection("foo", wcMajorityCollectionOpts)
barColl := client.Database("mydb1").Collection("bar", wcMajorityCollectionOpts)
Expand Down Expand Up @@ -2552,7 +2551,6 @@ func CausalConsistencyExamples(client *mongo.Client) error {

rc := readconcern.Majority()
wc := writeconcern.Majority()
wc.WTimeout = 1000
// Use a causally-consistent session to run some operations
opts := options.Session().SetDefaultReadConcern(rc).SetDefaultWriteConcern(wc)
session1, err := client.StartSession(opts)
Expand Down
12 changes: 0 additions & 12 deletions internal/integration/client_side_encryption_prose_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -56,16 +56,6 @@ const (
maxBsonObjSize = 16777216 // max bytes in BSON object
)

func containsSubstring(possibleSubstrings []string, str string) bool {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved this helper function to errors_test.go since it's not accessible there without being in the cse build set.

for _, possibleSubstring := range possibleSubstrings {
if strings.Contains(str, possibleSubstring) {
return true
}
}

return false
}

func TestClientSideEncryptionProse(t *testing.T) {
t.Parallel()

Expand Down Expand Up @@ -153,7 +143,6 @@ func TestClientSideEncryptionProse(t *testing.T) {

// Insert the copied key document into keyvault.datakeys with majority write concern.
wcMajority := writeconcern.Majority()
wcMajority.WTimeout = 1 * time.Second
wcMajorityCollectionOpts := options.Collection().SetWriteConcern(wcMajority)
wcmColl := cse.kvClient.Database(kvDatabase).Collection(dkCollection, wcMajorityCollectionOpts)
_, err = wcmColl.InsertOne(context.Background(), alteredKeydoc)
Expand Down Expand Up @@ -1891,7 +1880,6 @@ func TestClientSideEncryptionProse(t *testing.T) {
}

wcMajority := writeconcern.Majority()
wcMajority.WTimeout = 1 * time.Second
wcMajorityCollectionOpts := options.Collection().SetWriteConcern(wcMajority)
wcmColl := cse.kvClient.Database(kvDatabase).Collection(dkCollection, wcMajorityCollectionOpts)
_, err = wcmColl.Indexes().CreateOne(context.Background(), keyVaultIndex)
Expand Down
2 changes: 1 addition & 1 deletion internal/integration/client_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -327,7 +327,7 @@ func TestClient(t *testing.T) {
// apply the correct URI.
invalidClientOpts := options.Client().
SetServerSelectionTimeout(100 * time.Millisecond).SetHosts([]string{"invalid:123"}).
SetConnectTimeout(500 * time.Millisecond).SetSocketTimeout(500 * time.Millisecond)
SetConnectTimeout(500 * time.Millisecond).SetTimeout(500 * time.Millisecond)
integtest.AddTestServerAPIVersion(invalidClientOpts)
client, err := mongo.Connect(invalidClientOpts)
assert.Nil(mt, err, "Connect error: %v", err)
Expand Down