Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GODRIVER-2348 Make CSOT feature-gated behavior the default #1515

Open
wants to merge 68 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 54 commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
76fe99a
GODRIVER-2762 Use minimum RTT for CSOT maxTimeMS calculation instead …
prestonvasquez Dec 21, 2023
c96883a
GODRIVER-2762 Clean up rtt monitor code
prestonvasquez Dec 21, 2023
88e9f0d
GODRIVER-2762 Add command-execution.yml
prestonvasquez Dec 21, 2023
85812f6
GODRIVER-2762 Fix typo
prestonvasquez Dec 21, 2023
1a0517c
Merge branch 'master' into GODRIVER-2762
prestonvasquez Dec 29, 2023
49298ee
GODRIVER-2762 PR requests
prestonvasquez Dec 29, 2023
04aeb6d
GODRIVER-2762 Remove reference to RTT90 in max time calc
prestonvasquez Jan 5, 2024
2d61660
GODRIVER-2348 connectTimeoutMS to cover all blocking operations durin…
prestonvasquez Jan 11, 2024
0ba0d69
GODRIVER-2762 Fix rounding, cleanup deadline exceeded error
prestonvasquez Jan 11, 2024
bccb529
Merge branch 'master' into GODRIVER-2762
prestonvasquez Jan 12, 2024
f7d8111
GODRIVER-2348 merge master
prestonvasquez Jan 13, 2024
698e4e0
GODRIVER-2762 Use args in wait event
prestonvasquez Jan 13, 2024
8b10b54
Merge branch 'GODRIVER-2762' of github.com:prestonvasquez/mongo-go-dr…
prestonvasquez Jan 13, 2024
ce0c7c2
Update x/mongo/driver/topology/rtt_monitor.go
prestonvasquez Jan 13, 2024
6095151
Merge branch 'master' into GODRIVER-2762
prestonvasquez Jan 13, 2024
a2f0c9a
GODRIVER-2348 Merge 2762
prestonvasquez Jan 16, 2024
d10ced0
GODRIVER-2348 merge master
prestonvasquez Jan 16, 2024
9cb8028
Merge branch 'master' into GODRIVER-2348
prestonvasquez Jan 18, 2024
4669d9b
GODRIVER-2466 Implement assertEventCount
prestonvasquez Jan 19, 2024
7aa63f0
GODRIVER-2466 Fix typo
prestonvasquez Jan 19, 2024
45c6725
Merge remote-tracking branch 'origin/GODRIVER-2466' into GODRIVER-2348
prestonvasquez Jan 19, 2024
998b725
GODRIVER-2466 Remove duplicate failpoint warning
prestonvasquez Jan 19, 2024
15c6d42
Merge remote-tracking branch 'origin/GODRIVER-2466' into GODRIVER-2348
prestonvasquez Jan 19, 2024
9af952f
GODRIVER-2348 Update connectTimeoutMS to cover all blocking operation…
prestonvasquez Jan 19, 2024
2d53d6a
GODRIVER-2348 Deprecate socketTimeoutMS
prestonvasquez Jan 23, 2024
d2b8c50
Merge branch 'master' into GODRIVER-2348
prestonvasquez Jan 23, 2024
9c17094
Merge branch 'master' into GODRIVER-2348
prestonvasquez Jan 23, 2024
d9d61f4
GODRIVER-2348 Remove rw connection timeouts
prestonvasquez Jan 25, 2024
51d6c1b
Merge branch 'GODRIVER-2348' of github.com:prestonvasquez/mongo-go-dr…
prestonvasquez Jan 25, 2024
25368d6
Merge branch 'master' into GODRIVER-2348
prestonvasquez Jan 26, 2024
c32af34
GODRIVER-2348 Deprecate wtimeout
prestonvasquez Jan 30, 2024
54abbc5
GODRIVER-2348 Deprecate maxTimeMS
prestonvasquez Feb 1, 2024
a4c7211
Merge branch 'master' into GODRIVER-2348
prestonvasquez Feb 1, 2024
49427b3
GODRIVER-2348 Deprecate maxCommitTimeMS
prestonvasquez Feb 1, 2024
4a55e76
GODRIVER-2348 Cont. Deprecating MaxTimeMS
prestonvasquez Feb 2, 2024
9293b84
GODRIVER-2348 Implement server selection logic
prestonvasquez Feb 3, 2024
023a1d1
Merge branch 'master' into GODRIVER-2348
prestonvasquez Feb 6, 2024
f6a4f97
Merge branch 'master' into GODRIVER-2348
prestonvasquez Feb 7, 2024
2501fe8
GODRIVER-2348 Cont. implementing Server Selection req
prestonvasquez Feb 8, 2024
a56844e
GODRIVER-2348 Remove unused topology struct
prestonvasquez Feb 8, 2024
7b64cee
GODRIVER-2348 Rename maxTimeMS for cursor to maxAwaitTimeMS
prestonvasquez Feb 9, 2024
11d1f3b
GODRIVER-2348 Add retryability-legacy-timeouts back to UST
prestonvasquez Feb 9, 2024
6892b1f
GODRIVER-2348 Resolve connstring merge conflicts
prestonvasquez Feb 9, 2024
15d80e0
GODRIVER-2348 Implement change stream CSOT conditions
prestonvasquez Feb 10, 2024
99b4389
GODRIVER-2348 Remove CSOT-specific context opts
prestonvasquez Feb 13, 2024
9f4a8e5
GODRIVER-2348 Remove debug tools
prestonvasquez Feb 13, 2024
e69bc0a
GODRIVER-2348 Extend test coverage for csot
prestonvasquez Feb 13, 2024
16f1027
GODRIVER-2348 Resolve race condition
prestonvasquez Feb 14, 2024
818f7d1
GODRIVER-2348 Resolve merge conflict
prestonvasquez Feb 14, 2024
27ae440
GODRIVER-2348 Refine csot logic
prestonvasquez Feb 20, 2024
2c94bd9
GODRIVER-2348 Fix linting errors
prestonvasquez Feb 21, 2024
f65fb19
GODRIVER-2348 Remove lingering comments
prestonvasquez Feb 21, 2024
75e44e2
Merge branch 'master' into GODRIVER-2348
prestonvasquez Feb 21, 2024
4424192
GODRIVER-2348 Fix network error pool test
prestonvasquez Feb 21, 2024
9a5df59
Merge branch 'master' into GODRIVER-2348
prestonvasquez Feb 26, 2024
3043a64
GODRIVER-2348 Resolve PR conversations
prestonvasquez Feb 27, 2024
b2e8844
Update internal/csot/csot.go
prestonvasquez Feb 27, 2024
884f2ba
Merge branch 'master' into GODRIVER-2348
prestonvasquez Mar 8, 2024
db69938
GODRIVER-2348 Resolve merge conflict
prestonvasquez Mar 18, 2024
058b343
Merge branch 'GODRIVER-2348' of github.com:prestonvasquez/mongo-go-dr…
prestonvasquez Mar 18, 2024
00fb9da
GODRIVER-2348 Do not set transactional wtimeout
prestonvasquez Mar 19, 2024
b191ec7
GODRIVER-2348 Merge master
prestonvasquez May 1, 2024
755204f
Update mongo/client.go
prestonvasquez May 1, 2024
8e26315
Update x/mongo/driver/topology/topology_test.go
prestonvasquez May 1, 2024
2aff850
GODRIVER-2348 Resolve various PR issues
prestonvasquez May 8, 2024
731aca7
Merge branch 'GODRIVER-2348' of github.com:prestonvasquez/mongo-go-dr…
prestonvasquez May 8, 2024
3b40fa4
GODRIVER-2348 Resolve build failure
prestonvasquez May 9, 2024
1620f92
GODRIVER-2348 Resolve merge conflicts
prestonvasquez May 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
115 changes: 100 additions & 15 deletions internal/csot/csot.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,26 +11,111 @@ import (
"time"
)

type timeoutKey struct{}
type withoutMaxTime struct{}

// MakeTimeoutContext returns a new context with Client-Side Operation Timeout (CSOT) feature-gated behavior
// and a Timeout set to the passed in Duration. Setting a Timeout on a single operation is not supported in
// public API.
//
// TODO(GODRIVER-2348) We may be able to remove this function once CSOT feature-gated behavior becomes the
// TODO default behavior.
func MakeTimeoutContext(ctx context.Context, to time.Duration) (context.Context, context.CancelFunc) {
// Only use the passed in Duration as a timeout on the Context if it
// is non-zero.
cancelFunc := func() {}
if to != 0 {
ctx, cancelFunc = context.WithTimeout(ctx, to)
// WithoutMaxTime returns a new context with a "withoutMaxTime" value that
// is used to inform operation construction to not include "maxTimeMS" in a wire
// message, regardless of a context deadline. This is specifically used for
// non-awaitable hello commands.
func WithoutMaxTime(ctx context.Context) context.Context {
return context.WithValue(ctx, withoutMaxTime{}, true)
}

// IsWithoutMaxTime checks if the provided context has been assigned the
// "withoutMaxTime" value.
func IsWithoutMaxTime(ctx context.Context) bool {
return ctx.Value(withoutMaxTime{}) != nil
}

type clientLevel struct{}

func AsClientLevel(parent context.Context) context.Context {
return context.WithValue(parent, clientLevel{}, true)
}
matthewdale marked this conversation as resolved.
Show resolved Hide resolved

func IsClientLevel(ctx context.Context) bool {
matthewdale marked this conversation as resolved.
Show resolved Hide resolved
val := ctx.Value(clientLevel{})
if val == nil {
return false
}
return context.WithValue(ctx, timeoutKey{}, true), cancelFunc

return val.(bool)
}

// IsTimeoutContext checks if the provided context has been assigned a deadline
// or has unlimited retries.
func IsTimeoutContext(ctx context.Context) bool {
return ctx.Value(timeoutKey{}) != nil
_, ok := ctx.Deadline()

return ok || IsClientLevel(ctx)
}

// WithTimeout will set the given timeout on the context, if no deadline has
// already been set.
//
// This function assumes that the timeout field is static, given that the
// timeout should be sourced from the client. Therefore, once a timeout function
// parameter has been applied to the context, it will remain for the lifetime
// of the context.
func WithTimeout(parent context.Context, timeout *time.Duration) (context.Context, context.CancelFunc) {
prestonvasquez marked this conversation as resolved.
Show resolved Hide resolved
cancel := func() {}

// In the following conditions, do nothing:
// 1. The parent already has a deadline
// 2. The parent does not have a deadline, but a client-level timeout has
// been applied.
// 3. The parent does not have a deadline, there is not client-level timeout,
// and the timeout parameter DNE.
if IsTimeoutContext(parent) || timeout == nil {
prestonvasquez marked this conversation as resolved.
Show resolved Hide resolved
return parent, cancel
}

matthewdale marked this conversation as resolved.
Show resolved Hide resolved
// If a client-level timeout has not been applied, then apply it.
parent = AsClientLevel(parent)

// If the parent does not have a deadline and the timeout is zero, then
// set the zero value.
if timeout != nil && *timeout == 0 {
prestonvasquez marked this conversation as resolved.
Show resolved Hide resolved
return parent, cancel
}

// If the parent does not have a dealine and the timeout is non-zero, then
// apply the timeout.
return context.WithTimeout(parent, *timeout)
}

// WithServerSelectionTimeout creates a context with a timeout that is the
// minimum of serverSelectionTimeoutMS and context deadline. The usage of
// non-positive values for serverSelectionTimeoutMS are an anti-pattern and are
// not considered in this calculation.
func WithServerSelectionTimeout(
matthewdale marked this conversation as resolved.
Show resolved Hide resolved
parent context.Context,
serverSelectionTimeout time.Duration,
) (context.Context, context.CancelFunc) {
var timeout time.Duration

deadline, ok := parent.Deadline()
if ok {
timeout = time.Until(deadline)
}

// If there is no deadline on the parent context and the server selection
// timeout DNE, then do nothing.
if !ok && serverSelectionTimeout <= 0 {
return parent, func() {}
}

// Otherwise, take the minimum of the two and return a new context with that
// value as the deadline.
if !ok {
timeout = serverSelectionTimeout
} else if timeout >= serverSelectionTimeout && serverSelectionTimeout > 0 {
// Only use the serverSelectionTimeout value if it is less than the existing
// timeout and is positive.
timeout = serverSelectionTimeout
}

return context.WithTimeout(parent, timeout)
}

// ZeroRTTMonitor implements the RTTMonitor interface and is used internally for testing. It returns 0 for all
Expand Down
252 changes: 252 additions & 0 deletions internal/csot/csot_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,252 @@
// Copyright (C) MongoDB, Inc. 2024-present.
//
// Licensed under the Apache License, Version 2.0 (the "License"); you may
// not use this file except in compliance with the License. You may obtain
// a copy of the License at http://www.apache.org/licenses/LICENSE-2.0

package csot

import (
"context"
"testing"
"time"

"go.mongodb.org/mongo-driver/internal/assert"
)

func newTestContext(t *testing.T, timeout time.Duration) context.Context {
ctx, cancel := context.WithTimeout(context.Background(), timeout)
t.Cleanup(cancel)

return ctx
}

func newDurPtr(dur time.Duration) *time.Duration {
return &dur
}
matthewdale marked this conversation as resolved.
Show resolved Hide resolved

func TestWithServerSelectionTimeout(t *testing.T) {
t.Parallel()

tests := []struct {
name string
parent context.Context
serverSelectionTimeout time.Duration
wantTimeout time.Duration
wantOk bool
}{
{
name: "no context deadine and ssto is zero",
parent: context.Background(),
serverSelectionTimeout: 0,
wantTimeout: 0,
wantOk: false,
},
{
name: "no context deadline and ssto is positive",
parent: context.Background(),
serverSelectionTimeout: 1,
wantTimeout: 1,
wantOk: true,
},
{
name: "no context deadline and ssto is negative",
parent: context.Background(),
serverSelectionTimeout: -1,
wantTimeout: 0,
wantOk: false,
},
{
name: "context deadline is zero and ssto is positive",
parent: newTestContext(t, 0),
serverSelectionTimeout: 1,
wantTimeout: 1,
wantOk: true,
},
{
name: "context deadline is zero and ssto is negative",
parent: newTestContext(t, 0),
serverSelectionTimeout: -1,
wantTimeout: 0,
wantOk: true,
},
{
name: "context deadline is negative and ssto is zero",
parent: newTestContext(t, -1),
serverSelectionTimeout: 0,
wantTimeout: -1,
wantOk: true,
},
{
name: "context deadline is negative and ssto is positive",
parent: newTestContext(t, -1),
serverSelectionTimeout: 1,
wantTimeout: 1,
wantOk: true,
},
{
name: "context deadline is negative and ssto is negative",
parent: newTestContext(t, -1),
serverSelectionTimeout: -1,
wantTimeout: -1,
wantOk: true,
},
{
name: "context deadline is positive and ssto is zero",
parent: newTestContext(t, 1),
serverSelectionTimeout: 0,
wantTimeout: 1,
wantOk: true,
},
{
name: "context deadline is positive and equal to ssto",
parent: newTestContext(t, 1),
serverSelectionTimeout: 1,
wantTimeout: 1,
wantOk: true,
},
{
name: "context deadline is positive lt ssto",
parent: newTestContext(t, 1),
serverSelectionTimeout: 2,
wantTimeout: 2,
wantOk: true,
},
{
name: "context deadline is positive gt ssto",
parent: newTestContext(t, 2),
serverSelectionTimeout: 1,
wantTimeout: 2,
wantOk: true,
},
{
name: "context deadline is positive and ssto is negative",
parent: newTestContext(t, -1),
serverSelectionTimeout: -1,
wantTimeout: 1,
wantOk: true,
},
}

for _, test := range tests {
test := test // Capture the range variable

t.Run(test.name, func(t *testing.T) {
t.Parallel()

ctx, cancel := WithServerSelectionTimeout(test.parent, test.serverSelectionTimeout)
t.Cleanup(cancel)

deadline, gotOk := ctx.Deadline()
assert.Equal(t, test.wantOk, gotOk)

if gotOk {
delta := time.Until(deadline) - test.wantTimeout
tolerance := 10 * time.Millisecond

assert.True(t, delta > -1*tolerance, "expected delta=%d > %d", delta, -1*tolerance)
assert.True(t, delta <= tolerance, "expected delta=%d <= %d", delta, tolerance)
}
})
}
}

func TestWithTimeout(t *testing.T) {
t.Parallel()

tests := []struct {
name string
parent context.Context
timeout *time.Duration
wantTimeout time.Duration
wantDeadline bool
wantValues []interface{}
}{
{
name: "deadline set with non-zero timeout",
parent: newTestContext(t, 1),
timeout: newDurPtr(2),
wantTimeout: 1,
wantDeadline: true,
wantValues: []interface{}{},
},
{
name: "deadline set with zero timeout",
parent: newTestContext(t, 1),
timeout: newDurPtr(0),
wantTimeout: 1,
wantDeadline: true,
wantValues: []interface{}{},
},
{
name: "deadline set with nil timeout",
parent: newTestContext(t, 1),
timeout: nil,
wantTimeout: 1,
wantDeadline: true,
wantValues: []interface{}{},
},
{
name: "deadline unset with non-zero timeout",
parent: context.Background(),
timeout: newDurPtr(1),
wantTimeout: 1,
wantDeadline: true,
wantValues: []interface{}{},
},
{
name: "deadline unset with zero timeout",
parent: context.Background(),
timeout: newDurPtr(0),
wantTimeout: 0,
wantDeadline: false,
wantValues: []interface{}{clientLevel{}},
},
{
name: "deadline unset with nil timeout",
parent: context.Background(),
timeout: nil,
wantTimeout: 0,
wantDeadline: false,
wantValues: []interface{}{},
},
{
// If "clientLevel" has been set, but a new timeout is applied
// to the context, then the constructed context should retain the old
// timeout. To simplify the code, we assume the first timeout is static.
name: "deadline unset with non-zero timeout at clientLevel",
parent: AsClientLevel(context.Background()),
timeout: newDurPtr(1),
wantTimeout: 0,
wantDeadline: false,
wantValues: []interface{}{},
},
}

for _, test := range tests {
test := test // Capture the range variable

t.Run(test.name, func(t *testing.T) {
t.Parallel()

ctx, cancel := WithTimeout(test.parent, test.timeout)
t.Cleanup(cancel)

deadline, gotDeadline := ctx.Deadline()
assert.Equal(t, test.wantDeadline, gotDeadline)

if gotDeadline {
delta := time.Until(deadline) - test.wantTimeout
tolerance := 10 * time.Millisecond

assert.True(t, delta > -1*tolerance, "expected delta=%d > %d", delta, -1*tolerance)
assert.True(t, delta <= tolerance, "expected delta=%d <= %d", delta, tolerance)
}

for _, wantValue := range test.wantValues {
assert.NotNil(t, ctx.Value(wantValue), "expected context to have value %v", wantValue)
}
})
}

}
2 changes: 0 additions & 2 deletions internal/docexamples/examples.go
Original file line number Diff line number Diff line change
Expand Up @@ -1971,7 +1971,6 @@ func WithTransactionExample(ctx context.Context) error {

// Prereq: Create collections.
wcMajority := writeconcern.Majority()
wcMajority.WTimeout = 1 * time.Second
wcMajorityCollectionOpts := options.Collection().SetWriteConcern(wcMajority)
fooColl := client.Database("mydb1").Collection("foo", wcMajorityCollectionOpts)
barColl := client.Database("mydb1").Collection("bar", wcMajorityCollectionOpts)
Expand Down Expand Up @@ -2552,7 +2551,6 @@ func CausalConsistencyExamples(client *mongo.Client) error {

rc := readconcern.Majority()
wc := writeconcern.Majority()
wc.WTimeout = 1000
// Use a causally-consistent session to run some operations
opts := options.Session().SetDefaultReadConcern(rc).SetDefaultWriteConcern(wc)
session1, err := client.StartSession(opts)
Expand Down