Prepare v1.21.0 release (#4195)

CHANGELOG.md

@@ -2,6 +2,77 @@
 
 <!-- markdownlint-disable MD024 MD034 -->
 
+## [v1.21.0](https://github.com/FerretDB/FerretDB/releases/tag/v1.21.0) (2024-02-20)
+
+### New Features 🎉
+
+- Add experimental `SCRAM-SHA-1`/`SCRAM-SHA-256` authentication support by @henvic in https://github.com/FerretDB/FerretDB/pull/4078
+
+### Fixed Bugs 🐛
+
+- Reorganize and fix `update`/`upsert` logic by @wazir-ahmed in https://github.com/FerretDB/FerretDB/pull/4069
+
+### Enhancements 🛠
+
+- Improve capped collection cleanup by @wazir-ahmed in https://github.com/FerretDB/FerretDB/pull/4118
+- Make batch sizes configurable by @kropidlowsky in https://github.com/FerretDB/FerretDB/pull/4149
+
+### Documentation 📄
+
+- Fix Codapi file error by @Fashander in https://github.com/FerretDB/FerretDB/pull/4077
+- Add Tembo QA blog post by @Fashander in https://github.com/FerretDB/FerretDB/pull/4081
+- Update correct image link by @Fashander in https://github.com/FerretDB/FerretDB/pull/4116
+- Add Pulumi blog post by @Fashander in https://github.com/FerretDB/FerretDB/pull/4102
+- Add Tembo to README by @AlekSi in https://github.com/FerretDB/FerretDB/pull/4168
+- Remove some closed issues from documentation by @AlekSi in https://github.com/FerretDB/FerretDB/pull/4172
+
+### Other Changes 🤖
+
+- Use Go 1.22 and bump deps by @AlekSi in https://github.com/FerretDB/FerretDB/pull/4094
+- Add more fields to requests and responses by @rumyantseva in https://github.com/FerretDB/FerretDB/pull/4096
+- Revert SQLite version bump by @AlekSi in https://github.com/FerretDB/FerretDB/pull/4106
+- Refactor `bson2` package by @AlekSi in https://github.com/FerretDB/FerretDB/pull/4105
+- Use `bson2` package for wire queries and replies by @AlekSi in https://github.com/FerretDB/FerretDB/pull/4108
+- Make logger configurable in the embedded `ferretdb` package by @fadyat in https://github.com/FerretDB/FerretDB/pull/4028
+- Fix `envtool run test` `-run` and `-skip` flags by @henvic in https://github.com/FerretDB/FerretDB/pull/4101
+- Add MySQL backend collection by @adetunjii in https://github.com/FerretDB/FerretDB/pull/4083
+- Ignore `maxTimeMS` argument in `count`, `insert`, `update`, `delete` by @farit2000 in https://github.com/FerretDB/FerretDB/pull/4121
+- Use correct salt length by @AlekSi in https://github.com/FerretDB/FerretDB/pull/4126
+- Skip stuck tailable cursor test by @chilagrow in https://github.com/FerretDB/FerretDB/pull/4131
+- Enforce new authentication by @chilagrow in https://github.com/FerretDB/FerretDB/pull/4075
+- Replace `bson` with `bson2` in `wire` by @AlekSi in https://github.com/FerretDB/FerretDB/pull/4110
+- Improve `OP_MSG` validity checks by @AlekSi in https://github.com/FerretDB/FerretDB/pull/4135
+- Support speculative authenticate by @chilagrow in https://github.com/FerretDB/FerretDB/pull/4111
+- Add MySQL backend by @adetunjii in https://github.com/FerretDB/FerretDB/pull/4137
+- Fix `saslContinue` crashing due to not found authentication conversation by @henvic in https://github.com/FerretDB/FerretDB/pull/4129
+- Cleanup TODO for speculative authenticate by @chilagrow in https://github.com/FerretDB/FerretDB/pull/4143
+- Fix MySQL collection stats by @adetunjii in https://github.com/FerretDB/FerretDB/pull/4145
+- Use Go 1.22.1 by @AlekSi in https://github.com/FerretDB/FerretDB/pull/4155
+- Advertise SCRAM / SASL support in addition to PLAIN by @henvic in https://github.com/FerretDB/FerretDB/pull/4113
+- Add linter to check truncate tag in blog posts by @sbshah97 in https://github.com/FerretDB/FerretDB/pull/4139
+- Fix PLAIN mechanism authentication incorrectly working by @chilagrow in https://github.com/FerretDB/FerretDB/pull/4163
+- Improve `bson2` and `wire` logging by @AlekSi in https://github.com/FerretDB/FerretDB/pull/4148
+- Fix logging of deeply nested documents by @AlekSi in https://github.com/FerretDB/FerretDB/pull/4167
+- Support localhost exception by @chilagrow in https://github.com/FerretDB/FerretDB/pull/4156
+- Do not use the flow style in the diff output by @AlekSi in https://github.com/FerretDB/FerretDB/pull/4170
+- Do not use `fjson` by @AlekSi in https://github.com/FerretDB/FerretDB/pull/4175
+- Remove `fjson` package by @AlekSi in https://github.com/FerretDB/FerretDB/pull/4176
+- Fix `speculativeAuthenticate` panic on empty database by @chilagrow in https://github.com/FerretDB/FerretDB/pull/4178
+- Move old `bson` package by @AlekSi in https://github.com/FerretDB/FerretDB/pull/4177
+- Rename `bson2` to `bson` by @AlekSi in https://github.com/FerretDB/FerretDB/pull/4179
+- Move Docker build files by @AlekSi in https://github.com/FerretDB/FerretDB/pull/4180
+- Bump protobuf dependency to make CI happy by @AlekSi in https://github.com/FerretDB/FerretDB/pull/4187
+- Use authentication enabled docker for integration test by @chilagrow in https://github.com/FerretDB/FerretDB/pull/4160
+- Bump `pgx` by @AlekSi in https://github.com/FerretDB/FerretDB/pull/4190
+
+### New Contributors
+
+- @farit2000 made their first contribution in https://github.com/FerretDB/FerretDB/pull/4121
+- @sbshah97 made their first contribution in https://github.com/FerretDB/FerretDB/pull/4139
+
+[All closed issues and pull requests](https://github.com/FerretDB/FerretDB/milestone/63?closed=1).
+[All commits](https://github.com/FerretDB/FerretDB/compare/v1.20.1...v1.21.0).
+
 ## [v1.20.1](https://github.com/FerretDB/FerretDB/releases/tag/v1.20.1) (2024-02-19)
 
 ### What's Changed

internal/handler/msg_getlog.go

@@ -113,14 +113,17 @@ func (h *Handler) MsgGetLog(ctx context.Context, msg *wire.OpMsg) (*wire.OpMsg,
 				"The telemetry state is undecided.",
 				"Read more about FerretDB telemetry and how to opt out at https://beacon.ferretdb.com.",
 			)
-		case state.UpdateAvailable:
-			startupWarnings = append(
-				startupWarnings,
-				fmt.Sprintf(
+
+		case state.UpdateInfo != "", state.UpdateAvailable:
+			msg := state.UpdateInfo
+			if msg == "" {
+				msg = fmt.Sprintf(
 					"A new version available! The latest version: %s. The current version: %s.",
 					state.LatestVersion, info.Version,
-				),
-			)
+				)
+			}
+
+			startupWarnings = append(startupWarnings, msg)
 		}
 
 		var log types.Array

internal/util/state/state.go

@@ -38,6 +38,7 @@ type State struct {
 
 	// as reported by beacon, if known
 	LatestVersion   string `json:"-"`
+	UpdateInfo      string `json:"-"`
 	UpdateAvailable bool   `json:"-"`
 }
 
@@ -56,11 +57,11 @@ func (s *State) TelemetryString() string {
 
 // DisableTelemetry disables telemetry.
 //
-// It also sets LatestVersion and UpdateAvailable to zero values
-// to avoid stale values when telemetry is re-enabled.
+// It also resets other telemetry fields to avoid stale values when telemetry is re-enabled.
 func (s *State) DisableTelemetry() {
 	s.Telemetry = pointer.ToBool(false)
 	s.LatestVersion = ""
+	s.UpdateInfo = ""
 	s.UpdateAvailable = false
 }
 
@@ -95,6 +96,7 @@ func (s *State) deepCopy() *State {
 		BackendName:     s.BackendName,
 		BackendVersion:  s.BackendVersion,
 		LatestVersion:   s.LatestVersion,
+		UpdateInfo:      s.UpdateInfo,
 		UpdateAvailable: s.UpdateAvailable,
 	}
 }

internal/util/telemetry/reporter.go

@@ -64,6 +64,7 @@ type request struct {
 // response represents telemetry response.
 type response struct {
 	LatestVersion   string `json:"latest_version"`
+	UpdateInfo      string `json:"update_info"`
 	UpdateAvailable bool   `json:"update_available"`
 }
 
@@ -289,23 +290,21 @@ func (r *Reporter) report(ctx context.Context) {
 
 	r.L.Debug("Read telemetry response.", zap.Any("response", response))
 
-	if response.LatestVersion == "" {
-		r.L.Debug("No latest version in telemetry response.")
-		return
+	if response.UpdateInfo != "" || response.UpdateAvailable {
+		msg := response.UpdateInfo
+		if msg == "" {
+			msg = "A new version available!"
+		}
+
+		r.L.Info(msg, zap.String("current_version", request.Version), zap.String("latest_version", response.LatestVersion))
 	}
 
 	if err = r.P.Update(func(s *state.State) {
 		s.LatestVersion = response.LatestVersion
+		s.UpdateInfo = response.UpdateInfo
 		s.UpdateAvailable = response.UpdateAvailable
 	}); err != nil {
 		r.L.Error("Failed to update state with latest version.", zap.Error(err))
 		return
 	}
-
-	if s.UpdateAvailable {
-		r.L.Info(
-			"A new version available!",
-			zap.String("current_version", request.Version), zap.String("latest_version", s.LatestVersion),
-		)
-	}
 }

internal/util/telemetry/reporter_test.go

@@ -138,61 +138,51 @@ func TestReporterReport(t *testing.T) {
 		r, err := NewReporter(&opts)
 		require.NoError(t, err)
 
-		// Check the initial state of the provider, it has not called telemetry yet,
-		// no update is available and unaware of the latest version.
 		s := r.P.Get()
 		assert.False(t, s.UpdateAvailable)
 		assert.Empty(t, s.LatestVersion)
 
-		// Call the telemetry server and check the state of the provider to be updated.
 		r.report(testutil.Ctx(t))
 		assert.Equal(t, 1, serverCalled)
+
 		s = r.P.Get()
 		assert.True(t, s.UpdateAvailable)
 		assert.Equal(t, "v1.2.1", s.LatestVersion)
 
-		// Set update available to false on the beacon side, and call the telemetry server again.
 		telemetryResponse.UpdateAvailable = false
+
 		r.report(testutil.Ctx(t))
 		assert.Equal(t, 2, serverCalled)
 
-		// Expect the state of provider to be updated.
 		s = r.P.Get()
 		assert.False(t, s.UpdateAvailable)
 		assert.Equal(t, "v1.2.1", s.LatestVersion)
 
-		// Set update available to true and update version, and call the telemetry server again.
 		telemetryResponse.UpdateAvailable = true
 		telemetryResponse.LatestVersion = "v1.2.0"
+
 		r.report(testutil.Ctx(t))
 		assert.Equal(t, 3, serverCalled)
 
-		// Expect the state and the version to be updated.
 		s = r.P.Get()
 		assert.True(t, s.UpdateAvailable)
 		assert.Equal(t, "v1.2.0", s.LatestVersion)
 
-		// Disable telemetry and call the telemetry server again.
 		require.NoError(t, sp.Update(func(s *state.State) { s.DisableTelemetry() }))
-		r.report(testutil.Ctx(t))
 
-		// Expect no call to the telemetry server (number of calls should not change).
+		r.report(testutil.Ctx(t))
 		assert.Equal(t, 3, serverCalled)
 
-		// Expect no update available and latest version equal to the previous state.
 		s = r.P.Get()
 		assert.False(t, s.UpdateAvailable)
 		assert.Empty(t, s.LatestVersion)
 
-		// Enable telemetry
 		require.NoError(t, sp.Update(func(s *state.State) { s.EnableTelemetry() }))
-
-		// Set a newer version to expect.
 		telemetryResponse.LatestVersion = "v1.2.2"
+
 		r.report(testutil.Ctx(t))
 		assert.Equal(t, 4, serverCalled)
 
-		// Expect no update available and latest version equal to the previous state.
 		s = r.P.Get()
 		assert.True(t, s.UpdateAvailable)
 		assert.Equal(t, "v1.2.2", s.LatestVersion)
@@ -223,16 +213,11 @@ func TestReporterReport(t *testing.T) {
 		r, err := NewReporter(&opts)
 		require.NoError(t, err)
 
-		// Check the initial state of the provider, it has not called telemetry yet,
-		// no update is available and unaware of the latest version.
 		s := r.P.Get()
 		assert.False(t, s.UpdateAvailable)
 		assert.Empty(t, s.LatestVersion)
 
-		// Call the telemetry server, as telemetry is disabled, expect no update to the provider.
 		r.report(testutil.Ctx(t))
-
-		// Expect no call to the telemetry server (number of calls should not change).
 		assert.Equal(t, 0, serverCalled)
 
 		s = r.P.Get()

website/docusaurus.config.js

@@ -49,7 +49,7 @@ const config = {
 
           versions: {
             // the latest minus one minor
-            'v1.19': {
+            'v1.20': {
               banner: 'none',
             },
           },

website/versioned_docs/version-v1.18/configuration/operation-modes.md → website/versioned_docs/version-v1.21/configuration/operation-modes.md

@@ -35,15 +35,16 @@ Header diff:
 --- res header
 +++ proxy header
 @@ -1 +1 @@
--length:    63, id:   14, response_to:   24, opcode: OP_MSG
-+length:    64, id:  229, response_to:   24, opcode: OP_MSG
+-length:    63, id:    4, response_to:   13, opcode: OP_MSG
++length:   191, id:   53, response_to:   13, opcode: OP_MSG
 
 Body diff:
 --- res body
 +++ proxy body
-@@ -10,3 +10,3 @@
-         ],
--        "you": "127.0.0.1:57079",
-+        "you": "172.19.0.1:59824",
-         "ok": {
+@@ -7,4 +7,12 @@
+       "Document": {
+-        "you": "127.0.0.1:64795",
++        "you": "192.168.65.1:21365",
+         "ok": 1.0,
+       },
 ```

website/versioned_docs/version-v1.18/migration/premigration-testing.md → website/versioned_docs/version-v1.21/migration/premigration-testing.md

@@ -121,66 +121,37 @@ Continuing with the same example above, we can further examine the diff output w
 In the diff output below, however, we have discovered that the query cannot be serviced by our application because the `$first` accumulator operator is not implemented in FerretDB.
 
 ```diff
-2023-08-29T13:25:09.048+0200  WARN  // 127.0.0.1:33522 -> 127.0.0.1:27017  clientconn/conn.go:360 Header diff:
+Header diff:
 --- res header
 +++ proxy header
 @@ -1 +1 @@
--length:   140, id:    2, response_to:  156, opcode: OP_MSG
-+length:   181, id:  360, response_to:  156, opcode: OP_MSG
+-length:   140, id:    8, response_to:   19, opcode: OP_MSG
++length:   306, id:  128, response_to:   19, opcode: OP_MSG
 
 Body diff:
 --- res body
 +++ proxy body
-@@ -7,13 +7,41 @@
-         "$k": [
--          "ok",
--          "errmsg",
--          "code",
--          "codeName"
-+          "cursor",
-+          "ok"
-         ],
+@@ -7,6 +7,25 @@
+       "Document": {
+-        "ok": 0.0,
+-        "errmsg": "$group accumulator \"$first\" is not implemented yet",
+-        "code": 238,
+-        "codeName": "NotImplemented",
 +        "cursor": {
-+          "$k": [
-+            "firstBatch",
-+            "id",
-+            "ns"
-+          ],
 +          "firstBatch": [
 +            {
-+              "$k": [
-+                "_id",
-+                "firstPost"
-+              ],
-+              "_id": "Alice",
-+              "firstPost": {
-+                "$d": 1692527603134
-+              }
++              "_id": "Bob",
++              "firstPost": 2023-08-28T10:33:23.134Z,
 +            },
 +            {
-+              "$k": [
-+                "_id",
-+                "firstPost"
-+              ],
-+              "_id": "Bob",
-+              "firstPost": {
-+                "$d": 1693218803134
-+              }
-+            }
++              "_id": "Alice",
++              "firstPost": 2023-08-20T10:33:23.134Z,
++            },
 +          ],
-+          "id": {
-+            "$l": "0"
-+          },
-+          "ns": "test.posts"
++          "id": int64(0),
++          "ns": "test.posts",
 +        },
-         "ok": {
--          "$f": 0
--        },
--        "errmsg": "$group accumulator \"$first\" is not implemented yet",
--        "code": 238,
--        "codeName": "NotImplemented"
-+          "$f": 1
-+        }
++        "ok": 1.0,
        },
 ```