Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make maximum documents size configurable #4233

Closed
Gowee opened this issue Apr 12, 2024 · 5 comments · Fixed by #4294
Closed

Make maximum documents size configurable #4233

Gowee opened this issue Apr 12, 2024 · 5 comments · Fixed by #4294
Assignees
Labels
code/feature Some user-visible feature is not implemented yet
Milestone

Comments

@Gowee
Copy link

Gowee commented Apr 12, 2024

What should be done?

It appears that FerretDB inherit the 16MiB document size limits from MongoDB. It is not pretty reasonable nowadays.

I think we can have an option to tweak the default limit.

FerretDB version

v1.21.0

Backend

PostgreSQL 12.7

Steps to reproduce

mongoimport a document larger than 16MiB to FerretDB. Then mongoimport exits with the error Failed: an inserted document is too large.

@Gowee Gowee added code/feature Some user-visible feature is not implemented yet not ready Issues that are not ready to be worked on; PRs that should skip CI labels Apr 12, 2024
@AlekSi AlekSi added this to the Next milestone Apr 12, 2024
@AlekSi AlekSi changed the title Support documents larger than 16MiB Make maximum documents size configurable Apr 12, 2024
@AlekSi AlekSi removed the not ready Issues that are not ready to be worked on; PRs that should skip CI label May 17, 2024
@AlekSi
Copy link
Member

AlekSi commented May 17, 2024

Let's provide a testing CLI flag for that

@chilagrow chilagrow self-assigned this May 20, 2024
@chilagrow
Copy link
Contributor

The error appears to come from the driver. See https://github.com/mongodb/mongo-go-driver/blob/v1/x/mongo/driver/batches.go#L17. So the request didn't reached FerretDB as the driver returned error before that. Let's see what we can do before we add testing CLI flag.

@Gowee
Copy link
Author

Gowee commented May 20, 2024

So the request didn't reached FerretDB as the driver returned error before that.

It seems that MaxDocumentSize is a configurable value indicated by the server.

https://github.com/mongodb/mongo-go-driver/blob/345ea9574e28732ca4f9d7d3bb9c103c897a65b8/x/mongo/driver/operation.go#L713

@chilagrow
Copy link
Contributor

chilagrow commented May 20, 2024

That MaxDocumentSize is internal representation of server and not configurable for a driver. See information about x package https://github.com/mongodb/mongo-go-driver/tree/345ea9574e28732ca4f9d7d3bb9c103c897a65b8/x

Update:
I don't find implementation within FerretDB restricting maximum document size. And drivers are out of our control unfortunately.

@chilagrow
Copy link
Contributor

In fact, this is indeed configurable 🙈 The value that sets driver was the response from isMaster command's maxBsonObjectSize field.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
code/feature Some user-visible feature is not implemented yet
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

4 participants