Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(vitest-pool-workers): ignore sqlite shm and wal files (#5629) #5667

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

hansottowirtz
Copy link

See https://github.com/cloudflare/workerd/blob/d3b5b16588cc33c930584064995c4a3635d07c28/src/workerd/server/workerd.capnp#L613

Not sure if tests are needed, and how they should be set up, because I do not know how to set up sqlite to generate .sqlite-shm files

What this PR solves / how to test

Fixes #5629

Author has addressed the following

@hansottowirtz hansottowirtz requested a review from a team as a code owner April 19, 2024 18:17
Copy link

changeset-bot bot commented Apr 19, 2024

⚠️ No Changeset found

Latest commit: 297d3b4

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@mrbbot
Copy link
Contributor

mrbbot commented Apr 21, 2024

Hey! 👋

Thanks for this PR! I'm not on the team anymore, but I'm not sure about this fix. The reason we fail when we detect non .sqlite files is that these may contain data that we otherwise wouldn't be backing up. The .sqlite-wal file contains SQLite's write-ahead log which may contain things that haven't been written to the main .sqlite file. After each test, we abort all Durable Objects (including internal ones for things like KV and R2 storage), which should "checkpoint" all write-ahead logs into their corresponding .sqlite files and remove them. This error indicates that something was written between test execution and during the process of backing up the database. We fail here because this could result in the incorrect state being restored later on.

Ideally, we wouldn't need to worry about manipulating .sqlite files on disk, and could instead use SQLite SAVEPOINTs to implement the isolated storage stack with nested transactions (#5395). I'd encourage someone on the team to investigate this, as I think it would also significantly improve test performance.

@hansottowirtz
Copy link
Author

@mrbbot Thanks a lot for your reply, would ignoring only the .sqlite-shm files be okay?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

🐛 BUG: Durable Object tests won't run, .sqlite error
2 participants