feat: Allow specifying multiple migration sources in CLI #3177
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR extends the CLI to allow specifying multiple migration sources:
It will take the migrations within all specified folders, and consider each of them sorted by version in ascending order. This should be fully backwards compatible.
This change would allow separating migrations that set up table structure from those that create test data. This does not have an issue associated to it, please let me know if I should open one first.
Motivation (rather long)
Use case
When setting up a web project for local development, I like to include some test fixtures in the database, so that I can see how everything looks when populated with actual data. I create that test data from database dumps, so they are simple SQL files, just like migrations. So far, I solved this problem by having two separate directories in my project:
This way, I can run
sqlx migrate run
andsqlx migrate run --source fixtures
locally to create both the tables and the test data within them. For the production database, I only runsqlx migrate run
, so that the test data isn't included there. This works well enough in most cases.The problem
A problem arises when I need to create a migration for a newer version that changes an existing table. The migration itself should already make sure existing data will conform to the new data model. However, with the above setup, all fixture data is created after the last table migration, so they will have to comply to the new data model from the start.
Alternative solutions
To work around this issue, I can choose to either:
/migrations
folder (so that they are performed in order of their timestamps), then runsqlx migrate run
, and then move them back so I don't accidentally do the same in the production databaseBoth of these are undesirable. The former requires manual work every time the database needs to be initialized (or some sort of external tool that synchronizes a single folder with the contents of both other folders). The latter can take quite long depending on the amount of test data, and also requires manual work to undo in case of a
sqlx migrate revert
.Allowing to specify multiple sources with the CLI would solve the problem elegantly. I hope you'll consider including it in
sqlx
!