Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement database migrations for Postgres backend #1878

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

ManiacMaxo
Copy link

Description

Using the current implementation of "migrations" causes a heavy load on the database as it locks the tables during "create if not exists" while starting a worker process. This would be fine if only a single worker is used and there is no concurrent access to the database but in most scenarios you would use a WSGI server like gunicorn which spawns multiple workers which all try to apply migrations at the same time and lock the tables. Gunicorn may also restart a worker if it has not accepted a request in a specified timeout which will resurrect and try to apply migrations, thus locking the tables and all current requests going to other workers. The requested implementation will not lock tables every time and it will only do so if there are unapplied migrations. This also helps with updating the database schema obviously and there would not be need to check if a field exists like:

BEGIN
    ALTER TYPE history ADD ATTRIBUTE "user" text CASCADE;
EXCEPTION
    WHEN duplicate_column THEN RAISE NOTICE 'column "user" already exists in history type.';
END;

Changes
Include a brief summary of changes...

  • added migrations table
  • numbered sql file(s)
  • added logic for applying migrations only if they are not present

Checklist

  • Pull request is limited to a single purpose
  • Code style/formatting is consistent
  • All existing tests are passing
  • Added new tests related to change
  • No unnecessary whitespace changes

@ManiacMaxo
Copy link
Author

Code style seems irrelevant as most current code is not formatted with the same quotes anyways

@satterly satterly added priority: medium A problem affecting a core component for which there is a workaround enhancement New feature or request labels Dec 21, 2023
@satterly satterly force-pushed the master branch 2 times, most recently from 83a1dec to 2d14a76 Compare December 23, 2023 16:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request priority: medium A problem affecting a core component for which there is a workaround
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants