intial schema migrations setup for sqlalchemy models

[auvipy]

using alembic

[codecov]

Codecov Report

Patch and project coverage have no change.

Comparison is base (65ac2ac) 87.45% compared to head (3760587) 87.45%.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #8532   +/-   ##
=======================================
  Coverage   87.45%   87.45%           
=======================================
  Files         148      148           
  Lines       18499    18499           
  Branches     3158     3158           
=======================================
  Hits        16179    16179           
  Misses       2032     2032           
  Partials      288      288           

Flag	Coverage Δ
unittests	87.42% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Nusnus]

👑

[auvipy]

I need to get my hands dirty with this for a few more days to make it fully ready.

[ThiefMaster]

Is this really something that should be on the celery side?

• If I do not use alembic and just use create_all() to create my database tables, then I just need to make my SA aware of the models (ie by calling a function from celery that creates them using my application's declarative base)
• If I use alembic, I'm already maintaining my own revisions, so I would do the same as above, and use alembic to autogenerate the revision to create the celery-related models (if that's even needed: It's only needed when adding celery at a later stage, or when using alembic to create the initial database instead of letting SA's create_all() do this)

[auvipy]

I will share more context here.

[auvipy]

Is this really something that should be on the celery side?

* If I do not use alembic and just use `create_all()` to create my database tables, then I just need to make my SA aware of the models (ie by calling a function from celery that creates them using my application's declarative base)

* If I use alembic, I'm already maintaining my own revisions, so I would do the same as above, and use alembic to autogenerate the revision to create the celery-related models (if that's even needed: It's only needed when adding celery at a later stage, or when using alembic to create the initial database instead of letting SA's `create_all()` do this)

we got a regression report and had to revert #8475 (comment) so thought that adding migration is the safer bet. what do you think?

[ThiefMaster]

I think DB changes should be clearly documented in the changelog. But executing them extremely dependent on the environment.

Note: This is assuming that you do not maintain your completely separate sqlalchemy environment. If you do, then migrations in celery make sense (because it's completely independent of the application's database usage). But if you defer this to the application's sqlalchemy environment, then IMHO what I mentioned in my previous comment applies.

[auvipy]

is using multi db schema migrations a feasible option?