Support the ability to add changes to an on-going migration

[trobertson-cn]

We have a case where a typical development/release cycle will have dozens of commits by multiple developers over the course of days/weeks/months. While it would be preferable to clearly identify all schema changes up front during a cycle, it isn't really practical due to a number of factors. This means that over the course of a cycle, there will be multiple commits with changes to the same schema.
Using pgroll, we could treat each commit as its own migration, but we would prefer to treat all of the changes to a schema in a release as a single migration so that we have a clear A/B differentiation between those releases. For our test and local development systems, this poses a problem because the "continuous-delivery" nature of these systems means that per-commit schema changes have to be applied as the code is pulled/deployed. It would be great if pgroll would allow a migration to "start" and have then have additional schema changes "appended" before eventually being completed at the end of the development cycle for a given release.

[exekias]

Thanks for opening this issue!

This is a very interesting use case, my understanding is that we want several devs / teams to have in-flight migrations in parallel. The obvious challenges are:

• These migrations could affect each other: ie I rename a column that was deleted by another migration (this is partially related to Support multiple operations acting on the same object in one migration #239)
• Contracting a previously expanded schema becomes challenging as we need to know if it is safe to do it now, or some of the clients still depends on it.

I believe this should be possible to solve, we would need to keep track of all schema expansions and ensure the contract phase happens only after all references to the changed columns/tables are gone. To achieve this, every single migration would need to provide a pointer to what their base version is, so we know any old version without references can be contracted.

[trobertson-cn]

Thanks. I agree that #239 will quickly become necessary as it is quite possible the same "object" will be manipulated multiple times with our model.

I had envisioned this working like an open ledger of changes that could be appended to during the expansion phase. The thought being that there would be a single "ordered" list of changes without the notion of branching. Eventually we would "close" the expansion phase but I don't know if this would be strictly necessary from a usage perspective prior to the contraction, since the contraction would implicitly close the expansion phase.
To keep this manageable, I would expect that any change that was added to the ledger for a given expansion would be immutable and then pgroll would need some way to enforce that as well as identify and sequence the new changes being appended. Clients using the expanded schema would be responsible for adopting the changes as they're added to the ledger and rolled out.