Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove stream resources after drop from Postgres manifest #2563

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

FxKu
Copy link
Member

@FxKu FxKu commented Mar 4, 2024

Usually, the Postgres Operator does not drop resources when they are removed from the manifest like databases or users. This is due to safety reasons. However, for the slots defined in the Patroni section we already made an exception because of the danger unused slots in Postgres will have on the cluster health (disk space running full).

Therefore, if somebody wants to stop using streams we could and should also do the same cleanup for the affected database slots. But it gets a bit tricky. So far, it can also be seen as a feature that the Postgres Operator can be used to create stream CRDs so that the user does not have to write the yaml file. After the creation the resource could be copied to a repo and removed from the Postgres manifest. Nothing will happen. But we are learning that this is not the expectation of our users. When they remove the streams section they do it with the intention so stop using event streaming.

The state of the PR atm if only a quick first step of removing the stream CRDs when the stream section is removed. I have extended the unit test to cover this case. But, there are still many ToDos:

  • If the appplicationId is changed a new stream CRD will be created but the old one not deleted (len(slotsToSync) > 0)
  • Only stream resources are removed not database replication slots and publications. We would need another cluster field to remember all stream slots. We cannot use cluster.replicationSlots as they are synced before - see fix sync of stream slots #2194.
  • We cannot remove slots which are still specified under Patroni section.

We are already thinking of redefining the idea of how slots are handeled. It would be easier if their definition is handled with a separate resource, which the Postgres Operator can create and a another operator takes care of the rest.

@FxKu FxKu added this to the 2.0 milestone Mar 4, 2024
@FxKu FxKu modified the milestones: 2.0, 1.12.0 Mar 15, 2024
@FxKu FxKu modified the milestones: 1.12.0, 2.0, 1.13.0 May 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Open Questions
Development

Successfully merging this pull request may close these issues.

None yet

1 participant