feat: restore + anonymise

[revolunet]

restore and anonymify from prod to some development DB

• Prod job : pop a CNPG-tmp cluster from Prod WALs
• Prod job : run anonymize the new PG instance
• Prod job : dump to s3-tmp
• Prod job : destroy CNPG-tmp
• Trigger Preprod job: download and restore to Preprod CNPG from s3-tmp

Notes :

• the prod jobs run in a dedicated domifa-prod-dump namespace to prevent interference with prod
• il y a probablement un cleanup de l'image docker/s3-client pour permettre aussi le download+restore ?
• a priori il faudrait utiliser cette approche pour trigger les jobs

graph TD

GH[workflow dispatch update preprod data]-->DeployProd[kontinuous deploy prod job]
DeployProd-->CNPG1[Create tmp CNPG cluster from prod WALs]-->AnonymiseJob[anonymise]-->DumpJob[Dump]-->Store-->S3{S3}
GH2[Trigger Gh workflow preprod update]-->DeployPreProd[kontinuous deploy preprod job]
DeployPreProd-->DownloadJob-->RestoreJob
Store-->Destroy
Store-->GH2
DownloadJob-->S3[S3-tmp]
subgraph Prod
CNPG1
AnonymiseJob
DumpJob
Store
Destroy
end
subgraph PreProd
CNPG1
DownloadJob
RestoreJob
end
subgraph GitHub
GH
GH2
DeployProd
DeployPreProd
end

[codecov-commenter]

Codecov Report

Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (16a5c7f) 67.76% compared to head (5938998) 67.86%.

❗ Current head 5938998 differs from pull request most recent head 05e2da7. Consider uploading reports for the commit 05e2da7 to get more accurate results

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3055      +/-   ##
==========================================
+ Coverage   67.76%   67.86%   +0.09%     
==========================================
  Files         911      919       +8     
  Lines       14660    14686      +26     
  Branches     2112     2116       +4     
==========================================
+ Hits         9934     9966      +32     
+ Misses       4451     4445       -6     
  Partials      275      275              

Files	Coverage Δ
...ackages/backend/src/config/domifaConfig.service.ts	89.28% <ø> (ø)
...ase/services/_postgres/PG_CONNECT_OPTIONS.const.ts	52.17% <33.33%> (ø)

... and 124 files with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 16a5c7f...05e2da7. Read the comment docs.

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

[sonarcloud]

Quality Gate passed

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

[github-actions]

🎉 Deployment for commit 05e2da7 :

Ingresses

• 🚀 https://admin-domifa-restore.dev.fabrique.social.gouv.fr/
• 🚀 https://api-domifa-restore.dev.fabrique.social.gouv.fr/
• 🚀 https://domifa-restore.dev.fabrique.social.gouv.fr/
• 🚀 https://mon-domifa-restore.dev.fabrique.social.gouv.fr/

Docker images

• 📦 docker pull harbor.fabrique.social.gouv.fr/domifa/domifa/backend:sha-05e2da7414260909e0a69adada0c684e74247c61
• 📦 docker pull harbor.fabrique.social.gouv.fr/domifa/domifa/frontend:sha-05e2da7414260909e0a69adada0c684e74247c61
• 📦 docker pull harbor.fabrique.social.gouv.fr/domifa/domifa/portail-admins:sha-05e2da7414260909e0a69adada0c684e74247c61
• 📦 docker pull harbor.fabrique.social.gouv.fr/domifa/domifa/portail-usagers:sha-05e2da7414260909e0a69adada0c684e74247c61

Debug

• 📕 Loki logs for namespace domifa-restore
• 📈 Pods monitoring for namespace domifa-restore
• 📈 Workloads monitoring for namespace domifa-restore
• 🐘 CNPG pg

[github-advanced-security]

This pull request sets up GitHub code scanning for this repository. Once the scans have completed and the checks have passed, the analysis results for this pull request branch will appear on this overview. Once you merge this pull request, the 'Security' tab will show more code scanning analysis results (for example, for the default branch). Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results. For more information about GitHub code scanning, check out the documentation.