[feature] delete MinIO artifacts on pipeline run deletion

[tmvfb]

Feature Area

/area frontend
/area backend
/area components

What feature would you like to see?

When pipeline run gets deleted, remove associated artifacts from MinIO storage and from artifacts list/DB.

What is the use case or pain point?

Consistently running kubeflow pipelines on a production cluster produces a lot of artifacts that use up a lot of storage resources. That leads to a necessity of periodical storage clean up. This process could be made much easier if the artifacts got automatically deleted on run deletion.

I found some issues that seem to address either this or similar matter, namely #8104, #5509, #5667, #5783 (related to DB entries), #7939 (might be related).

Comment #8104 (comment) describes exactly the behaviour I would expect from kubeflow pipelines as an unexperienced user.

Is there a workaround currently?

Our solution is manual cleanup which is surely not effective. If there is currently a way to setup an automated cleanup, we'd be happy to know!

---

Love this idea? Give it a 👍.

[HumairAK]

So artifacts aren't necessarily tied to a specific pipeline run, for instance:

Pipeline Run can produce Artifact X

But Artifact X can go on to later be used by a different pipeline run (cached).

I'm not sure the details of imported artifacts, but I think you could also technically import Artifact X in a completely separate pipeline run.

So we would need to consider how to handle these cases.