oss artifact: get 400 error from OSS server when OSSLifecycleRule is set

[AlbeeSo]

Pre-requisites

• I have double-checked my configuration
• I can confirm the issue exists when I tested with :latest
• I have searched existing issues and could not find a match for this bug
• I'd like to contribute the fix myself (see contributing guide)

What happened/what did you expect to happen?

OSS does not support to set different rules at a time for a same prefix and that's why I received a 400 error.

And Lifecycle rules of OSS is a little bit complicated which always also need to consider if the bucket is Versioning enabled. (refers to https://www.alibabacloud.com/help/en/oss/developer-reference/manage-versioning-3?spm=a2c63.p38356.0.0.e8367781VCXnwa)
A rule can contains 5 different configurations:
(Refers to https://github.com/aliyun/aliyun-oss-go-sdk/blob/3804c1371ab84adf9bacf7d940281b3fb39deb75/oss/type.go#L70C1-L85C2)

• Expiration: used to configure when to delete the objects (or turn to non-current version if Versioning enabled).
• Transitions: used to configure when to convert the storage type of current version objects.(like Infrequent Access)
• NonVersionExpiration: similar to Expiration but for non-current version objects.
• NonVersionTransitions: similar to Transitions but for non-current version objects.
• AbortMultipartUpload: used to configure when to abort (or delete) the uploaded parts created by a interrupted multipart upload due to network or something. (Parts will reuse if retry to upload again, but will be storaged in OSS if the upload task is given up.)

So, I suggest to modified the rule and only keep the Expiration, AbortMultipartUpload, and Transitions configurations as IMO, if users enable BucketVersioning, deleting the non-version objects by markDeletionAfterDays is unexpected.

Version

latest

Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: set-lifecycle-
spec:
  entrypoint: main
  templates:
    - name: main
      metadata:
        annotations:
          k8s.aliyun.com/eci-extra-ephemeral-storage: "5Gi"
          k8s.aliyun.com/eci-use-specs : "ecs.g7.xlarge"
      container:
        image: alpine:latest
        command:
          - sh
          - -c
        args:
          - |
            mkdir -p /out
            dd if=/dev/random of=/out/lifecycle.txt bs=2M count=1024
            echo "created files!"
      outputs:
        artifacts:
          - name: out
            path: /out/lifecycle.txt
            oss:
              endpoint: http://oss-cn-zhangjiakou-internal.aliyuncs.com
              bucket: my-argo-workflow
              key: argo-workflows/test/lifecycle
              accessKeySecret:
                name: my-argo-workflow-credentials
                key: accessKey
              secretKeySecret:
                name: my-argo-workflow-credentials
                key: secretKey
              lifecycleRule:
                markInfrequentAccessAfterDays: 1
                markDeletionAfterDays: 3

Logs from the workflow controller

kubectl logs -n argo deploy/workflow-controller | grep ${workflow}

Logs from in your workflow's wait container

time="2024-04-09T16:04:23.147Z" level=info msg="Save artifact" artifactName=out duration=1.270302632s error="oss: service returned error: StatusCode=400, ErrorCode=InvalidRequest, ErrorMessage=\"Found same prefix in different rules\", RequestId=6615670711BF5732361C11E2, Ec=0014-00000085" key=argo-workflows/test/lifecycletime="2024-04-09T16:04:23.147Z" level=error msg="executor error: oss: service returned error: StatusCode=400, ErrorCode=InvalidRequest, ErrorMessage=\"Found same prefix in different rules\", RequestId=6615670711BF5732361C11E2, Ec=0014-00000085"

[terrytangyuan]

Sounds good! Great to know that you are using OSSLifecycleRule that I added a long time ago.