fix: use multipart upload method to put files larger than 5Gi to OSS. Fixes #12877

[AlbeeSo]

Fixes #12877

Motivation

argo-workflows/workflow/artifacts/oss/oss.go

Line 342 in 0b625a6

return bucket.PutObjectFromFile(objectName, path)

OSS SDK client offers several functions help upload local files to server. And PutObjectFromFile uses Simple Upload which usually causes (or always from my experience) InvalidArgument or EntityTooLarge error when uploading files larger than 5Gi.
More info plz see: https://www.alibabacloud.com/help/en/oss/you-cannot-upload-large-objects-by-using-simple-upload?spm=a3c0i.23458820.2359477120.55.7ef56e9boHbYeC
Multipart Upload is the recommended way.

Modifications

When trying to save files in OSS by Save Func of ArtifactDriver Interface:

• Still use Simple Upload method if file to saved is not larger than 5Gi.
• Or else split it into size of capacity / 5Gi +1 parts and upload by Multipart Upload method.

Verification

Plz test with a simple workflow like

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: artifact-
spec:
  entrypoint: main
  templates:
    - name: main
      container:
        image: alpine:latest
        command:
          - sh
          - -c
        args:
          - |
            mkdir -p /out
            dd if=/dev/random of=/out/testfile.txt bs=20M count=1024
            echo "created files!"
      outputs:
        artifacts:
          - name: out
            path: /out/testfile.txt
            oss:
              endpoint: http://oss-cn-zhangjiakou-internal.aliyuncs.com
              bucket: bucket
              key: argo-workflows/test/bigfile
              accessKeySecret:
                name: my-argo-workflow-credentials
                key: accessKey
              secretKeySecret:
                name: my-argo-workflow-credentials
                key: secretKey

[shuangkun]

Some comments here。

[AlbeeSo]

Some comments here。

Done, PTAL. :)

[terrytangyuan]

Thanks!

[agilgur5]

I know this is mostly taken from the Alibaba docs, but I wonder if the chunking / UploadPart section could be parallelized?

If so, could potentially contribute that back upstream if Alibaba's docs are open source (idk?)

[agilgur5]

Backported cleanly into release-3.5 as 1c1f433