Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrectly compressed output artifact on Windows #10562

Closed
2 of 3 tasks
kalb1 opened this issue Feb 21, 2023 · 9 comments
Closed
2 of 3 tasks

Incorrectly compressed output artifact on Windows #10562

kalb1 opened this issue Feb 21, 2023 · 9 comments
Labels
area/artifacts S3/GCP/OSS/Git/HDFS etc area/windows Windows Container support P3 Low priority type/bug

Comments

@kalb1
Copy link

kalb1 commented Feb 21, 2023

Pre-requisites

  • I have double-checked my configuration
  • I can confirm the issues exists when I tested with :latest
  • I'd like to contribute the fix myself (see contributing guide)

What happened/what you expected to happen?

Output archive artifact compress folder incorrectly in case of windows image. Instead of keeping the folder structure, it puts all files in the root folder and appends the folder names to the file names, e.g structure:
argo_test/
|----- test.txt

In the compressed artifact, there will be the empty argo_test folder and the file called argo_test\test.txt
The problem does not persist on Linux pods.

I assume this should be a 7zip misconfiguration.

Version

v3.4.4

Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: artifact-passing-
spec:
  entrypoint: artifact-example
  templates:
  - name: artifact-example
    steps:
    - - name: hello-win
        template: hello-win

  - name: hello-win
    nodeSelector:
          kubernetes.io/os: windows
    container:
      image: mcr.microsoft.com/windows/nanoserver:1809
      command: ["cmd", "/c"]
      args: ["mkdir argo-test && echo Welcome from windows container! > argo-test/test.txt"]
    outputs:
      artifacts:
      - name: hello-win
        path: /argo-test
        archive:
          tar:
            compressionLevel: 0

Logs from the workflow controller

-

Logs from in your workflow's wait container

-
@kalb1 kalb1 added the type/bug label Feb 21, 2023
@kalb1 kalb1 changed the title Incorrectly compressed archived output artifact on Windows Incorrectly compressed output artifact on Windows Feb 22, 2023
@juliev0 juliev0 added the P3 Low priority label Mar 9, 2023
@stale
Copy link

stale bot commented Mar 25, 2023

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If this is a mentoring request, please provide an update here. Thank you for your contributions.

@stale stale bot added the problem/stale This has not had a response in some time label Mar 25, 2023
@weretheone
Copy link

Unfortunately this issue is still affecting our environment. We was able to overcome the situation with disabling compression and handling the artifacts differently, but it would be nice to see this issue resolved some way.

@stale stale bot removed the problem/stale This has not had a response in some time label Mar 27, 2023
@PeterKoegel
Copy link
Contributor

This issue seems to be caused by the way how tar archives are created in util/archive/archive.go tarDir()

There the nameInArchive for each file is determined which ends up in the tar file as a file header. This works fine on Linux but on Windows a path with backslashes instead of forward slashes is written to the tar file. When such a tar file the is unpacked on Linux the result is as described above (the directory structure is lost and the complete path with backslashes is ending up in the filename).

@Transmitt0r
Copy link
Contributor

So we should always use forward slashes in the header? Since that seems to work on both platforms.

@kalb1
Copy link
Author

kalb1 commented May 17, 2023

@Transmitt0r I tried with forward slashes as you can see in the example, but unfortunately it did not work fine for me.

@PeterKoegel
Copy link
Contributor

@Transmitt0r Yes, changing the code to always use forward slashes in the header worked for me. After that the .tgz output artifacts produced by a Windows step could be used as input artifacts for Windows and Linux steps.

When opening the created .tgz files with something like 7-zip they also had the right directory structure only when they had forward slashes as directory seperators.

@kalb1
Copy link
Author

kalb1 commented May 17, 2023

@PeterKoegel Which version did you use for checking this?

@PeterKoegel
Copy link
Contributor

@kalb I will create a pull request today with the changes I did to get it work. I had to change a line of code that creates the tar files and build my own argoexec container from that.

@kalb1
Copy link
Author

kalb1 commented May 17, 2023

Awesome, ty @PeterKoegel

terrytangyuan pushed a commit that referenced this issue May 25, 2023
…Fixes #10562 (#11097)

Signed-off-by: Peter Kögel <peter.koegel@vector.com>
JPZ13 pushed a commit to pipekit/argo-workflows that referenced this issue Jul 4, 2023
dpadhiar pushed a commit to dpadhiar/argo-workflows that referenced this issue May 9, 2024
…Fixes argoproj#10562 (argoproj#11097)

Signed-off-by: Peter Kögel <peter.koegel@vector.com>
Signed-off-by: Dillen Padhiar <dillen_padhiar@intuit.com>
@agilgur5 agilgur5 added area/artifacts S3/GCP/OSS/Git/HDFS etc area/windows Windows Container support labels Jun 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/artifacts S3/GCP/OSS/Git/HDFS etc area/windows Windows Container support P3 Low priority type/bug
Projects
None yet
Development

No branches or pull requests

6 participants