Zipped Artifact Downloads corrupt/incomplete

[lhl2617]

Describe the bug

• Zipped Artifacts obtained via the Zipped Artifacts Downloads are corrupt/incomplete
• Related: Size limits? #9 (comment)

Version

• V1
• V2

Environment

• self-hosted
• Linux
• Windows
• Mac

Screenshots
N/A

Run/Repo Url
https://github.com/lhl2617/upload-artifacts-dd/actions/runs/729682844

How to reproduce

• See https://github.com/lhl2617/upload-artifacts-dd/actions/runs/729682844
• Download either of the two 1GB artifacts
  • While internally in the action (up|down)load works (this is a separate gzip mechanism I suppose), the file downloaded in the UI, which is dynamically created, only downloads 1MB out of 1.1GB.
  • I thought this was due to block size when I created the file using dd, but using a different block size yields identical results--is this due to chunking?

Additional context

• w.r.t. Size limits? #9, it would be better to have clearer documentation w.r.t. max file size and other limitations. AFAIK artifacts do not have a documented limit, but anecdotally I've seen uploads of >10GB work.

[EthanSK]

Any info on this? It's quite important

[aminya]

I have the same issue. After debugging for so many days, I think the issue with my artifacts is GitHub Actions.
The artifacts uploaded here are faulty:
https://github.com/zeromq/zeromq.js/actions/runs/754240241

I can confirm that the issue is related to the size of the artifact. In my original build, artifact size was ~100MB uncompressed (35MB zipped), and the downloaded file was faulty. But when I reduced the number of artifacts, the downloaded files were normal.

[aminya]

The code that @actions/upload-artifact uses for artifact uploading is here if anyone wants to take a look:
https://github.com/actions/toolkit/blob/4bf916289e5e32bb7d1bd7f21842c3afeab3b25a/packages/artifact/src/internal/upload-http-client.ts#L196

This description of this line of code is interesting to me:
https://github.com/actions/toolkit/blob/4bf916289e5e32bb7d1bd7f21842c3afeab3b25a/packages/artifact/src/internal/upload-http-client.ts#L319

[lhl2617]

The code that @actions/upload-artifact uses for artifact uploading is here if anyone wants to take a look:
https://github.com/actions/toolkit/blob/4bf916289e5e32bb7d1bd7f21842c3afeab3b25a/packages/artifact/src/internal/upload-http-client.ts#L196

This description of this line of code is interesting to me:
https://github.com/actions/toolkit/blob/4bf916289e5e32bb7d1bd7f21842c3afeab3b25a/packages/artifact/src/internal/upload-http-client.ts#L319

@aminya I think these functions are for internally uploading the artifacts for use by later jobs in the action (this uses gzip for large files); in my experience downloading in a later job is fine. The download UI and the octokit API for retrieving a download link uses another API (using zip). I think this is not open source but rather on Github's backend?

Would be nice if we can at least have a reply on this issue as it is quite severe!

[aminya]

As far as I can see everything is open source. This library uses create and then uploadArtifact. uploadArtifact function uses uploadArtifactToFileContainer which finally calls uploadFileAsync that I linked in the above comment.

[lhl2617]

@aminya I'm not entirely sure about that but see: #107 (comment)

Notably https://github.com/actions/toolkit/blob/master/packages/artifact/docs/additional-information.md#is-my-artifact-compressed

[aminya]

Oh, yeah. Definitely, GitHub's UI and the code related to the ZIP creation is not open source. What I meant was what this action does here. Maybe we should open a GitHub support ticket instead of an issue in this repository that apparently uploads the files correctly.

[lhl2617]

I've opened one :)

[lhl2617]

Apologies w.r.t. original issue: the downloaded zip files are in fact not corrupt but just compressed (uncompressing them yields the original expected file).

I'm still seeing spotty behavior and files getting dropped, so I've referred to
#9 (comment)
and
#190 (comment)
in the follow-up in the ticket.

[lhl2617]

Hi lhl2617,

No worries! I've followed up with the team and they did mention they are aware of these reports regarding artifacts being incomplete when downloaded.

While it has been added to their product backlog as an issue to be investigated, we don't have an ETA on when a fix will be shipped.

Our roadmap is now publicly visible, so we recommend that you keep an eye on the GitHub Blog and roadmap for the latest announcements about new features.

Cheers,
GitHub Support

🌟Find answers to common questions and learn with other GitHub users in the GitHub Community Forum🌟

Response from Github support

[immanishbainsla]

I am facing this issue : artifact file original size is 245MB but when downloaded it's size is just 22b.
also file name is "zip" without any extension.
just curious whether it's the compression that reduced 245MB to 22 bits?
ref: https://github.com/immanishbainsla/fabric8-analytics-vscode-extension/actions/runs/863711441

[konradpabjan]

Closing this as a duplicate of #156 (will be tracked here)

Some information on what is going on here: #74 (comment)

We've been making gradual updates to the backend so that this happens less frequently. Still no immediate plans for options via the UI or new APIs to download artifacts without zipping given our current priorities and roadmap