Unable to download artifacts larger than 2^32-1 bytes

[gbhand]

Really appreciate the work you've done here!

I've recently been trying to download a 3GB artifact and keep running into this error:

I'm fairly certain this is a file size issue as uploading a file exactly 2147483646 bytes in size works fine:

I believe this is the line that throws the error

action-download-artifact/main.js

Line 138 in af92a84

const adm = new AdmZip(Buffer.from(zip.data))

It seems like the Buffer objects can only hold 2^32-1 bytes.

I'd make a fork myself (maybe reading the zip's data as a stream?) but I'm pretty novice with NodeJS. Handling larger files could be a useful option but I understand the project may have other plans.

Thanks!

[dawidd6]

I had trouble here in the past making it work in JavaScript. Would appreciate a PR making things better in terms of handling bigger files.

[radimkohout]

In my head, there is one possible solution, which is to download artifact to file, unZIP it, end then delete the ZIP file. The downside is that you need to have double the space. If this is an acceptable solution, I can make it and PR it.

[dawidd6]

Would be great if you or someone else could first confirm this workaround really fixes the issue. If it does, then we can introduce a new switch.

[radimkohout]

Ok, give me some time.

[radimkohout]

I can't find any proper way, how to do it in node, but it can be done by command-line tools (wget on Linux, Invoke-WebRequest/ something else in Windows PowerShell).

[daleyjem]

Can't you just add an action input to not unpack, and let the consumer do that themselves if they want to (using tar or something)?

[radimkohout]

@daleyjem I've done something like that there: https://github.com/GarnetOS/action-download-artifact
But note, that it's Linux dependent, and needs wget and unzip installed.

[daleyjem]

Thanks @radimkohout. That's not quite what I meant though. I just want to be able to download an unpacked/unzipped artifact. Once I get it, I can just add my own extra step to unpack it, or do whatever I want with it. For me, the real "gold" with this action is being able to download from another run

I submitted a pull request with what I'm talking about: #150

[radimkohout]

@daleyjem Yes, I know, but it is my solution for my usage, where I need large artifacts. So I know, it is not ideal, but it is usable currently.

[dawidd6]

Switch for skipping the unpacking process has been added.