Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replacing archiver package with direct use of zip-stream for fixing macOS upload issues #1690

Open
wants to merge 75 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
1f22f9f
adding debug for mac upload testing
vmjoseph Mar 15, 2024
4617da2
audit fix
vmjoseph Mar 15, 2024
4090623
update debug statement
vmjoseph Mar 28, 2024
13d981a
Adding buffer and zlib options
vmjoseph Mar 28, 2024
e71ddb9
updating append and directory zips
vmjoseph Mar 28, 2024
14eee6f
updating event handlers
vmjoseph Mar 28, 2024
6e4aebb
cleaning up unused imports
vmjoseph Mar 28, 2024
6defe19
replacing constructor for zipstream
vmjoseph Mar 28, 2024
fda9d58
replacing constructor for zipstream with default
vmjoseph Mar 28, 2024
560ae4d
updating archive import
vmjoseph Mar 28, 2024
8e29fb2
adding default back to zipstream call
vmjoseph Mar 28, 2024
97629b8
adding archiver back for download
vmjoseph Mar 28, 2024
ca87c5e
testing debug statements
vmjoseph Mar 28, 2024
7441cc7
Removing pipe
vmjoseph Mar 28, 2024
06482c6
adding promise to upload
vmjoseph Mar 28, 2024
d97edf7
cleaning up warning callack
vmjoseph Mar 28, 2024
088b976
cleaning up callback methods
vmjoseph Mar 28, 2024
9299663
adding promise all
vmjoseph Mar 28, 2024
b1f55c6
adding finalize
vmjoseph Mar 28, 2024
1875173
update loop for upload
vmjoseph Mar 28, 2024
bef1fc5
adding check for running
vmjoseph Mar 28, 2024
fd88cbe
adding check for running
vmjoseph Mar 28, 2024
66343fa
add resolve all promises again
vmjoseph Mar 28, 2024
c7de68f
updating resolves
vmjoseph Mar 28, 2024
fe0c0de
adding lock
vmjoseph Mar 28, 2024
97e4fcf
wait on write stream
vmjoseph Mar 28, 2024
f33a3f4
Remove finalize
vmjoseph Mar 28, 2024
180b75b
wrap promises in chain
vmjoseph Mar 28, 2024
077846e
adding readable check
vmjoseph Mar 28, 2024
73f526b
adding closed check
vmjoseph Mar 28, 2024
26b62e0
adding more logging
vmjoseph Mar 28, 2024
e4c0440
adding more logging
vmjoseph Mar 28, 2024
a768aa3
adding more logging
vmjoseph Mar 28, 2024
d597cf2
adding upload stream check
vmjoseph Mar 28, 2024
6ac9cbf
check if upload and zipload streams are writable
vmjoseph Mar 28, 2024
d2d6999
adding write check
vmjoseph Mar 28, 2024
31a6086
Adding concurrency and upload response logs
vmjoseph Mar 28, 2024
c9e825e
add blob client property check
vmjoseph Mar 28, 2024
0a0e70d
Add another catch to see if upload stream is failing
vmjoseph Mar 28, 2024
5959333
defaulting compression level
vmjoseph Mar 28, 2024
17c4cab
adding constant zlib compression level
vmjoseph Mar 28, 2024
b6c87ce
Adding more error handlrs
vmjoseph Mar 28, 2024
9322468
Adding pipe back
vmjoseph Mar 28, 2024
bc893bf
adding compression level back
vmjoseph Mar 28, 2024
5288db3
removing debugging statements
vmjoseph Mar 28, 2024
0f23ae1
re-adding old async/await pattern
vmjoseph Mar 28, 2024
d02a834
Remove other debug statements
vmjoseph Mar 28, 2024
6d94ad1
Removing uneeded libs
vmjoseph Mar 28, 2024
a2852ce
adding catch to upload zip
vmjoseph Mar 28, 2024
c14e304
adding more debugging statements
vmjoseph Mar 28, 2024
90ee020
cleaning up debug statements
vmjoseph Apr 1, 2024
2da5288
cleanup
vmjoseph Apr 1, 2024
2d06555
Merge branch 'main' into vmjoseph/node-js-monitor
vmjoseph Apr 1, 2024
2f2738e
removing old packages
vmjoseph Apr 1, 2024
4c878a6
adding back archiver
vmjoseph Apr 1, 2024
dea3595
re-adding archiver for download
vmjoseph Apr 1, 2024
2c4f0f5
adding error checks
vmjoseph Apr 1, 2024
6eff4e9
adding more debug statements
vmjoseph Apr 1, 2024
4778aeb
adding asnyc handler back
vmjoseph Apr 1, 2024
23039a4
adding asnyc handler back
vmjoseph Apr 1, 2024
a8fa53b
test async eachof
vmjoseph Apr 1, 2024
a2a8a72
test queue
vmjoseph Apr 1, 2024
a926fff
adding queue back
vmjoseph Apr 1, 2024
84d3cef
Removing extra logs
vmjoseph Apr 1, 2024
96ef8d5
adding more error handling
vmjoseph Apr 1, 2024
c5f3463
adding more descriptive error
vmjoseph Apr 2, 2024
38f54e9
Adding debug statement for upload
vmjoseph Apr 2, 2024
ce60352
adding asysnc zip entry logic handling
vmjoseph Apr 8, 2024
e7f4586
Merge branch 'main' into vmjoseph/node-js-monitor
vmjoseph Apr 8, 2024
4f54b86
updating errors
vmjoseph Apr 8, 2024
67c3837
updating errors
vmjoseph Apr 8, 2024
6e1d754
updating tests
vmjoseph Apr 8, 2024
da5e926
removing promise wrap
vmjoseph Apr 8, 2024
bcaba45
reverting http update
vmjoseph Apr 8, 2024
42b0077
resolve merge conflicts
jtamsut Apr 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 3 additions & 3 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

28 changes: 27 additions & 1 deletion packages/artifact/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion packages/artifact/package.json
Expand Up @@ -53,7 +53,8 @@
"crypto": "^1.0.1",
"jwt-decode": "^3.1.2",
"twirp-ts": "^2.5.0",
"unzip-stream": "^0.3.1"
"unzip-stream": "^0.3.1",
"why-is-node-running": "^2.2.2"
},
"devDependencies": {
"@types/archiver": "^5.3.2",
Expand Down
8 changes: 8 additions & 0 deletions packages/artifact/src/internal/upload/upload-artifact.ts
@@ -1,3 +1,4 @@
import whyIsNodeRunning from 'why-is-node-running'
import * as core from '@actions/core'
import {
UploadArtifactOptions,
Expand Down Expand Up @@ -107,6 +108,13 @@ export async function uploadArtifact(
core.info(
`Artifact ${name}.zip successfully finalized. Artifact ID ${artifactId}`
)
if (core.isDebug()) {
setTimeout(function () {
core.debug('Processes keeping upload stream running:')
whyIsNodeRunning()
}, 500)
}
//

return {
size: uploadResult.uploadSize,
Expand Down
1 change: 0 additions & 1 deletion packages/artifact/src/internal/upload/zip.ts
Expand Up @@ -29,7 +29,6 @@ export async function createZipUploadStream(
core.debug(
`Creating Artifact archive with compressionLevel: ${compressionLevel}`
)

const zip = archiver.create('zip', {
highWaterMark: getUploadChunkSize(),
zlib: {level: compressionLevel}
Expand Down