Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to import zipped CSV file #1416

Closed
mofojed opened this issue Jul 21, 2023 · 1 comment · Fixed by #1457
Closed

Unable to import zipped CSV file #1416

mofojed opened this issue Jul 21, 2023 · 1 comment · Fixed by #1457
Assignees
@mofojed @mattrunyon
Labels
bug Something isn't working
Milestone
August 2023

Comments

@mofojed
Copy link
Member

mofojed commented Jul 21, 2023

Description

Looks like we broke uploading zipped CSV files at some point.

Steps to reproduce

  1. Drag and drop a zipped CSV file into the console (attached below) and click "Upload"

Expected results

  1. CSV uploads, table opens

Actual results

  1. Fails to upload. No error displayed to user. Error in the console logs is:
[CsvInputBar] Error: nodestream is not supported by this platform
    at e.checkSupport (utils.js:352:15)
    at Kte.toNodejsStream (StreamHelper.js:196:15)
    at LM.nodeStream (zipObject.js:85:58)
    at Kt.parse (CsvTypeParser.js:165:42)
    at oo.parse (CsvParser.js:124:16)
    at lp.handleFile (CsvInputBar.js:184:12)
    at CsvInputBar.js:209:16

Additional details and attachments
table-2023-07-21-110601.zip

Versions

Engine Version: 0.27.0
Web UI Version: 0.44.1
Java Version: 11.0.19
Barrage Version: 0.6.0
@mofojed mofojed added bug Something isn't working triage Issue requires triage and removed triage Issue requires triage labels Jul 21, 2023
@mofojed mofojed self-assigned this Jul 21, 2023
@mofojed mofojed added this to the July 2023 milestone Jul 21, 2023
@mofojed mofojed mentioned this issue Aug 1, 2023
Closed
@mofojed
Copy link
Member Author

mofojed commented Aug 4, 2023

May update JSZip at the same time: #1080

@mattrunyon mattrunyon mentioned this issue Aug 23, 2023
Merged
@mofojed mofojed modified the milestones: July 2023, August 2023 Aug 28, 2023
mofojed pushed a commit that referenced this issue Aug 31, 2023
@mattrunyon
fix: Zip CSV uploads not working (#1457)
08d0296 
Fixes #1080. Fixes #1416 

Updates jszip and uses the internal stream helper instead of nodestream
which we would need a polyfill for. The progress on zip uploads is a bit
odd since we use the unzip progress. It seems papaparse loads more from
the stream while processing/before processing and can finish reading the
zip before it's done processing chunks.

Also, uploading the tables seems to be a blocking operation. Not sure if
that's an easy fix, but after the 50% mark on uploading a large zip,
there are some blocks on the main thread (marching ants freezes as an
indicator).

There is a ~2GB limit for JSZip it seems
Stuk/jszip#777
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mofojed mofojed

@mattrunyon mattrunyon

Labels
bug Something isn't working
Projects
None yet
Milestone
August 2023
Development

Successfully merging a pull request may close this issue.

fix: Zip CSV uploads not working mattrunyon/web-client-ui
2 participants
@mofojed @mattrunyon