Skip to content
This repository has been archived by the owner on Jan 18, 2024. It is now read-only.

[expo-cli] Fix parallel uploads #2736

Merged
merged 2 commits into from Oct 13, 2020
Merged

Conversation

LinusU
Copy link
Contributor

@LinusU LinusU commented Sep 30, 2020

This fixes the issue I commented about here: https://github.com/expo/expo-cli/pull/2504/files#r497315193

[...] with this change, only one single process can upload one build artefact at a time, since otherwise the file will be deleted from under that process. E.g. trying to run expo upload:ios and expo upload:android in parallel will have one of them fail with either:

Error setting value '/tmp/expo-submission-service/foo-xxxxxx-signed.aab' for option 'aab'
Could not find aab file at path '/tmp/expo-submission-service/foo-xxxxxx-signed.aab'

or:

Failed to upload the standalone app to the app store.
Could not find file at path '/tmp/expo-submission-service/foo-xxxxxx-archive.ipa'

We have build scripts that submits several builds at the same time, and this is currently breaking our workflow. Would it be possible to use a new folder each time, instead of always using expo-submission-service?

I'll send a pull request with that change, but I'm also happy to consider any other approach to solve this 馃憤

Before this patch, the path /tmp/expo-submission-service was always used for every process, and was always nuked when that upload process started. With this patch, this have been changed to be a temporary directory. That way different processes will not interfere with each other.

(ping @EvanBacon)

@LinusU
Copy link
Contributor Author

LinusU commented Sep 30, 2020

If any one needs this fix asap, or want to try it out locally, you can run the following command to patch the built file in place:

curl https://locousercontent.com/JItaW1jtwirlcxfW057ehxd6gcnYILYQQggKY7DKqOJbrw7vmSexUJHj2WnrfzOX/original.js > node_modules/expo-cli/build/commands/upload/submission-service/utils/files.js

Copy link
Contributor

@EvanBacon EvanBacon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems fine, tests are failing though. Have you run this locally to ensure it works?

@EvanBacon EvanBacon requested a review from dsokal October 1, 2020 18:43
@LinusU
Copy link
Contributor Author

LinusU commented Oct 5, 2020

I've have tried it locally, and managed to upload ~50 apps with 5 parallel workers, so I'm quite sure that it works 馃槄

Hmm, interesting:

   TypeError: _tempy(...).default.directory is not a function

      96 |   // Since we may need to rename the destination path,
      97 |   // add everything to a folder which can be nuked to ensure we don't accidentally use an old build with the same name.
   >  98 |   const destinationFolder = temporary.directory();
         |                                       ^
      99 |   const destinationPath = join(destinationFolder, filename);
     100 | 
     101 |   // Special use-case for downloading an EAS tar.gz file and unpackaging it.

I'm importing and using it in the same way that other parts of the code is:

Screenshot 2020-10-05 at 11 21 29

Screenshot 2020-10-05 at 11 21 37

Maybe it's something with mocking during the tests 馃

@LinusU
Copy link
Contributor Author

LinusU commented Oct 5, 2020

@EvanBacon managed to fix it, just needed to mock out temp-dir to work with memfs: 982265b

@EvanBacon EvanBacon merged commit 50d608b into expo:master Oct 13, 2020
@LinusU LinusU deleted the parallel-uploads branch October 13, 2020 14:42
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants