Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ignore Read-Only Filesystems #358

Merged
merged 2 commits into from Apr 27, 2021

Conversation

metaskills
Copy link
Contributor

In repose to #355 where using the Load Path Cache with AWS Lambda results in small differences with the $LOAD_PATH and tries to dumps new data to a read-only filesystem. We can ignore this error but I thought I would outline a few questions that others might have.

Can you use Lambda Containers `/tmp` vs the `/var/task/tmp` directory?
No. Building a container with a `/tmp` directory will be automatically emptied out when the container image instance is loaded by Firecracker.
Can you copy on boot to the `/tmp` directory?
Yes. But that would negate most performance wins.

Example Container/Runtime $LOAD_PATH Differences

Assuming l is the Lambda Runtime load path and d is the ECR image via Docker. The differences would be.

>> d - l
=> ["/var/lang/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib"]
>> l - d
=> ["/var/task", "/var/runtime/lib", "/opt/ruby/lib", "/var/runtime/gems/bundler-2.2.16/lib"]

@metaskills
Copy link
Contributor Author

I did the CLA but I am not sure how to re-run the check.

@ghost ghost removed the cla-needed label Apr 27, 2021
@@ -91,6 +91,7 @@ def dump_data
FileUtils.mv(tmp, @store_path)
rescue Errno::EEXIST
retry
rescue Errno::EROFS
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer a more general solution. I think you can rescue SystemCallError instead.

If there is a file system permission issue or whatever, etc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@casperisfine
Copy link
Contributor

Looks good to me. Did you try this in your prod environment?

Also ultimately I'd suggest to do whatever is necessary so that your runtime $LOAD_PATH is the same as the build time one.

But either way this is a good fix IMHO, and I'll merge on green CI.

@casperisfine casperisfine merged commit 2d40bd3 into Shopify:master Apr 27, 2021
@shopify-shipit shopify-shipit bot temporarily deployed to rubygems April 27, 2021 12:09 Inactive
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants