- Sponsor
-
Notifications
You must be signed in to change notification settings - Fork 2.4k
Add fog_options to configuration that can be passed to the fog #2135
Conversation
it "applies the options to the fog #create call" do | ||
files = stub | ||
@dummy.avatar.stubs(:directory).returns stub(files: files) | ||
files.expects(:create).with(has_entries(:multipart_chunk_size => 104857600)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Line is too long. [86/80]
Use the new Ruby 1.9 hash syntax.
This is great, thank you @jeremywadsack! Can you please squash your commits together? Then I'll merge. Thank you! |
Also, can you please add a note in the NEWS file? Thanks! |
38895e7
to
f60eb85
Compare
@tute: Done. Note that branched from 4.3.5 because I couldn't get |
Please do rebase against latest commit in that branch and we merge. Thanks! On Wednesday, March 23, 2016, Jeremy Wadsack notifications@github.com
|
…e method to configure multipart_chunk_size
f60eb85
to
7c153c7
Compare
@tute all ready for you. |
Thank you! :) |
Will cherry-pick your commit into master, for v5 as well. |
We found that uploading large files to S3 would result in a socket error ("connection reset by peer") occasionally and lately much more consistently. In researching this I saw that many people got this error when uploading too large of a file without multipart chunking. I would have assumed fog did this automatically but the default chunk size may be too high. In order to address this I wanted to drop the chunk size to 100MB. Rather than hard-code this I opted to expose a `fog_option` configuration option that lets me pass any additional options I want to the fog's `#create` call. This is similar to the `fog_attributes` option implemented in CarrierWave which [addresses the same problem](http://stackoverflow.com/a/11867978/201911). We've been running this now for a week in production and it seems to resolve the issue. #2135
I put in the master commit as commit message your PR description, which is quite great. Forgot to ask for that in v4.3 branch. |
We found that uploading large files to S3 would result in a socket error ("connection reset by peer") occasionally and lately much more consistently. In researching this I saw that many people got this error when uploading too large of a file without multipart chunking. I would have assumed fog did this automatically but the default chunk size may be too high. In order to address this I wanted to drop the chunk size to 100MB.
Rather than hard-code this I opted to expose a
fog_option
configuration option that lets me pass any additional options I want to the fog's#create
call. This is similar to thefog_attributes
option implemented in CarrierWave which addresses the same problem.We've been running this now for a week in production and it seems to resolve the issue.