Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

gcloud backend and App Engine Standard: credentials config problem for buckets with uniform access #909

Closed
przem8k opened this issue Jul 24, 2020 · 3 comments
Closed

gcloud backend and App Engine Standard: credentials config problem for buckets with uniform access #909

przem8k opened this issue Jul 24, 2020 · 3 comments

Comments

@przem8k
Copy link

przem8k commented Jul 24, 2020

I recently integrated django-storages to store user-uploaded files in my personal project on App Engine Standard.

Here's what does work perfectly: if the bucket used to store files has fine-grained access policy and I set GS_DEFAULT_ACL = 'publicRead',everything "just works". In particular, I don't seem to need to set GOOGLE_APPLICATION_CREDENTIALS – the credentials present in the App Engine environment seem to be enough to authenticate the requests.

However, this does not seem to work if the bucket has uniform access policy (which could make sense to use, I want all my files in this bucket to be world-readable):

  • if with uniform access policy we set GS_DEFAULT_ACL = ‘publicRead’, the requests to GCS fail with 4xx, because we're trying to pass an ACL in the request (while the bucket is configured for uniform access policy)
  • if with uniform access policy we set GS_DEFAULT_ACL = None, then django-storages seem to assume that we need to use local credentials to sign the URL and fails with the error below
File "/env/lib/python3.7/site-packages/google/cloud/storage/_signing.py", line 55, in ensure_signed_credentials
"details.".format(type(credentials), SERVICE_ACCOUNT_URL)
AttributeError: you need a private key to sign credentials.the credentials you are currently using <class 'google.auth.compute_engine.credentials.Credentials'>

It seems that to make it possible to use buckets with uniform access policy w/o local credentials (using the ones present in the App Engine environment) it should be possible to configure django-storages so that it doesn't try to sign anything while making the request (like with GS_DEFAULT_ACL = ‘publicRead’) but also doesn't pass the ACL when making the GCS request.

in the meantime, the workaround I found to work is just to use a bucket with fine-grained access control – I wrote some more notes on this @ https://pnote.eu/notes/django-app-engine-user-uploaded-files/ .
@przem8k przem8k mentioned this issue Jul 24, 2020
Open
dulacp added a commit to dulacp/django-storages that referenced this issue Jul 26, 2020
@dulacp
Add a new GS_QUERYSTRING_AUTH param to avoid signing urls
1f2b142 
To support Uniform permissions buckets on Google Cloud Storage, we need to keep `GS_DEFAULT_ACL` to `None`, but it forces each url to be signed, which is useless since the uniform permission is usually meant to give world read access. This new parameter solves this use case reported in jschneier#783, jschneier#846 and jschneier#909
This was referenced Jul 26, 2020
Open
Closed
mands pushed a commit to datapane/django-storages that referenced this issue Nov 10, 2020
@dulacp @mands
Add a new GS_QUERYSTRING_AUTH param to avoid signing urls
221055b 
To support Uniform permissions buckets on Google Cloud Storage, we need to keep `GS_DEFAULT_ACL` to `None`, but it forces each url to be signed, which is useless since the uniform permission is usually meant to give world read access. This new parameter solves this use case reported in jschneier#783, jschneier#846 and jschneier#909
@mands mands mentioned this issue Nov 10, 2020
Merged
@sakimyto
Copy link

I had exactly the same problem. Thank you.

jschneier pushed a commit that referenced this issue Nov 16, 2020
@mands @dulacp
Add setting to disable signed URLs for GCS uniform buckets (#952)
bf30e29 
* Add a new GS_QUERYSTRING_AUTH param to avoid signing urls

To support Uniform permissions buckets on Google Cloud Storage, we need to keep `GS_DEFAULT_ACL` to `None`, but it forces each url to be signed, which is useless since the uniform permission is usually meant to give world read access. This new parameter solves this use case reported in #783, #846 and #909

* Add documentation for the new parameter GS_QUERYSTRING_AUTH

* Minor logic refactor to no_signed_url

Co-authored-by: Pierre Dulac <dulacpier@gmail.com>
@jschneier
Copy link
Owner

Hello, I believe #952, which was just merged, fixes this. Please confirm.

@sakimyto
Copy link

Hello, I believe #952, which was just merged, fixes this. Please confirm.

Looks great. Thanks.

@KrustyHack KrustyHack mentioned this issue Sep 27, 2021
Open
mlazowik pushed a commit to qedsoftware/django-storages that referenced this issue Mar 9, 2022
@mands @dulacp @mlazowik
Add setting to disable signed URLs for GCS uniform buckets (jschneier…
b7906cb 
…#952)

* Add a new GS_QUERYSTRING_AUTH param to avoid signing urls

To support Uniform permissions buckets on Google Cloud Storage, we need to keep `GS_DEFAULT_ACL` to `None`, but it forces each url to be signed, which is useless since the uniform permission is usually meant to give world read access. This new parameter solves this use case reported in jschneier#783, jschneier#846 and jschneier#909

* Add documentation for the new parameter GS_QUERYSTRING_AUTH

* Minor logic refactor to no_signed_url

Co-authored-by: Pierre Dulac <dulacpier@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@przem8k @jschneier @sakimyto