New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(cred): Use in built library instead of forced cred #1340
base: master
Are you sure you want to change the base?
Conversation
addab9f
to
34f4c52
Compare
Signed-off-by: dark0dave <dark0dave@mykolab.com>
7bd0625
to
c0ff345
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, have a couple questions to start; am interested in getting this in for sure.
Please also update the documentation once the design is finalized |
6a86f86
to
d5e8ee1
Compare
Can do. |
13655bb
to
fd6702d
Compare
@jschneier thank you for taking the time to review this it is much appreciated. We are using my fork in prod and I would prefer to use the mainline package. Is there anything further needed. Thanks again. |
4d9daf3
to
e8be761
Compare
storages/backends/gcloud.py
Outdated
if self.project_id is None: | ||
self.project_id = project_id | ||
credentials.refresh(requests.Request()) | ||
if not hasattr(credentials, "service_account_email"): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What determines if the hasattr
check fails? Also, if it fails is it an error not to set sa_email
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In some situations (workload identity) if you try to use the default service account the service account email can be missing. In this situation its best to provide the sa_email so you can sign urls.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, you are saying that this must be set in some situation? Can you add that to the documentation.
793debd
to
3d2a913
Compare
@jschneier I have updated |
@@ -54,7 +54,7 @@ dropbox = [ | |||
"dropbox>=7.2.1; python_version<'3.12'", | |||
] | |||
google = [ | |||
"google-cloud-storage>=1.27", | |||
"google-cloud-storage>=2.14", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you link to why we need to bump this requirement?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No. Buts generally good to bump the library.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It’s good to bump your application requirements but as a library I’m trying to maintain as much flexibility for users as possible. If we need a new feature then bumping is worthwhile.
A major version bump in particular could induce work and compat issues to users.
default_params = { | ||
"bucket_bound_hostname": self.custom_endpoint, | ||
params = { | ||
"service_account_email": self.credentials.service_account_email, | ||
"access_token": self.credentials.token, | ||
"credentials": self.credentials, | ||
"expiration": self.expiration, | ||
"version": "v4", | ||
} | ||
params = parameters or {} | ||
|
||
for key, value in default_params.items(): | ||
if value and key not in params: | ||
params[key] = value | ||
|
||
if self.custom_endpoint: | ||
params["api_access_endpoint"] = self.custom_endpoint |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This set of changes reverts using the parameters
bit of url()
. Sorry if there is no test currently covering this.
storages/backends/gcloud.py
Outdated
if self.project_id is None: | ||
self.project_id = project_id | ||
credentials.refresh(requests.Request()) | ||
if not hasattr(credentials, "service_account_email"): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, you are saying that this must be set in some situation? Can you add that to the documentation.
db8f377
to
e016da1
Compare
Signed-off-by: dark0dave <dark0dave@mykolab.com>
Closes #941