Support for skipping ADC/using only gcloud auth?

[tfh-cri]

As a developer, when using the plugin in conjunction with twine for uploading packages to a registry, I don't typically want to have the GOOGLE_APPLICATION_CREDENTIALS set to a local key, since I'm using gcloud auth login for most other things. And, developing on a local workstation, I don't have a reachable metadata service either.

In that usage, a typical session looks something like:

-> % twine upload --non-interactive --repository-url  https://europe-west2-python.pkg.dev/MYPROJECT/py-test-repo/ dist/somepackage-2.0.0* --verbose -c 'hello'
Uploading distributions to https://europe-west2-python.pkg.dev/MYPROJECT/py-test-repo/
  dist/somepackage-2.0.0-py3-none-any.whl (261.8 KB)
WARNING:root:Failed to retrieve Application Default Credentials: Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create credentials and re-run the application. For more information, please see https://cloud.google.com/docs/authentication/getting-started
WARNING:root:Trying to retrieve credentials from gcloud...
username set from keyring
INFO:twine.auth:username set from keyring
WARNING:google.auth.compute_engine._metadata:Compute Engine Metadata server unavailable on attempt 1 of 3. Reason: timed out
WARNING:google.auth.compute_engine._metadata:Compute Engine Metadata server unavailable on attempt 2 of 3. Reason: [Errno 113] No route to host
WARNING:google.auth.compute_engine._metadata:Compute Engine Metadata server unavailable on attempt 3 of 3. Reason: timed out
WARNING:google.auth._default:Authentication failed using Compute Engine authentication due to unavailable metadata server.
WARNING:root:Failed to retrieve Application Default Credentials: Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create credentials and re-run the application. For more information, please see https://cloud.google.com/docs/authentication/getting-started
WARNING:root:Trying to retrieve credentials from gcloud...
password set from keyring
INFO:twine.auth:password set from keyring
username: oauth2accesstoken
INFO:twine.repository:username: oauth2accesstoken
password: <hidden>
INFO:twine.repository:password: <hidden>
Uploading somepackage-2.0.0-py3-none-any.whl

Where the ADC error spam and delays checking for a metadata service somewhat hinder the usability of this approach.

If I explicitly set GOOGLE_APPLICATION_CREDENTIALS to something invalid, it appears to skip the metadata check, but still not ideal:

> % GOOGLE_APPLICATION_CREDENTIALS=/dev/null twine upload --non-interactive --repository-url  https://europe-west2-python.pkg.dev/MYPROJECT/py-test-repo/ dist/somepackage-2.0.0* --verbose -c 'hello' 
Uploading distributions to https://europe-west2-python.pkg.dev/MYPROJECT/py-test-repo/
  dist/somepackage-2.0.0-py3-none-any.whl (261.8 KB)
  dist/somepackage-2.0.0.linux-x86_64.tar.gz (148.6 KB)
  dist/somepackage-2.0.0.tar.gz (256.0 KB)
WARNING:root:Failed to retrieve Application Default Credentials: ('File /dev/null is not a valid json file.', JSONDecodeError('Expecting value: line 1 column 1 (char 0)'))
WARNING:root:Trying to retrieve credentials from gcloud...
username set from keyring
INFO:twine.auth:username set from keyring
WARNING:root:Failed to retrieve Application Default Credentials: ('File /dev/null is not a valid json file.', JSONDecodeError('Expecting value: line 1 column 1 (char 0)'))
WARNING:root:Trying to retrieve credentials from gcloud...
password set from keyring
INFO:twine.auth:password set from keyring
username: oauth2accesstoken
INFO:twine.repository:username: oauth2accesstoken
password: <hidden>
INFO:twine.repository:password: <hidden>
Uploading somepackage-2.0.0-py3-none-any.whl

I'm not sure what the best solution that doesn't compromise usability or debugging for the probably-more-common automation/CI uses, but one idea would be checking for an explicit GOOGLE_APPLICATION_CREDENTIALS value and skipping the ADC attempt if it matches (or running it but suppressing output?) I was thinking maybe /dev/null as a flag value, since it shouldn't cause any harm to any other processes if they also happen to read it and attempt ADC.

A separate env var (KEYRING_ARTIFACTREG_AUTH_GCLOUD_FIRST or something?) could also maybe work.

One final option might be checking if the supplied username is some specific value, but that relies on all the keyring-consuming apps to correctly allow for the keyring interface modifying the provided username to get_credential(), which apparently at least Twine doesn't do properly.

See fork/commit referenced below (https://github.com/camresp/artifact-registry-python-tools/tree/poc-skip-adc) for a hacky version of the various options, the 2 env-var variants work, but the username doesn't, at least with twine.

Happy to make a better PR if there's interest in one or more of the approaches.

[megan-kuo]

Hi @tfh-cri,

Thanks for filing this issue!

We're definitely open to ideas for this feature. I'll be working on a doc that will propose the changes needed and will update this bug with more information.

[saorio]

I would like to use this in GCP, so my request is to handle credentials from the metadata server.

[leoskyrocker]

In our org, we install GAR packages locally for development. It is not a good use case to use a service account in this situation because we want to be able to tell which user accessed GAR. So, we are using the gcloud's auth login instead of the application default (adc).

However, currently it slows down the pip interactions if we don't specify the adc since it retries 3 times for every GAR interaction, so this is a legitimate issue for us. Our workaround is to use the gcloud login as the ADC (so we do gcloud auth login --update-adc), but this is more like a workaround we use because of the limitation from this lib.