Feature request: automatic identity deduction a la google.auth.default()

[danking]

AFAICT, it's currently required to specify exactly one authentication type in fs.gs.auth.type. This is fine for clusters, but annoying for non-expert users who go back and forth between local prototyping on a laptop and execution on a cluster. For those non-expert users, they have to figure out where spark-defaults.conf is stored and edit that file. To edit that file they have to further know that the configuration variable must be preceded by spark.hadoop..

In contrast, the google-auth Python library provides google.auth.default() which allows users and libraries to request credentials in an environment independent way. It just "does the right thing".

Concretely the request is for:

1. A "DEFAULT" option to fs.gs.auth.type which uses the exact same precedence as google.auth.default(), and
2. That the default setting for fs.gs.auth.type be "DEFAULT"