refactor!: Revamp apiEndpoint

[danielbankhead]

With the advent of TPC, there’s a lot of complexity with apiEndpoints. To improve, we should expose an async Storage#getEndpoint() method and remove the apiEndpoint/useAuthWithCustomEndpoint properties/parameters.

Additionally, if STORAGE_EMULATOR_HOST is detected and used, a 'passthrough' AuthClient should be used by default (which can be overwritten via the existing authClient parameter) - removing the need for the useAuthWithCustomEndpoint parameter.

This will greatly clean-up the code base as:

• Customers will not be required to manually provide the universeDomain - as we can determine it asynchronously via GoogleAuth#getUniverseDomain. This is a major convenience for TPC customers.
• We can offer a predictable, secure experience by not disabling auth by default when an apiEndpoint has been provided
• Classes will no longer have to distinguish between auth and non-auth contexts - we can simply use auth everywhere uniformly

Related:

• feat!: Align Support for Storage Emulation #2092
• feat: PassThrough AuthClient google-auth-library-nodejs#1771

Examples:

nodejs-storage/src/resumable-upload.ts

Lines 284 to 288 in 755c9c2

	authClient: {
	request: <T>(
	opts: GaxiosOptions
	) => Promise<GaxiosResponse<T>> | GaxiosPromise<T>;
	};

nodejs-storage/src/resumable-upload.ts

Lines 370 to 398 in fcf3b6c

	this.apiEndpoint = `https://storage.${universe}`;
	if (cfg.apiEndpoint && cfg.apiEndpoint !== this.apiEndpoint) {
	this.apiEndpoint = this.sanitizeEndpoint(cfg.apiEndpoint);
	const hostname = new URL(this.apiEndpoint).hostname;
	// check if it is a domain of a known universe
	const isDomain = hostname === universe;
	const isDefaultUniverseDomain = hostname === DEFAULT_UNIVERSE;
	// check if it is a subdomain of a known universe
	// by checking a last (universe's length + 1) of a hostname
	const isSubDomainOfUniverse =
	hostname.slice(-(universe.length + 1)) === `.${universe}`;
	const isSubDomainOfDefaultUniverse =
	hostname.slice(-(DEFAULT_UNIVERSE.length + 1)) ===
	`.${DEFAULT_UNIVERSE}`;
	if (
	!isDomain &&
	!isDefaultUniverseDomain &&
	!isSubDomainOfUniverse &&
	!isSubDomainOfDefaultUniverse
	) {
	// a custom, non-universe domain,
	// use gaxios
	this.authClient = gaxios;
	}
	}

nodejs-storage/src/nodejs-common/util.ts

Lines 157 to 165 in 755c9c2

	/**
	* If true, just return the provided request options. Default: false.
	*/
	customEndpoint?: boolean;
	/**
	* If true, will authenticate when using a custom endpoint. Default: false.
	*/
	useAuthWithCustomEndpoint?: boolean;

nodejs-storage/src/nodejs-common/util.ts

Lines 798 to 809 in 755c9c2

	const authorizeRequest = async () => {
	if (
	reqConfig.customEndpoint &&
	!reqConfig.useAuthWithCustomEndpoint
	) {
	// Using a custom API override. Do not use `google-auth-library` for
	// authentication. (ex: connecting to a local Datastore server)
	return reqOpts;
	} else {
	return authClient.authorizeRequest(reqOpts);
	}
	};

[danielbankhead]

Update: The PassThroughClient is available in Auth v9.7.0:

• chore(main): release 9.7.0 google-auth-library-nodejs#1767