(WIP): Add ability to generate documentation from auto-generated typings

[MathBunny]

Summary

Adds the ability to generate documentation web-pages from the auto-generated typings.

Currently, the repository is an intermediate state where all but some of the services have the ability to auto-generate typings (namely auth, machine-learning and storage cannot). In the current TypeDoc setup, we cannot generate the documentation web-pages when relying only on the auto-generated d.ts files.

Note: This is still WIP.

Background

The first observation is that the generated filenames were incorrect; more specifically the namespace prefix was missing (i.e., credential.html vs admin.credential.credential.html).

Formerly, modules mode for TypeDoc was attempted but it was found that some of the generated .html files had different composition, so another approach is taken here.

In short, the purpose of choosing modules is the generated typings from TypeDoc doesn't including the namespace prefix in the filename, but by switching to modules the folder name is prefixed allowing us to interpolate, as here:

firebase-admin-node/docgen/generate-docs.js

Lines 293 to 373 in ebf660d

	/**
	* TypeDoc needs to be executed in module mode to determine the namespace
	* prefix in filenames when the typings are auto-generated. However,
	* there needs to be extra processing for normalizing it to
	* follow the filename convention established.
	*
	* For example: admin.remoteConfig.RemoteConfigUser.html
	*
	* There are two cases to consider:
	*
	* Case #1: _auth_d_.admin.auth.auth.html
	* The namespace is included after _d_ We pick the _d_. to be a delimiter
	* and only retain the substring after it. This is used when there is a
	* manually curated .d.ts file.
	*
	* Case #2: _remote_config_remote_config_api_client_d_.remoteconfigparameter.html
	* We pick the _d_. to be a delimiter, determine that remote-config is the
	* namespace by examining folders in src, and then prepend it to the suffix.
	* This is used for auto-generated typings.
	*/
	function removeModulePrefix() {
	fs.readdirSync(docPath).forEach(filename => {
	const delimiter = '_d_.';
	const cutoff = filename.indexOf(delimiter);
	if (cutoff !== -1) {
	const prefix = filename.substring(0, cutoff);
	const suffix = filename.substring(cutoff + delimiter.length);
	// If the suffix already contains admin, then we can simply strip
	// the prefix (case 1). Otherwise, we need to infer the namespace based
	// on the prefix (case 2).
	// _remote_config_remote_config_api_client_d_.remoteconfigparameter.html
	// ^-------------------------------------^ ^------------------------^
	// prefix suffix
	if (suffix.indexOf('admin') !== -1) {
	const renamedFilename = filename.substr(cutoff + delimiter.length);
	fs.renameSync(`${docPath}/${filename}`,
	`${docPath}/${renamedFilename}`);
	} else {
	const srcPath = `${repoPath}/src`;
	const tokens = prefix.split('_');
	const services = fs.readdirSync(srcPath).filter(
	f => fs.statSync(path.join(srcPath, f)).isDirectory());
	let serviceName = '';
	let found = tokens.some((token) => {
	if (serviceName.length !== 0) {
	serviceName += '-';
	}
	serviceName += token;
	if (services.includes(serviceName)) {
	return true;
	}
	});
	// Despite being located in the respective folder, the class may not
	// belong to a namespace and its inferred namespace should be
	// discarded.
	// For example: firebase.google.com/docs/reference/admin/node/TopicMessage
	const noServiceNameList = [
	'topicmessage.html', 'tokenmessage.html', 'conditionmessage.html'
	];
	if (noServiceNameList.includes(suffix)) {
	serviceName = '';
	}
	if (!found) {
	console.log(`ERROR could not infer the namespace for ${prefix}`);
	} else {
	const camelCaseServiceName = serviceName.split('-').join('');
	// Handles case of file without a namespace.
	const renamedFilename = camelCaseServiceName.length !== 0 ?
	`admin.${camelCaseServiceName}.${suffix}` : suffix;
	fs.renameSync(`${docPath}/${filename}`, `${docPath}/${renamedFilename}`);
	}
	}
	}
	});
	}

Approach

This PR takes the approach of using TypeDoc's file mode as it is using today. Now, the src folder is traversed and the barrel export files are introspected to determine the namespace prefix instead.

The technique chosen is to build an index mapping type-names to their parent folder name based on src. The underlying assumption for this is that the folder-name corresponds to the namespace (does not apply for credential but I addressed it in another PR) and type names are unique (does not apply for query so far, but this can be manually patched).

---

Additional Issues

1. Hyperlinks are broken for links generated by TypeDoc (can be resolved by using previously constructed index).
2. Interfaces are incorrectly labelled as implements: any (can be resolved via regex)
   • Incorrect header as it has a missing namespace prefix
   • INTERNAL should not appear (can be resolved via regex)
   • Accessors is actually Properties
   • Missing top-level documentation example

Modularized (Local)	Production (Devsite)

3. app: FirebaseApp vs app: _admin.app.App (can be resolved via regex)
4. Even when re-exporting classes as interfaces, constructors still appear (can be resolved via @hidden)
5. When using export import pattern, the exported types appear as variables (can be resolved via regex)
   • Minimal reproducible example: MathBunny/typedoc-namespace-bug-demo
   • GitHub issue: TypeStrong/typedoc#1353

Modularized (Local)	Production (Devsite)

6. The version number needs to be updated for TypeDoc in order for export import to even appear (went up from 0.15.0 to 0.18.0)
   • Manually changed one of the flags to hide private members as they appear on doc pages as of 0.16.0
7. Different behavior for inheriting from non-exposed types.
   For example:

Modularized (Local)	Production (Devsite)

8. RTDB types are not exported (similar to Firestore).

Several other minor fixes:

1. Resolved the broken .html link on index.html for service pages when run locally

[MathBunny]

Several notes:

1. Added @hidden to all the INTERNAL field variables and removed the associated regex expressions for removing them.
2. Changed the algorithm for building the namespace index, instead now the toc.yaml is leveraged rather than observing the barrel exports in src folders.
3. The links are broken in the service overview pages under variables, but I thought of not touching it yet until the bug is fixed on TypeDoc. The behavior is that it links to a relative section in the current page rather than the actual interface (i.e., message.html#className versus message.className.html.

[hiranya911]

Superseded by #1072