Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement types-as-a-service and check bundles with generated types #1959

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

mrbbot
Copy link
Contributor

@mrbbot mrbbot commented Apr 4, 2024

Hey! 👋 This PR implements a Worker for generating TypeScript types given a compatibility date and set of compatibility flags. The aim here is to remove the confusing entrypoints system from @cloudflare/workers-types, and replace this with...

  1. A deployed Worker for generating types based used by the Workers playground and quick editor.
  2. A command that runs this Worker locally and automatically generates types for the compatibility_date and compatibility_flags in your wrangler.toml, using the user's locally installed version of workerd.

Our previous type generation didn't support generating types for built-in modules. Given lots of features are now within modules (cloudflare:workers, nodejs_compat, etc), this doesn't quite cut the mustard anymore. This new system supports generating types for C++ and TypeScript modules! 🎉 This means...

  • We can now type-check built-in TypeScript bundles in /src/node and /src/cloudflare with auto-generated types, as opposed to handwritten definitions and @types/node
  • Users won't need to include @types/node when using nodejs_compat, and will only get types for Node APIs we support

Type checking our own code with types generated from our code risks introducing circular build dependencies. To get around this, this PR splits up registering C++ modules (index.h) and registering modules from bundles (index-bundles-rtti.h). api-encoder.c++ only includes index.h, whereas the workerd:rtti module includes both. A new //types:types_internal entrypoint has been added that uses the output from api-encoder.c++ to generate a version of the types containing definitions from C++ concatenated with /types/defines/*.d.ts. Bundles are type checked with this, then included in a new //src/workerd/api:index_bundles_rtti target which is referenced by //src/workerd/server:server.

The PR doesn't change the format of the @cloudflare/workers-types package for now—we still generate multiple entrypoints for each compatibility date that changes the public API surface. We'll probably want to deprecate this package at some point, in favour of the new yet-to-be-written tool for generating types locally.

Potential Improvements

  • Cache generated types in worker by workerd version and compatibility flags
  • Auto-include comments from @types/node like we do with lib.webworker.d.ts
  • Skip types generation on non-opt builds/tests (the Worker is ~10MB uncompressed and unminified, which is pretty slow to load/run in debug builds)

@fhanau
Copy link
Collaborator

fhanau commented Apr 4, 2024

Thank you for doing this! I'd be happy to contribute to reviewing this, but the sheer size of the change will make this more difficult. Could this be split up into several commits/PRs? AFAICS at least part of this is linting/refactoring of existing code that may be intended for the types worker but could be landed independently first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants