Improve typings: ESM, `AcornJsxParser` class and `tokTypes` const #130

brettz9 · 2022-02-10T11:01:49Z

As explained at acornjs/acorn#1104 , we're looking to add type support to Espree (and other projects in the ESLint space), and in the case of your project, it looks like we could use this missing class and ESM import ability (especially since limiting ourselves to basic JSDoc for the TS declaration file generation makes it more useful not to have to reimplement types ourselves).

I'm pretty new to using TypeScript, so hope I've gotten things done all right. It seems I needed to remove the export = jsx and instead use export default jsx; to support ESM, but hopefully someone more competent in TS can give it a good look.

Also, the PR doesn't cover all methods or properties, but I hope this will at least improve things (and at least meets our immediate needs).

Best wishes...

0. depends on acorn and acorn-jsx being updated per PRs a. acornjs/acorn#1104 (merged but not yet released) b. acornjs/acorn#1105 c. acornjs/acorn-jsx#130 1. Refactor to avoid `typedef` shortcuts when not desired for export 2. Avoid dummy class, so parse JavaScript with https://github.com/es-joy/jsdoc-eslint-parser feat: Adds JSDoc-based TypeScript declaration file Also: 1. chore: adds `editorconfig` 2. refactor: Removes unused esprima 3. refactor: changes to force EspreeParser constructor to convert `new String` to plain string (for typing) 4. refactor: switches to `Object.keys` to avoid `hasOwnProperty` and easier for typing 5. refactor: drops a use of `acorn.Parser.extend` for typing purposes 6. refactor: checks for existence of `tokens` in `tokenize` (as may be absent) 7. refactor: checks for existence of `firstNode.range` and `firstNode.loc` in `parse` (as may be absent) 8. refactor: checks for existence of `extra.lastToken.range` and `extra.lastToken.loc` in `parse` (as may be absent) 7. feat: throws specific error if `jsx_readString` superclass undefined 8. refactor: checks for existence of `lastTemplateToken.loc` and `lastTemplateToken.range` in `token-translator.js` (as may be absent)

RReverser · 2022-02-12T19:09:29Z

index.d.ts

+  jsx_readString(quote: number): void;
+}
+
+export as namespace jsx;


Interesting, TIL this works.

This declares jsx is available as a global namespace, which is incorrect.

0. depends on acorn and acorn-jsx being updated per PRs a. acornjs/acorn#1104 (merged but not yet released) b. acornjs/acorn#1105 (merged but not yet released) c. acornjs/acorn-jsx#130 1. Refactor to avoid `typedef` shortcuts when not desired for export 2. Avoid dummy class, so parse JavaScript with https://github.com/es-joy/jsdoc-eslint-parser feat: Adds JSDoc-based TypeScript declaration file Also: 1. chore: adds `editorconfig` 2. refactor: Removes unused esprima 3. refactor: changes to force EspreeParser constructor to convert `new String` to plain string (for typing) 4. refactor: switches to `Object.keys` to avoid `hasOwnProperty` and easier for typing 5. refactor: drops a use of `acorn.Parser.extend` for typing purposes 6. refactor: checks for existence of `tokens` in `tokenize` (as may be absent) 7. refactor: checks for existence of `firstNode.range` and `firstNode.loc` in `parse` (as may be absent) 8. refactor: checks for existence of `extra.lastToken.range` and `extra.lastToken.loc` in `parse` (as may be absent) 7. feat: throws specific error if `jsx_readString` superclass undefined 8. refactor: checks for existence of `lastTemplateToken.loc` and `lastTemplateToken.range` in `token-translator.js` (as may be absent)

brettz9 · 2022-02-12T22:22:41Z

I've now added missing ~~properties and~~ methods as well as fixed the return value of the constructor to give typeof AcornJsxParser instead of typeof acorn.Parser

0. depends on acorn and acorn-jsx being updated per PRs a. acornjs/acorn#1104 (merged but not yet released) b. acornjs/acorn#1105 (merged but not yet released) c. acornjs/acorn-jsx#130 1. Refactor to avoid `typedef` shortcuts when not desired for export 2. Avoid dummy class, so parse JavaScript with https://github.com/es-joy/jsdoc-eslint-parser feat: Adds JSDoc-based TypeScript declaration file Also: 1. chore: adds `editorconfig` 2. refactor: Removes unused esprima 3. refactor: changes to force EspreeParser constructor to convert `new String` to plain string (for typing) 4. refactor: switches to `Object.keys` to avoid `hasOwnProperty` and easier for typing 5. refactor: drops a use of `acorn.Parser.extend` for typing purposes 6. refactor: checks for existence of `tokens` in `tokenize` (as may be absent) 7. refactor: checks for existence of `firstNode.range` and `firstNode.loc` in `parse` (as may be absent) 8. refactor: checks for existence of `extra.lastToken.range` and `extra.lastToken.loc` in `parse` (as may be absent) 7. feat: throws specific error if `jsx_readString` superclass undefined 8. refactor: checks for existence of `lastTemplateToken.loc` and `lastTemplateToken.range` in `token-translator.js` (as may be absent)

brettz9 · 2022-03-09T11:47:50Z

Any idea on chances of being able to give a review? Thanks...

0. depends on acorn and acorn-jsx being updated per PRs a. acornjs/acorn#1104 (merged but not yet released) b. acornjs/acorn#1105 (merged but not yet released) c. acornjs/acorn-jsx#130 1. Refactor to avoid `typedef` shortcuts when not desired for export 2. Avoid dummy class, so parse JavaScript with https://github.com/es-joy/jsdoc-eslint-parser feat: Adds JSDoc-based TypeScript declaration file Also: 1. chore: adds `editorconfig` 2. refactor: Removes unused esprima 3. refactor: changes to force EspreeParser constructor to convert `new String` to plain string (for typing) 4. refactor: switches to `Object.keys` to avoid `hasOwnProperty` and easier for typing 5. refactor: drops a use of `acorn.Parser.extend` for typing purposes 6. refactor: checks for existence of `tokens` in `tokenize` (as may be absent) 7. refactor: checks for existence of `firstNode.range` and `firstNode.loc` in `parse` (as may be absent) 8. refactor: checks for existence of `extra.lastToken.range` and `extra.lastToken.loc` in `parse` (as may be absent) 7. feat: throws specific error if `jsx_readString` superclass undefined 8. refactor: checks for existence of `lastTemplateToken.loc` and `lastTemplateToken.range` in `token-translator.js` (as may be absent)

…and `tokTypes` const

…nting error)

RReverser · 2022-04-28T10:35:16Z

Hey, sorry, following the invasion of my home country I wasn't on Github much nor in the right mindset for reviews.

From a quick look this seems reasonable, although I'm a bit rusty. Did you have a chance to try this on a real-world project or anyone else to look at this, just to be sure?

brettz9 · 2022-04-28T12:09:05Z

My sympathies for such severe difficulties, and very understandable regarding reviews.

I haven't been able to get others to look at it, and I don't have real world uses outside of use within the espree parser (which uses the types internally), but I have confirmed that those additions which are relevant to our project are correctly helping our project enforce the expected types (and they do not do so otherwise). The expected types also show up in my TS-aware IDE (Atom with atom-typescript), whether requiring or importing.

brettz9 · 2022-04-28T12:15:49Z

Just one point I might add--Acorn has merged some updates to some other type changes I offered and released as part of v8.7.1 , so technically the types would require an update of acorn to that version to work as expected, though I expect given your broad peerDependencies support, that you wouldn't want to bump those versions. But just letting you know (that such a version of Acorn is needed for proper TS support).

brettz9 · 2022-05-05T03:31:05Z

@remcohaszing : Having contributed the original TS declaration file, do you think you could take a look at these additions/changes to give an extra confirmation here about whether the type changes should work as expected?

remcohaszing

Thanks for this addition! I believe it’s very valuable to add these types, but I did find some incorrect type definitions.

I made fiddled a bit based on these changes and I believe the following types are correct:

import * as acorn from 'acorn';

declare function jsx(options?: jsx.Options): (BaseParser: typeof acorn.Parser) => new () => jsx.AcornJsxParser

declare namespace jsx {
  export interface Options {
    allowNamespacedObjects?: boolean;
    allowNamespaces?: boolean;
  }

  export const tokTypes: {
    jsxName: acorn.TokenType,
    jsxText: acorn.TokenType,
    jsxTagEnd: acorn.TokenType,
    jsxTagStart: acorn.TokenType
  } & typeof acorn.tokTypes

  export type TokContexts = {
    tc_oTag: acorn.TokContext,
    tc_cTag: acorn.TokContext,
    tc_expr: acorn.TokContext
  };

  export interface AcornJsxParser extends acorn.Parser {
    readonly acornJsx: {
      tokTypes: typeof tokTypes;
      tokContexts: TokContexts
    };

    jsx_readToken(): string;
    jsx_readNewLine(normalizeCRLF: boolean): void;
    jsx_readString(quote: number): void;
    jsx_readEntity(): string;
    jsx_readWord(): void;
    jsx_parseIdentifier(): acorn.Node;
    jsx_parseNamespacedName(): acorn.Node;
    jsx_parseElementName(): acorn.Node | string;
    jsx_parseAttributeValue(): acorn.Node;
    jsx_parseEmptyExpression(): acorn.Node;
    jsx_parseExpressionContainer(): acorn.Node;
    jsx_parseAttribute(): acorn.Node;
    jsx_parseOpeningElementAt(startPos: number, startLoc?: acorn.SourceLocation): acorn.Node;
    jsx_parseClosingElementAt(startPos: number, startLoc?: acorn.SourceLocation): acorn.Node;
    jsx_parseElementAt(startPos: number, startLoc?: acorn.SourceLocation): acorn.Node;
    jsx_parseText(): acorn.Node;
    jsx_parseElement(): acorn.Node;
  }
}

export = jsx;

remcohaszing · 2022-05-05T09:02:48Z

index.d.ts

+  jsx_readString(quote: number): void;
+}
+
+export as namespace jsx;


This declares jsx is available as a global namespace, which is incorrect.

index.d.ts

…a named constant)

… interface

…call signature and `tokTypes` property in place of a separate `const tokTypes`

brettz9 · 2022-05-13T00:46:56Z

Now that I've applied most of your fixes, here's what I think is left of mine differing from yours, @remcohaszing :

I made fiddled a bit based on these changes and I believe the following types are correct:

declare function jsx(options?: jsx.Options): (BaseParser: typeof acorn.Parser) => new () => jsx.AcornJsxParser

declare namespace jsx {

<...SNIP...>

  export const tokTypes: {
    jsxName: acorn.TokenType,
    jsxText: acorn.TokenType,
    jsxTagEnd: acorn.TokenType,
    jsxTagStart: acorn.TokenType
  } & typeof acorn.tokTypes

For mostly aesthetic rather than functional reasons, I changed this to be under one object:

declare const jsx: {
  tokTypes: typeof acorn.tokTypes;
  (options?: jsx.Options): (BaseParser: typeof acorn.Parser) => jsx.AcornJsxParser
}

It seemed to me that putting tokTypes visibly on the same object clears up any confusion some like myself might have in that tokTypes was not a named export. But again, I realize this is probably just my preference.

Regarding AcornJsxParser, there are two ways in which mine differs:

I don't think new () without any arguments would work here, as acorn-jsx reuses acorn's constructor which has its own parameter type expectations.
I've split off the constructor and interface I think by necessity.

For the constructor type, I have this:

  export interface AcornJsxParser extends Pick<P, keyof P> {
    readonly acornJsx: {
      tokTypes: TokTypes;
      tokContexts: TokContexts
    };

    new (options: acorn.Options, input: string, startPos?: number): IAcornJsxParser;
  }

Unless defining a class--which we don't want to do since we want to export the type but not suggest a class is being exported--this was I think necessary because:

With interfaces, TypeScript expects separate types for the constructor and for the instance interface. The static property acornJsx cannot be defined on the instance type as it is actually on the constructor type.
We still need the static acorn property from the Acorn Parser. To get this extension, I tried it with just extends P:

  type P = typeof acorn.Parser;
  export interface AcornJsxParser extends P {
    // <SNIP>
    new (options: acorn.Options, input: string, startPos?: number): IAcornJsxParser;  
  }

...but TypeScript complained for some reason that P and IAcornJsxParser were not the same base type (when one extends something, one would think a different type would be expected, but I guess interface constructor extending is normally expected to just add some static properties).

In any case, we do to all appearances need a separate IAcornJsxParser interface since we're creating instances and instances distinct from acorn's Parser. But though I couldn't get this by extending it with Acorn's Parser directly, I added the extends Pick<P, keyof P> so as to ensure we were grabbing the static acorn property off of acorn's Parser (and any other static properties that might be added in the future). We could thus get all the static properties along with indicating the creation of our own separate interface, IAcornJsxParser.

Then there is the more straightforward instance interface itself:

  export interface IAcornJsxParser extends acorn.Parser {
    jsx_readToken(): string;
    <SNIP>

remcohaszing · 2022-05-13T08:51:55Z

index.d.ts


-declare const jsx: (options?: jsx.Options) => (BaseParser: typeof Parser) => typeof Parser;
+declare const jsx: {
+  tokTypes: typeof acorn.tokTypes;


If this is written as part of the namespace, it can be imported like this when transpiling to commonjs:

import { tokTypes } from 'acorn-jsx'

Whether or not this is better, depends on the library’s intend and is subjective. I personally think that’s slightly better in this case, but I don’t have a very strong opinion on this.

Shouldn’t the type be like this?

tokTypes: jsx.TokTypes

Regarding use of named exports, Node.js docs state:

CommonJS modules consist of a module.exports object...

The detection of named exports is based on common syntax patterns but does not always correctly detect named exports. In these cases, using the default import form described above can be a better option.

So I'm wondering if that would be safe.

However, you are right that I should be using the jsx tokTypes instead of the Acorn one here. But it seems then that trying to fix this brings up two problems:

Errors are given suggesting that the jsx in this case will be understood as the exported const rather than the namespace which we want. Renaming the namespace to jsxNS, fixes the error but then the namespace isn't exported.

And because only one export is allowed at the top level, I can't apparently move interface TokTypes to the top level without being redundant (i.e., I can have one copy at the top level which is not exported and one copy within the namespace which, as you say, is auto-exported).

I solved this I think by reexporting the non-exported interface with an exported type alias. Let me know if you think this is all suitable.

index.d.ts

remcohaszing · 2022-05-13T09:10:23Z

index.d.ts

+
+  type tokTypes = typeof acorn.tokTypes;
+
+  export interface TokTypes extends tokTypes {
+    jsxName: acorn.TokenType,
+    jsxText: acorn.TokenType,
+    jsxTagEnd: acorn.TokenType,
+    jsxTagStart: acorn.TokenType
+  }


I suggest to do the following:

Suggested change

type tokTypes = typeof acorn.tokTypes;

export interface TokTypes extends tokTypes {

jsxName: acorn.TokenType,

jsxText: acorn.TokenType,

jsxTagEnd: acorn.TokenType,

jsxTagStart: acorn.TokenType

}

export interface TokTypes extends typeof acorn.tokTypes {

There’s no need to export the tokTypes type from acorn. Also it’s generally preferred to use the CamelCase for type names, which isn’t used for tokTypes. By not exporting them, this is avoided altogether.

The reason I added a type alias was because I get an "expression expected" error if directly adding it as in your suggestion. See microsoft/TypeScript#14757 on why TypeScript isn't supporting this.

Should I move the type alias out of the namespace so it is not exported then?

And if so, would you prefer for CamelCase that I change it to AcornTokTypes?

Also, and I guess this might have been inadvertent in making the commit suggestion, we don't want to delete the properties in the interface.

The reason I added a type alias was because I get an "expression expected" error if directly adding it as in your suggestion. See microsoft/TypeScript#14757 on why TypeScript isn't supporting this.

Should I move the type alias out of the namespace so it is not exported then?

And if so, would you prefer for CamelCase that I change it to AcornTokTypes?

I've gone ahead and added a commit to do both of these things. Let me know if this is ok.

remcohaszing · 2022-05-13T09:14:16Z

index.d.ts

+    jsxTagStart: acorn.TokenType
+  }
+
+  export interface Options {


Types in a namespace are always exported, so the export keyword is redundant.

It’s not wrong, but personally I prefer to remove them.

Ok. Applied. TS unfortunately isn't very helpful in their docs in helping give a better understanding of ambient namespaces like this; it seems to only speak there of browser global usage. I guess this may be because modules are moving to become the preferred way, and the fact that we still need to understand the old way when in use in CJS was lost along the way.

remcohaszing · 2022-05-13T09:16:12Z

The types are correct now. I just added some comments on some stylistic nits.

…es within namespace, rename to use Ctor convention

brettz9 · 2022-05-15T04:40:15Z

I've applied the stylistic changes. There is, per your question, indeed, one non-stylistic issue remaining, however (re: jsx.TokTypes).

… the `tokTypes` type

remcohaszing

LGTM

brettz9 · 2022-11-06T22:09:37Z

@RReverser : With an additional review (by @remcohaszing) having been done, are you ok to merge?

RReverser

Yes, and thanks for the ping - haven't looked much at Github back in May and things got lost afterwards.

RReverser · 2022-11-06T23:43:58Z

Actually, sorry - one question - I understand the need to expose token types, but can you explain why the jsx_* methods need to be exposed in the typings? They're kind of implementation details, does something in espree depend on accessing those methods directly?

brettz9 · 2022-11-07T03:02:27Z

espree is extending the acorn or acorn-jsx parser class. Unfortunately, the branch I'm working on is in an unstable state atm, so I can't confirm now that those methods were necessary. I seem to recall at least some were, but I could be mistaken.

RReverser · 2022-11-09T15:09:25Z

I suppose there isn't much harm - acorn-JSX remained pretty much unchanged for years - just slightly worried about exposing implementation details as typing become part of semver.

RReverser · 2022-11-14T16:39:58Z

Given that it hasn't changed in a while, I suppose this is fine.

brettz9 mentioned this pull request Feb 12, 2022

@types for espree? eslint/espree#529

Open

brettz9 force-pushed the d.ts branch from 41f88a4 to 67f0988 Compare February 12, 2022 05:54

brettz9 changed the title ~~feat: Support ESM declaration imports; add an AcornJsxParser class~~ feat: Support ESM declaration imports; add an AcornJsxParser class and tokTypes const Feb 12, 2022

RReverser reviewed Feb 12, 2022

View reviewed changes

brettz9 mentioned this pull request Apr 26, 2022

feat: Programmatic building of type-checkable JS and declaration files eslint/espree#544

Closed

brettz9 added 7 commits April 26, 2022 14:50

feat: Support ESM declaration imports; add an AcornJsxParser class …

7b59516

…and `tokTypes` const

feat: add missing tokContexts property on acornJsx

dda96cc

feat: add missing jsx_ methods to AcornJsxParser

baa0a61

feat: add instance properties to AcornJsxParser

8526524

fix: return AcornJsxParser from constructor

def9edd

drop properties belonging on Acorn instead

2383cca

fix: use import with namespace specifier for acorn (avoids interop li…

e4eca02

…nting error)

brettz9 force-pushed the d.ts branch from b9a87f8 to e4eca02 Compare April 26, 2022 06:50

remcohaszing suggested changes May 5, 2022

View reviewed changes

brettz9 added 5 commits May 6, 2022 09:48

fix: switch tokContexts to type TokContexts (as not available as …

d97bd96

…a named constant)

refactor: avoid exporting jsx as named export

620551b

fix: in place of exporting non-existent class, export constructor and…

835ea91

… interface

refactor: use without prefix

5459984

refactor: export TokTypes interface and a single jsx export with …

af6247d

…call signature and `tokTypes` property in place of a separate `const tokTypes`

fix: drop extra export as namespace

d80ceef

Redracer93 approved these changes May 12, 2022

View reviewed changes

fix: use exports = jsx (and use jsx.Options)

1c2fa2c

remcohaszing reviewed May 13, 2022

View reviewed changes

brettz9 added 3 commits May 15, 2022 12:08

refactor: prefer interface over type, avoid redundant export on typ…

4cfaf9f

…es within namespace, rename to use Ctor convention

refactor: avoid extra type alias

41e8fcd

refactor: avoid exporting tokTypes and use CamelCase

71b7e61

brettz9 added 2 commits May 15, 2022 12:40

refactor: for now add a commit which redundantly but safely indicates…

b976601

… the `tokTypes` type

refactor: remove some redundancy

fd68170

remcohaszing approved these changes May 16, 2022

View reviewed changes

RReverser approved these changes Nov 6, 2022

View reviewed changes

RReverser changed the title ~~feat: Support ESM declaration imports; add an AcornJsxParser class and tokTypes const~~ Improve typings: ESM, AcornJsxParser class and tokTypes const Nov 14, 2022

RReverser merged commit 8ed96d6 into acornjs:master Nov 14, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve typings: ESM, `AcornJsxParser` class and `tokTypes` const #130

Improve typings: ESM, `AcornJsxParser` class and `tokTypes` const #130

brettz9 commented Feb 10, 2022

RReverser Feb 12, 2022

remcohaszing May 5, 2022

brettz9 commented Feb 12, 2022 •

edited

brettz9 commented Mar 9, 2022

RReverser commented Apr 28, 2022

brettz9 commented Apr 28, 2022

brettz9 commented Apr 28, 2022 •

edited

brettz9 commented May 5, 2022

remcohaszing left a comment

remcohaszing May 5, 2022

brettz9 commented May 13, 2022 •

edited

remcohaszing May 13, 2022

brettz9 May 15, 2022

brettz9 May 15, 2022

remcohaszing May 13, 2022

brettz9 May 15, 2022 •

edited

brettz9 May 15, 2022

brettz9 May 15, 2022 •

edited

remcohaszing May 13, 2022

brettz9 May 15, 2022

remcohaszing commented May 13, 2022

brettz9 commented May 15, 2022

remcohaszing left a comment

brettz9 commented Nov 6, 2022 •

edited

RReverser left a comment

RReverser commented Nov 6, 2022

brettz9 commented Nov 7, 2022

RReverser commented Nov 9, 2022

RReverser commented Nov 14, 2022

Improve typings: ESM, AcornJsxParser class and tokTypes const #130

Improve typings: ESM, AcornJsxParser class and tokTypes const #130

Conversation

brettz9 commented Feb 10, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brettz9 commented Feb 12, 2022 • edited

brettz9 commented Mar 9, 2022

RReverser commented Apr 28, 2022

brettz9 commented Apr 28, 2022

brettz9 commented Apr 28, 2022 • edited

brettz9 commented May 5, 2022

remcohaszing left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brettz9 commented May 13, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brettz9 May 15, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brettz9 May 15, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

remcohaszing commented May 13, 2022

brettz9 commented May 15, 2022

remcohaszing left a comment

Choose a reason for hiding this comment

brettz9 commented Nov 6, 2022 • edited

RReverser left a comment

Choose a reason for hiding this comment

RReverser commented Nov 6, 2022

brettz9 commented Nov 7, 2022

RReverser commented Nov 9, 2022

RReverser commented Nov 14, 2022

Improve typings: ESM, `AcornJsxParser` class and `tokTypes` const #130

Improve typings: ESM, `AcornJsxParser` class and `tokTypes` const #130

brettz9 commented Feb 12, 2022 •

edited

brettz9 commented Apr 28, 2022 •

edited

brettz9 commented May 13, 2022 •

edited

brettz9 May 15, 2022 •

edited

brettz9 May 15, 2022 •

edited

brettz9 commented Nov 6, 2022 •

edited