Features support data source

[babel-bot]

Issue originally reported by @chicoxyzzy in babel/babel-preset-env#117

Currently we use compat-table as a source of feature support data source.

There are some issues with this approach:

• currently we are use outdated info
• babel-preset-env needs build script to be maintained and updated every time compat-table changes
  • current build script is outdated
  • we consider ES6 features only, for ES2016+ features we need to modify build script
  • every change in compat-table can break build script (see recent example)
  • mapping from test names to babel features names may break
  • there is no any workflow in compat-table contributing that guarantees that changes will be reviewed before merging (some commits are pushed directly to gh-pages branch)
• compat-table wasn't designed to be a data source and it's not easy to rewrite it because it's data, build script and UI are coupled

I'm going to create an alternative to compat-table which:

• will hold all necessary data in npm as json
• will be modular (one feature - one json file) so projects like babel-preset-env will be able to use only part of data they need
• will use test262 instead of alternative tests because
  • tests unlikely will change their name
  • these tests are used by browsers vendors so they are more precise
  • every new test262 test can be automatically handled and added to feature support data

Any thoughts?

[babel-bot]

Comment originally made by @hzoo

Well I would of kept it on gh-pages branch but you said at the time to fix it to a specific sha. Definitely can be improved but because we have to build it manually we can at least create a diff if something is wrong before publishing so nothing automatically breaks

[babel-bot]

Comment originally made by @chicoxyzzy

@hzoo yes, I'll update current build script before I'll start working on that new project

[babel-bot]

Comment originally made by @chicoxyzzy

cc @ljharb @zloirock @kangax @webbedspace I would love to see what do you think!

For now I think the best solution is to support both compat-table and a new project which goal is to provide data to any other projects (new version of compat-table could be build on top of it too). compat-table is definitely best source of info for now but we can do better!

[babel-bot]

Comment originally made by @amilajack

I hope this doesn't segment the community. compat-table is already pretty popular and has a lot of traction. What I would really like is something like is to have the estree AST node type with the feature. For example, SIMD would have something like:

{
  name: 'SIMD',
  ast: {
    node: 'MemberExpression',
    id: 'SIMD'
  }
  // ..
}

This would make it much easier for tooling to take advantage of it

[babel-bot]

Comment originally made by @ljharb

the way compat-table is currently built, it'd be really hard to update the data in the resulting webpage - ie, we'd need an otherwise unnecessary commit to trigger the auto-deployment.

I do think it would be reasonable to create a directory inside compat-table for the data, and publish that for other things to consume.

[babel-bot]

Comment originally made by @chicoxyzzy

@ljharb I like this idea and that is exactly what I started to do in compat-table/compat-table#977. It needs some further work though and I'm going to finish it so it'll be easier for people to use in their projects. However I still think we can do better and create new data source which will be more stable and won't require a lot of refactoring each time something changes. It won't require to repeat things happening in test262 repo

[babel-bot]

Comment originally made by @kangax

I haven't been following compat-table development lately but the idea has always been to extract data into a separate repo for easy npm-based (or other) consumption, just like @ljharb mentions.

Modularity — and not having to download unnecessary data — is nice, but I'm not sure why test262 would be better? test262 tests are very granular so they'd need to be somehow grouped anyway, which is basically what compat-table does (with less precision and "unofficially").

I'm all for better solutions but adding yet another source of truth (test262, compat-table, something else) might not the best thing for the users :) Unless, of course, we make compat table delegate to that new data. But then, why not just change compat table data in the first place?

Please correct me if I misunderstood something.

[babel-bot]

Comment originally made by @hzoo

not sure about test262 as well. Getting json or just the data would be good for env. We have already had discussion a bit about that granularity/grouping - when do we decide that Babel should transform rather than use native?

If a browser implements most but not all of the spec? (failing tests)?
Or if Babel itself doesn't implement something correctly either? Basically babel/babel-preset-env#30 but then there are cases where things aren't so independent

(and fyi a lot the people commenting here have worked on compat-table)

[babel-bot]

Comment originally made by @amilajack

A solution I thought of was writing a small module that makes network requests to compat-table, but this comes with its own set of issues.

• Pros
  • Always fetching latest version of compat-table
• Cons
  • Requires network connection
  • network request significantly slower than disk io

[babel-bot]

Comment originally made by @hzoo

I didn't want to auto fetch compat-table anyway since there can be bugs/issues so being able to diff the same version helps to find those (a bug in either env or compat-table). It's easy to just release a new version

[babel-bot]

Comment originally made by @hzoo

(Don't know why that got closed..)

[babel-bot]

Comment originally made by @kangax

I thought that was your way of dropping the mic 😂

[babel-bot]

Comment originally made by @chicoxyzzy

Thank you all for chiming in!

@kangax there are some problems with current version of compat-table. For example something may change in proposal on esnext tab and tests become obsolete. Those tests will be wrong until someone notice that and fix them. This may happen even after moving proposal to es2016+ tab. Another examples are when we split one test into subtests or change test name. Apps depending on that data will be broken.

@hzoo I agree that autofetch is a bad idea. However we can use semver as caniuse do.

I don't want to create another compat-table. Actually what I want to research if we can use test262 harness to collect historical data an use it.

Let's assume proposal we have new test for new feature:

• Someone adds tests to test262
• Someone adds tests to compat-table (it's already an issue here because these tests my vary more and more over time)
• We generate raw data from compat-table
• Now other projects can use this data (compat-table can use it too)

As I said tests will vary over time. Also compat-table may change a lot (split test to subtests and change test name). Now every project that uses compat-table data should fix it's code.

That is a lot of mechanical work to do by hands in every project.

That's what we can improve:

• Someone adds test to 262
• We collect test262 harness data
• Projects use that data

If something changes in test262 we can just collect new data. I think it's much more simpler. Of course the will be some problems but they are likely much simper than updating every project by hand.

[babel-bot]

Comment originally made by @chicoxyzzy

The problems for such project will be:

• collecting data from various environments on different platforms
• every test262 harness run take a lot of time
• we still need to test features under flags somehow

[babel-bot]

Comment originally made by @chicoxyzzy

To be clean: I don't propose a compat-table replacement. I propose to create another project which will be created with idea of sharing always actual data for other projects in mind. It should collect this data as much automatically as possible and be stable as possible. So other projects will be able to solve their issues not fixing mapping and parsing of always changing data from compat-table. We can't do compat-table stable enough to prevent this.

[babel-bot]

Comment originally made by @chicoxyzzy

If you'll look at http://kangax.github.io/compat-table/es2016plus/ you'll see that it's broken. It was usable three commits before but now it requires a lot of work, then other projects which use it (let's assume we have a lot projects depend on compat-table) should update. It's a lot of work for many people. We can improve this

[babel-bot]

Comment originally made by @chicoxyzzy

In other words other projects shouldn't rely on compat-table because of unpredictable changes in it

[babel-bot]

Comment originally made by @ljharb

Not on master, surely - but they can rely on a commit SHA just fine.

[babel-bot]

Comment originally made by @chicoxyzzy

This means that each project will still need to update it's features mapping and possibly some logic to update to recent commits

[babel-bot]

Comment originally made by @ljharb

They'll have to do that anyways, if proposals change.

[babel-bot]

Comment originally made by @chicoxyzzy

In that case it's easier for them to not rely on any commit but to have their own data.

[babel-bot]

Comment originally made by @chicoxyzzy

But the idea is to share that data among all projects

[babel-bot]

Comment originally made by @ljharb

I don't see why. If a proposal changes such that compat-table data changes in a breaking way, everyone in the ecosystem has to update with the same changes. Whether the compat-table data lives inside the compat-table repo, or in another project, the amount of data churn and breakage is identical, and everyone will still have to identically respond to it.

[babel-bot]

Comment originally made by @xtuc

Was mentionned by @chicoxyzzy and I think this could work well with #4987.

I already solve some problems with the running time or the selection of tests to be run.

It won't cover the entier table.

@ljharb Thanks, I updated.

[babel-bot]

Comment originally made by @Stale[bot]

This issue has been automatically marked as stale because it has not had recent activity 😴. It will be closed if no further activity occurs. Thank you for your contributions 👌!

[chicoxyzzy]

VSCode now uses MDN's compat-data for testing features support. I hope we'll also be able to use compat-data one day. It's much more machine-readable then compat-table which we use currently
https://twitter.com/robertnyman/status/1031561769364410370