New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Roadmap to a v3.0.0 of flow-typed #1494
Comments
Oh man, it's like you read my mind, I had just started writing up a list this weekend. I think you hit many of the current pain points that need to be addressed. There are a few more things I'd like to add as (stretch) goals to a 3.0 launch, so once I give this a good read I'll edit this comment. /cc @jedwards1211 @calebmer @a-tokyo @pascalduez @mwalkerwells |
To be honest, currently I lack the time to help a lot. |
I think this #1471 should be on 3.0 |
@sibelius I like the idea! Question is how far we could automate this for existing type-defs. I guess it is more of an manual task, but I guess we could prepare at least a markdown file automatically by parsing the type-def and prepare a common structured |
It could be done after the release |
@Inkdpixels you must not have seen my PR (#1435) to allow packages to optionally provide their own definitions for flow-typed, instead of storing them in this central repository, so that even private packages can have definitions for multiple versions of flow (which is not possible with I think flow-typed should support both ways of doing things, because:
|
@jedwards1211 That's super cool! 🎉 And you are right, I haven't seen this PR until now! And of course both self-provided and community-type-defs should be supported. Moving the existing community-type-defs of It's all about keeping things simple, otherwise we would end up handling two logical branches on how |
Npm only would simplify things for the user and you guys, but it would make things harder for package maintainers. I can understand if some package maintainers don't want to spend time dealing with flow types, even if it just means accepting PRs for flow type updates. |
I think flow should download typings for all project dependencies itself. Without |
It is great to see some movement in this area. I really like using flow but the handling of type definitions is lagging quite a bit compared to the TypeScript type definitions. On the long term I would love to see the management of the type definitions as part of flow itself as suggested by @XaveScor. From a "user perspective", the type definitions should just be part of the actual dependencies and in a perfect world, I would like to add a new dependency and automa(gi)cally also have the corresponding type definitions available in flow. Each npm package would contain it's versioned type definitions, they will be installed together with all other assets and flow just knows how to access and use them. To get there, the proposal of @Inkdpixels would be a big step forward, using npm could simplify the CLI alot and having the definitions properly described in my I personally would probably move the definitions into separate single responsibility repositories. |
Hey @jedwards1211, thanks for your comment!
I guess there is some misunderstanding going on, as stated, this proposal or having flow-typed type-defs in npm packages do not conflict with your comments at all! 😄 Let's again use the Once the package author changes his mind, he can simply move the community type-defs into the library package and everything will be fine, as your referenced PR also does. Maybe I am missing something, but as far as I can see, there are no drawbacks for anyone, neither us as the maintainers of |
A key point here is that it does not technically need to be moved in its own repo; all definitions could live under the same project very easily, like DT does it. |
@Inkdpixels oh I see. Now that I read your proposal a bit more carefully, I think relying on the same major/minor version in the defs package is risky. For instance, I'm working on type defs for Considering that, I think it would be best to have folders inside In general all the cases I've seen where people try to make one package track the version of another tend to be fraught with problems. |
@gantoine Most definitely, both are options with benefits and drawbacks which we should outweigh against each other - I just tried to keep the comment simple, since both options (mono vs single-repos) where already outlined in my initial post :) @jedwards1211 You are absolutely right, especially after me seeing your PR I do find "keeping the directory structure to resolve the type-defs" as the way to go. My initial thought with the configuration/file-path-pointers was to not restrict people to a certain directory structure, but I guess this is one of those cases where a restriction is better for everybody since it will be immediately clear where type-defs are located! 👍 |
Alright, I went ahead and created a github project for the V3 release. Issues and PRs which have the release tag or are located in the 3.0.0 Project should be discusses as to their inclusion in the next release. I'll list them out for ease of reading. If you feel as if there are other issues which should be addresses for the V3 release, please comment below. There are a small number of proposed enhancements to the CLI which haven't been added, as it may drastically change with this release.
If you'd like to discuss any of the changes, including but not limited to purpose, proposed execution, work distribution/assignment or edge-cases, please comment directly on the related issue(s). |
@Inkdpixels @gantoine I think we also should support |
I like the direction v3 is headed. +1 for a monorepo with lerna or yarn workspaces I'm concerned though about keeping up the quality of libdefs (especially published via a possibly private npm package). Could we still run tests against different versions of flow? |
Adding support for pre-releases, like discussed in #392 would also be something to consider for |
If it's not too late, I want to make sure that being able for libdefs to declare dependencies on other libdefs isn't lost in the v3 work; we've got a bunch of express packages' libdefs that I can't PR because they require definitions like |
I have a similar issue with trying to type the various ProseMirror packages, as many of them depend on each other. |
@revyh you have a good point, I agree that |
@jedwards1211 glad to hear, that you agree. And you right, I forgot that |
@revyh I think the main reason I proposed supporting self-provided defs via |
I maintain a large ecosystem (130~) of libraries that all heavily rely on flow and other packages that have flow. I can't possibly manage pull requests for all of these over time. Right now I'm basically flying blind: Everything is any'd. Whatever it takes to make flow definition sharing easier I'm in, but I have to have this be easier. |
@krainboltgreene I'm not following your complaint, please be more specific about what has to be easier. |
I really like flow and therefore also very much depend on the quality of the flow-typed ecosystem. |
@doberkofler There is progress being made, but this is a major change, so we're still working out the details. The most complex changes will require a major overhaul of the CLI tool (or a complete rewrite), and additional tooling. We want to get it right the first time, and not have to publish multiple breaking changes, forcing contributors and users to re-learn the tool. |
The CLI tool needs a complete rewrite pretty badly. The code is, no offense to anyone, a complete disaster that's nearly impossible to decipher, particularly the runTests command which is full of 100+ line functions that stack await call after await call such that it's impossible to follow the execution context of the CLI utility. There's probably been five/ten times I've wanted to change something in the CLI and just flat gave up because it's not worth the trouble to decipher god modules' god functions' giant async-await stacks. I actually have linked junior developers to runTests.js to demonstrate why async-await isn't the full solution to writing readable async code! This is so oblique that you're chasing contributors away. The things that contain logic like downloading flow binaries, generating flow def files, etc. need to be pulled into generic utilities of some sort - classes, functions, whatever - and those need to be included by the CLI such that there is no coupling between the two. Right now the two are so tightly coupled that it's extremely difficult to modify because it's extremely difficult to test. Sorry for the rant, it's been building for months. I'll volunteer 6-10 hours a week to a CLI rewrite if that's what it takes. |
I think we need to add more contributors to make this repo move faster |
Guys, you can rewrite code. Create PRs. We have another problem. We need more reviewers. @EdwardDrapkin most of ugly code are old flow versions compatibility. I think first step to increate cli quality - drop backward compatibility with outdated flow versions. runTests may be reduce more two times after drop. I fear create the big PR #2193. Because first step - rewrite download flow. I can rewrite fully run-tests. But need more reviewers :( |
@gantoine I’m not sure if the CLI is the biggest problem and I’m also not sure that the result has to be “perfect” the first time. I really believe, that the availability and quality of the definitions is the key problem. A better CLI would definitively help, by improving the process of submitting PR’s for the definitions but from a client perspective I only really miss a proper “outdated” functionality. |
@XaveScor I also never understood the need to support so many old flow versions. At least for me the latest 3 versions would be all I really need and would support. What really bothers me, is the fact that the definition files automatically revert back to the stub versions if no “appropriate” definition file can be found forcing me to keep all my definitions in my local repository and then carefully manually merging them file by file. |
@doberkofler 3 versions is super few. I have projects where we don't updated Flow 2-3 months. I think we need define outdated flow version. For example, flow versions over half a year is outdated. If we decide it we can drop support of 0.0.0-0.53.0 versions. Most of run-tests code if compatibility with <= 0.53. |
The new/improved tool would allow for simpler library creation and submission. And by moving them to NPM packages, we get library deprecation basically for free. No need to apologize, we didn't write it. 😅 In fact, none of us who maintain it were part of the initial build of the CLI. And yeah, it is an absolute mess. |
This is not realistic. We manage about 100+ packages at my work. It's super difficult to use and share typed source code across different versions of flow. Thus, when we upgrade...we have to upgrade everything. And since this is a huge task logistically and politically we're still on v0.53.1. As I've mentioned in the past, the flow-typed CLI could help with this by providing someway to resolve different libdefs for different versions of flow for private packages. In any case, private or not, expecting people to be to have an up-to-date flow version is just not reasonable. That being said, I agree with @XaveScor that we could drop support for <0.53.x since there was such an overhaul. |
@AndrewSouthpaw: I have 124 libraries that I maintain. They all have 100% type annotation. They are all intertwined (seriously the graph is wild). I want to use these libraries and have the type analysis. As I understand it currently if I want flow-typed to "install" these type annotations I have to create a pull request that adds 124 libraries worth of type annotations that I have to write by hand. Then when any of those 124 type annotations changes I have to add between 1..124 new files via pull request. Lets ignore for a moment that my choices are either a single huge PR or 124 PRs and that I have to wait for someone to review that PR. If I don't do any of this (the easiest path) flow-typed will just create stubs. |
@krainboltgreene hard question. I think you can create index.flow.js in your packages. If you leave from your projects community can create package in flow-typed repo. I think official typings should store in source repo. |
@XaveScor Sorry, what question are you referring to? |
@krainboltgreene You could also just create your own type annotations locally, define the library in your If your libraries have Flow annotations already, though, I would expect them to work as-is for type analysis. Do they not? |
Not as I understand it. I welcome critique on my setup to fix this as I
want people who use my work to have type safety. See:
https://github.com/unctionjs
…On Fri, May 25, 2018, 8:07 PM Andrew Smith ***@***.***> wrote:
@krainboltgreene <https://github.com/krainboltgreene> You could also just
create your own type annotations locally, define the library in your
.flowconfig, and maintain it locally. No need to go through flow-typed
and deal with PRs. We don't get the benefit of your work, but that's your
call of whether it's worth your time.
If your libraries have Flow annotations already, though, I would expect
them to work as-is for type analysis. Do they not?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#1494 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAUb2QwUnuuw8ClhSnYk0u4fgkZjRoJnks5t2Mb7gaJpZM4QWGpL>
.
|
Hm, bummer. Well, that's a limitation with flow, and not in our hands unfortunately. |
@EdwardDrapkin I think your experience with async/await call stacks has been biased by running fully transpiled code (as the cli in this project does) rather than using I would recommend reconfiguring the development scripts in this project to run the CLI with |
The last comment here is May 2018, but now is March 2019. Is there any good news for serving types from npm? Looking forward to it so much. |
@zhaoyao91 I gave up waiting and migrated to TypeScript. The quality and availability of the type definitions is just amazing. |
Have to say we did the same thing here. There's some headaches in the migrations, but the quality of tooling is definitely a huge leap forward. |
Here is the current plan: #3083 |
Roadmap to a v3.0.0 of flow-typed
I recently took the time to read through the opened issues and as others have already mentioned I also do think that there is room for improvement in how
flow-typed
works. A few opened issues already address various topics which, in my eyes, would lead to breaking changes and therefore to a new major release (following the semantic release versioning) and currently hinder the flow community as a whole to progress to a matured state (in my eyes).As far as I can see there are no specific plans, at least on GitHub issues, which are dedicated to a release plan or how flow-typed will progress in the longer run. This is why I created this issue.
I would like and could contribute a few weeks of my freelance time to this project in the upcoming three months, but before starting and risking to throw away time and code, it would make sense to agree on a certain set of problems that we want to solve and maybe even to agree on a schedule for this (major) release.
Serving type definitions via npm
While taking a look through the opened issues, I faced that most of them are in regards or at least related to the current implementation of having all definitions inside the repository of
flow-typed
itself, e.g.Both issues stopped at one certain problem, the version pinning between ...
To solve this, I would propose to adapt the proposal of @EdwardDrapkin in his comment on #990.
How do we handle the versioning of the typing and the associated library?
First and foremost we should use existing functionalities as much as possible, e.g. the
peerDependencies
feature of npm/yarn. This way we can specify supported version(-range)s with almost no effort! Secondly I would propose that type definitions will follow a strict semver versioning out of the box.Let's use an imaginary typing package, e.g.
@flowtyped/axios
, by the time of writing this theaxios
library is currently published as version0.17.0
. The first two numbers, namely the major and minor version, should always match the version of the targeted library version that the typing supports...What about bug fixes to the typing?
... the patch version whatsoever can be used independently to release bugfixes of the typing itself, so a case such as the following will be completely valid since the API of axios is still the same and only a bugfix of the typing was published.
Another beneficial result of this would be that users would get automatic updates/fixes in case they prepend the caret (
^
) to their typing versions within thepackage.json
.What about the supported flow version of the typing?
Again I would propose to use the
peerDependency
feature to specify a supported semver range offlow-bin
.But of course this would not allow serving types for multiple versions of flow in the same typing release. To allow this, I would follow @thejameskyle propose in #990 to introduce a mapping within the
package.json
to do this with a few adaptions, e.g.You see that the property itself is named
flow-typed
and notflow
, I think this is a bit safer since otherwise it could conflict in the future if flow itself would be configured in thepackage.json
. I would also add a more genericlatest
key which should always be present, more on this later on.What about packages that do not follow strict semver versioning?
Introducing a boolean property in the
package.json
configuration of the typing could solve this, e.g.Changes to the CLI
Installing new typings via
flow-typed install
Since typings would be served via npm, the
flow-typed install
command would need to be reworked from the ground up. Step-by-step it would ...package.json
for dependencies as it does currently.devDependencies
.npm install
oryarn add
in the background. (This is also something that we would need to make configurable)But how should flow resolve the types within
node_modules
automatically? I think there are two options, either each typing package has it's ownpostinstall
andpreuninstall
script which would copy the files into the${process.cwd()}/flow-typed/npm
folder or we solve this issue in a central place with a new CLI command...Symlinking/copying the installed types into
flow-typed/npm
withflow-typed create-symlinks
This command would ...
grep
through allpackage.json
files in thenode_modules
folder of the project forpackage.json
files with a "flow-typed" section defined.pkg["flow-typed"].types
configuration.flow-bin
version.pkg["flow-typed"].types.latest
version with a warning.latest
, aflow-typed.lock
file will be created with information for the next run (especially to avoid prompts on CI).flow-typed/npm
folder.This way, the following issues would also be solved since the
create-symlinks
will also respectflow-typed
configurations in library packages and not only typings in the@flowtype
scope:So you see the
flow-typed install
command would only be executed upon installing new packages and theflow-typed create-symlinks
command would be run upon setting up a project or during CI / CD.Migrating the old type definitions into separate repositories
Of course this would totally conflict with the current setup, hence a new major release would need to be published, but I think it's not a huge task to create a migration script which will automatically transform the existing type definitions into their own repositories while keeping their git histories using the
git filter-branch
command. Not huge because the existing definitions follow a strict convention in regards to their structure and tests.The only question I have is, if the definitions should be moved into separate git repositories under the
flow-typed
organization or if we create a new mono-repository namelyflow-typed-definitions
in which all definitions could be placed.I wouldn't like to keep the current structure where the CLI is in the same repository as the definitions since this causes way more noise and it makes separating CLI and definition pull requests harder, so I guess a separate mono-repository or even single repositories for each definition would make more sense.
Moving all definitions into a mono-repository
@flowtyped
npm scope and GitHub organization.Moving definitions into single responsibility repositories
semantic-release
andcommitizen
IMO moving them into single responsibility repositories would be more beneficial, but I really don't know how new definitions would be handled, initially authors could just create their own definition repositories on their own and transfer them into the
@flowtype
after a review process or something along the lines, but that is definitely out of my scope.The tooling won't be a problem as far as I can see since it could be abstracted into it's separate CLI, so up to our next point! :-)
Creating a
flow-typed-definition
CLISo in the process of creating the migration script for definitions, most of the logic could be easily abstracted into a
flow-typed-definition
CLI, similar tocreate-react-app
it could be a central point to create new definition repositories with ease and therefore solve [CONTRIB] Make it easier to work on libdefs #146.This CLI could also hold commands for definition repository maintainers, e.g. the test script that we currently hold within
flow-typed
itself (run_def_tests.sh
) and more...Fin.
All of these proposals on how things could work should be seen as a basis for discussions, not as carved in stone and perfectly thought out.
I did not intend to undermine the already existing issues, but to have a central place for discussions and to take a look at them from a greater perspective. While I know that this is a pretty big proposal / change, I do think that the flow community would really appreciate it if we would solve these bigger pain-points/issues sooner than later, which of course should not mean that we should push everything out in a quick-shot move.
Let's take the time to discuss everything and try to find a format for a good
flow-typed
configuration within the project / within type packages that will be extendable in the longer run! :-) ❤️/cc @DullReferenceException @TrySound @marudor @jeffmo @thejameskyle @EdwardDrapkin @bengotow @gtrufitt @lydell @samwgoldman @ryyppy @zertosh @callumlocke @gantoine @doberkofler
The text was updated successfully, but these errors were encountered: