Implement an RFC process for significant changes

[nzakas]

Motivation

Since ESLint started, we've always had a very loose process for making changes. People can open issues for their suggestions and we could then debate in the comments and make a decision. I think that worked well for most of the project's lifetime, but now that the ESLint codebase has grown to be quite complicated, I think it's time to implement a more thorough review process.

Problems with the Current Process

I think issue #10855 is a good example of the problem with the current process for significant changes. I started with a proposal and got great feedback in the comments. I then wanted to update the proposal to make it clear what I would be implementing, so I edited the issue description. Unfortunately, there had already been 👍s on the original description that stuck after I edited it. It was also hard to show how the proposal had progressed, so I ended up using strike-through format to show that, which is a bit difficult on the eyes.

The problem gets worse for issues with long conversations such as #3458 and #3565, both of which have seen multiple proposals and designs discussed, which then spawn more issues to clean up the discussion. I'd argue that these are both significant enough that a design process would be greatly beneficial.

Proposal Overview

I propose that we implement a request for comment (RFC) process similar to, but not exactly the same as, the one used by Ember. Quick overview:

1. Setup a separate RFCs repo.
2. Any significant change would require a design doc to be added to the RFCs.
3. Feedback happens on the pull request, and the design doc is updated based on that feedback.
4. When the design doc is agreed upon, it is merged into the RFCs repo and anyone may start implementing by opening a pull request on the source code repo.

As part of this process, an issue would no longer be required to implement a significant change because all of the details would be in the RFC.

Again, this is just a high-level description and I expect that we would need to iterate on a process that works best for ESLint.

Pros and Cons

I think this approach has several benefits:

1. Having complete design docs that are easy to refer back to later. This is helpful not just during implementation but also after, as a reference of the thinking behind significant changes.
2. Easier to follow feedback gathering and incorporation.
3. Issues on the main ESLint repo will now be for smaller changes and bug fixes, making them easier to deal with.
4. Setting clear expectations about the amount of work required to implement and maintain significant changes.

And there are, of course, some downsides:

1. A bit of misdirection in how to propose significant changes to ESLint. It won't be immediately obvious that an issue isn't the right way to start, so we'd need to work on how to communicate that.
2. Could result in fewer contributions. Whenever a bar is raised to contributing something, it's possible fewer people will contribute. I would argue that dissuading contributions that aren't well-thought-out with a rationale and design is a good thing because it saves the team time and energy, but it's always possible we would miss out on some good stuff too.
3. Another repo to monitor for changes. Hopefully this would be offset by a decrease in issues on the main ESLint repo, but that's hard to predict ahead of time.

Additional Q&A

What is a Significant Change?

We can always debate this, but I would start with:

• Any new CLI feature
• Any new core feature
• Any new rule
• Any breaking change

Each of these carries with it maintenance burden that the team needs to evaluate.

What would a design doc entail?

Again, I think this is something we can debate about, but I think the Ember RFC template is a good starting point. I'd look for:

1. Summary
2. Motivation
3. Detailed Design
4. How to Document
5. Drawbacks
6. Backwards Compatibility Analysis
7. Alternatives
8. Open Questions
9. Help Needed

Thoughts?

Initially I'm just looking for a "hey, this looks like a good idea" or a "no, I don't like this" from the team rather than getting hung up on specific parts of the proposal that we can change.

[not-an-aardvark]

This looks like a good idea. I think it would be great to have a higher bar for documenting big design decisions and making sure the details/alternatives have been considered.

When we discuss the details of this proposal more concretely, I might suggest not requiring an RFC for new rules, but we don't need to get hung up on that now.

[nzakas]

TSC Summary: This proposal seeks to introduce an RFC process for feature development and other significant changes. It would involve creating a markdown document and submitting it to an RFC repo where the feedback would be given and worked into the submission. When the proposal is accepted, it would be merged and remain as an artifact explaining the design and motivation for the change. A PR could then be submitted to the main ESLint repo without opening an issue.

TSC Question: Shall we accept this proposal? (I'm willing to write up the first RFC so we can evaluate the process and make tweaks.)

[ilyavolodin]

Sounds like a more organized way of requesting large changes. But I do share a concern for higher bar of entry and as such lower contributions from the community.

[kaicataldo]

I'm generally in favor of this, as I do think our current process breaks down when a proposal requires significant bikeshedding. My biggest concern is that managing two separate repositories for issues might lead to some confusion, since there's already some confusion regarding documentation (issues about documentation are frequently created on the eslint.github.io repo).

Would we move any existing proposals that are significant changes (as defined above) to the new RFC repository and ask new issue creators to close the issue in the eslint repository and to create a new issue in said repository?

It might be nice to reach out to the Ember team to see what they like/dislike about their process.

[nzakas]

My thought was that the RFC repo would only allow PRs and not allow issues at all, so I think that would make things a bit easier. I would envision for existing open issues that we'd ask for an RFC, and then once a PR is created, close the original issue and direct everyone to put their comments on the pull request from then on.

…

On Thu, Nov 8, 2018 at 6:00 AM Ilya Volodin ***@***.***> wrote: Sounds like a more organized way of requesting large changes. But I do share a concern for higher bar of entry and as such lower contributions from the community. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#11047 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACWkpaQGtYQ57nFVcYO6PlnIQ7UyJl1ks5utDkUgaJpZM4YHMTl> .

--

______________________________ Nicholas C. Zakas @SlickNet Author, Principles of Object-Oriented JavaScript <http://amzn.to/29Pmfrm> Author, Understanding ECMAScript 6 <http://amzn.to/29K1mIy>

[btmills]

We're moving forward with this proposal after discussing it in today's meeting. To start out, we'll do RFCs for new CLI features, new core features, and breaking changes, and once we've iterated on that process, we can consider using it for new rules as well. Thanks @nzakas for volunteering to set up the RFCs repo and templates!

[nzakas]

I've setup the new repo and have submitted a PR with the proposed process and template: eslint/rfcs#2

[nzakas]

This process is now and up running, so closing this issue.