strategy for sbt-typelevel

[armanbilge]

There's been discussions here and on Discord. I just want to try and get something written down.

Disclaimers:

1. All my own opinions/interpretations of said discussions. If I got something wrong, correct me. If it's a good idea, it was probably someone else's (read: @vasilmkd), so credit to them ;)
2. Nothing is set in stone. First of all, because somebody would have to do this work for it to begin to be set :)
3. It's really hard to make everyone happy, but let's try our best :)
4. (Edited) and will be the first of many edits. Lot of content here, so I'll probably keep coming back and trying to update/flesh things out.

This is a really really big yak, but I think we all feel the payoff would be worth it. I'm trying to separate the layers in comments below, and as people find time and energy we can shave them bit-by-bit. This may take a really long time to accomplish, but that's ok :)

Table-of-contents (forthcoming, please wait for comments below):

1. GitHub Actions
2. versioning
3. mima
4. gpg/pgp
5. (sonatype) publishing
6. scalac settings
7. one sbt plugin to rule them all ;) aka sbt-typelevel
8. Other goodies, like websites.

I mentioned it below in #11 (comment): I think it could be fantastic if these can be mostly independent plugins, except for sbt-typelevel of course.

[armanbilge]

GitHub Actions

1. I think biggest concern here is the bus factor. Bringing it under typelevel umbrella will help that.
2. Handle JVM/JS/Native crosses out-of-the-box, besides the scala version and jdk matrices. Hello, tensors!
3. There's probably additional interesting open issues etc., worth looking through at some point.
4. Extract out a pure-generating plugin? There's maybe some interesting direction to make something more general/low-level, so an arbitrary number of completely custom workflows can be defined in sbt. The schema is defined here so we could automatically generate case-classes.

[rossabaker]

2. I've been doing T-shapes for matrices on some projects: all the Scala versions on a default Java, and all the Java versions on a default Scala. Cuts a lot of common matrices from 9 to 5 with little loss of confidence.

[armanbilge]

Done in 53f42e4.

[armanbilge]

Versioning

1. There's discussion of extracting out a versioning plugin from sbt-spiewak in Extract out a versioning plugin djspiewak/sbt-spiewak#3. See also.
2. The baseVersion setting is most important: it enables you to bump your version, e.g. 0.1 -> 0.2 or 3.2 -> 3.3 to indicate you are working on the next series. This has implications for binary compatibility, snapshot-versioning, etc.. Changing this setting communicates this important information to mima plugin, publishing plugin, etc.
3. IMO extracting this out is a very good place to start, and where I personally have my eye at the moment.

[armanbilge]

MiMa

1. Getting this right deserves a plugin of its own. The most important thing seems to be selecting prior artifacts to compare against based on an accurate version number (hence the importance of a versioning plugin).
2. Some projects enforce backwards and forwards binary compatibility within series, so it might be nice to bake that in.
3. My personal agenda here, but I think the filters as currently implemented can be dangerous (see cats 2.7.0 broke binary compatibility for Scala 3 (because mima wasn't run) cats#4062 (comment)). Seems like filters should be tied to specific version changes. Not a priority, just wanted to get it down.

[armanbilge]

GPG/PGP signing

Here's my crude summary:

1. Some of us have keys with passwords, others without. Annoying for anybody to change their keys.
2. Some of us care a lot about signing artifacts, some of us deal with it because that's just how it works.
3. If you care strongly about how artifacts should be signed, please, we want to hear from you. If you don't care, please enhance your calm, we'll try very hard to find a way to make your keys work with whatever signing strategy is decided so you don't have to change them.
4. Based on previously expressed opinions, sbt-gpg seems to be the way to go here. Except, bus factor might not be great there. Meanwhile, sbt-pgp may have caught on to what sbt-gpg got right.

[rossabaker]

That's horrid. Appropriately horrid. 👍

[armanbilge]

BTW, @ChristopherDavenport suggested that we could do the key generation thing in CI. Currently, sbt-gh-actions generates a clean workflow and a CI workflow. It could also generate a third workflow, run by a "dispatch trigger" (i.e., you have to click a button and provide some inputs in web UI) that could:

1. generate a key
2. set it as a repo secret
3. send it to a key server

This would make it extremely easy.

[armanbilge]

Although, not sure if the default runner has permission to set secrets, and adjusting that is too complicated. Also maybe nobody wants to encourage this kind of thing :P

[rossabaker]

Is that a lazy init? Only does it if there's not a secret already populated? That sounds nice. Does GHA support web inputs? I've never seen this sort of "web continuation" outside Jenkins.

[armanbilge]

Does GHA support web inputs?

It does! See https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow

[armanbilge]

Sonatype Publishing

1. Important: support both local and ci publishing.
2. Contentious: snapshot publishing. Flexibility seems achievable, see Publish snapshots by default and add an override for hash releases djspiewak/sbt-spiewak#86
3. Nice-to-have: easy opt-in to s01.

[armanbilge]

scalac settings

1. good defaults. Besides bm4 and kind projector, not too sure about this, you are just gonna have to battle it out ;)
2. easy to make said defaults go away
3. cross-friendly, i.e. scala 2.12/2.13/3.x specific-setttings and JVM/JS/Native-specific settings
4. This one also seems like low-hanging fruit, if someone wants to work on this.

[armanbilge]

sbt-typelevel

Maybe too ambitious, but I'd like to imagine each of the above as independent plugins, as much as possible.

The job of sbt-typelevel would be to bring them all together, and do smart things with them, e.g.: don't publish if mima fails. (Apologies, running a bit dry here for now :)

[rossabaker]

I watch 75 repositories, and few of them have a maintainer who knows or wants to know the fine points of SBT. My desiderata:

1. A meta-plugin that works out of the box so the novice Typelevel maintainer doesn't need to think to publish.
2. Shrinks the typical build.sbt (or extensions like sbt-http4s-org) to the point that updates to this plugin keep that build fresh.
3. A bus factor worthy of its status as critical infrastructure.

I think the top-level plugin in #12 is getting us closer.