FlatBuffers 2.0 tracking issue

[aardappel]

This issue is to keep track of the FlatBuffers 2.0 release, which is intended to be "soon", but also "when its done".

2.0 you ask? Wasn't the last one 1.12 ? Yes, because:

• We're going to attempt to adhere as best we can to "Semantic Versioning" from now on. Past releases were simply 1.0 thru 1.12 with no regard for how many breaking changes were in a release. Generally, every release so far has been breaking for at least 1 language, but we have not expressed that. If we continue our habit of occasionally breaking APIs, the release after this one may thus have to be 3.0 instead of 2.1, but we'll see. Our version architect is @krojew
• We actually do seem to have more breaking changes than usual, with especially Rust getting a big overhaul, but also Swift, Python and others. So 2.0 seems appropriate.

What are we waiting for? We are waiting for some of these larger breaking changes to "settle down". This thread will link those issues, once all are merged we'll get the release going.

Also worth mentioning: we are attempting to work towards a structure where FlatBuffers development is more "distributed", with clear individual maintainers for each language, that may at some point may make language package releases and such independently of the main FlatBuffers releases. But this will take time.

[aardappel]

@rw @mikkelfj @krojew @dbaileychess @paulovap @dnfield @CasperN @mustiikhalil @vglavnyy please link this issue from your issues.

[aardappel]

Also, even though "2.0" sounds like a big deal, lets not line up any new features that haven't started yet for this release. We just want to finish/polish what we have. Any new adventures are best started afterwards.

Once we have all major features in, we'll probably have 1 week of "only safe fixes get merged" before release. I know we can do this with branches instead, but this should be simple enough.

And yes, to be clear, like the issues already linked above, link issues TO here, not FROM here, that way we can easily see which ones are still open.

[mikkelfj]

Not to confuse with FlatBuffers format v2 I presume.

As to things desirable for 2.0, Go seems to be missing some features like fixed length arrays and optional scalars. More prominent documentation (outside of Github issues) of such missing features would be highly useful.

[aardappel]

@mikkelfj well, that's exactly what I am saying above: this being "2.0" is NOT the time to get creative and think of what features really "should be" in 2.0. It's just another release. It would be great to work on these features for Go, but if they haven't started yet, I suggest they land after 2.0. It will be a while before we have full feature parity in languages.

[mikkelfj]

Yes, but a better overview of the feature set would be a start.

[aardappel]

Ok, I just pinged the issues/PRs linked to this, so we can start working towards getting this to closure. Once there are no more blocking issues linked to this we'll get a release going.

[aardappel]

#5840 and #6351 are not blockers for release (anymore).

[vglavnyy]

It needs to close these issues (Werror) before release: #6337, #5930. I will prepare a solution.

[mustiikhalil]

As we discussed in the grpc enhancement and rework directory for grpc, the only blocking part is step one. and the rest are none blockers. Just wanted to mention this here to make sure that we don't wait longer than what we need to release it

[aardappel]

Looking at the above list, seems most things are merged or not blockers anymore.

Seems like the only remaining blocker is C++ gRPC fix @mustiikhalil ?

@vglavnyy @krojew Seems you guys are ready for a 2.0?

@dbaileychess @CasperN @paulovap any concerns from your side?

After we're all ok, we'll do a week or so during which only small fixes / doc improvements may land, and then I will start the process as in https://github.com/google/flatbuffers/wiki/FlatBuffers-release-process

[paulovap]

No concerns from my side.

[mustiikhalil]

Seems like the only remaining blocker is C++ gRPC fix @mustiikhalil ?

From my side there is the C++ & Java grpc. The Java grpc can land in 2.1 since I honestly couldn't get it to work, and it only fetches the 1.2 grpc lib so it won't be an issue for now. I've already asked for help here #6443

[krojew]

Before that, let's not forget to unify language versions.

[mustiikhalil]

Just updated the sample files for swift, plus added an attribute for the structs. Nothing blocking, major, or breaking.

[krojew]

@CasperN what do you think about removing all deprecated items from Rust crate for 2.0.0?

[CasperN]

@CasperN what do you think about removing all deprecated items from Rust crate for 2.0.0?

I'm indifferent because I see no rush to remove them. Also, I think the Rust crate should continue being disconnected with the main semver since we're still in active development and I'd prefer the higher release cadence.

[aardappel]

@CasperN but isn't the point, besides semver, that users can see that Rust 2.0 is related to the other languages at 2.0 in terms of use of the same version and features of flatc ? Having Rust singled out at 0.9 or whatever is very confusing, and it be good to stop doing that.

You can still release 2.0.1 or whatever to Cargo if you need to.

Again, it's semver.. it is not supposed to speak to the maturity of something, but to the amount of breakage that has happened in the past. Apparently. I am new to this to ;)

Unless I hear otherwise, I will be versioning things and releasing on Monday, so if someone has other ideas, better speak up now.

[CasperN]

What if I want to release breaking rust changes in the interim between flatc releases?
If I bump the major sem-ver, then the next flatc sember may be > 3.0.0, which seems unideal.
Should I just not publish breaking rust changes until the next flatc release? This would force people who build flatc at HEAD to depend on the rust library from git too... which I guess could be okay

[aardappel]

Yes, that is complicated, but there's no great way to do that.

I do think that from now on, breaking stuff should expect to live on unversioned master for a while. That gives us the opportunity to see the consequences with early adopters, and potentially change to be less breaking before a release.

We also should do releases a bit more often than once a year :)

The alternative of giving all languages their own unrelated versioning is waaay to chaotic I think, as I described before. We want to make it easier for users to know that all their parts are from the same version, not harder.

[aardappel]

Ok, I am going to get the release going.. no more merging! :)

[aardappel]

Getting there: https://github.com/google/flatbuffers/releases/tag/v2.0.0

Next up is pushing to the various language repos. If you are the maintainer of one, please do so now.

[aardappel]

https://pypi.org/project/flatbuffers/2.0/

[aardappel]

https://www.npmjs.com/package/flatbuffers

[aardappel]

If people already have things they know need fixing, the next release tracking issue: #6636 :)

[mustiikhalil]

Swift is out too!
SPM
CocoaPods

[aardappel]

https://mvnrepository.com/artifact/com.google.flatbuffers/flatbuffers-java

[aardappel]

flatbuffers-java-grpc should be up soon too.

npm package was update to 2.0.1 due to missing files.

@rw @CasperN wanna update https://crates.io/crates/flatbuffers ?

[aardappel]

Dart publish script appears broken, maybe something for whoever knows Dart, e.g. @dnfield

[krojew]

@CasperN any chance for Rust 2.0? Also, we should really provide a README at https://crates.io/crates/flatbuffers.

[CasperN]

sure, I'll add a README and publish on Monday

[krojew]

@CasperN any chance for this 2.0.0 crate? :)

[CasperN]

Done, thanks for the reminder

[bjornharrtell]

Wouldn't it be nice with an official NuGet package too? @dbaileychess do you agree? Anything I can do to assist?

[dbaileychess]

Yes, that sounds like a good plan. I haven't personally push a Nuget package, and there seems to be some difficulty in doing so according to #4162. But we should strive to make it better.

[aardappel]

Java users: please use 2.0.1, that includes the fix for: #6658

[vaind]

Is there a date this should go live? Dart null safety is a breaking change (#6282) but not ready yet.

[aardappel]

@vaind the Dart publish script appears broken, waiting for @dnfield or anyone else to fix it.

[aardappel]

Note: JS users should now be using the fixed 2.0.3 package I just published.

[zamazan4ik]

Just in case, if you are interested: https://github.com/zakarumych/rust_serialization_benchmark/tree/alkahest

Maybe it can be helpful for comparing flatbuffers serialization speed with other libraries and will be reasoning to speed-up somehow the library.