Drop support for FreeBSD 10 and 11 from std

[asomers]

Rust's standard library currently uses a FreeBSD 10 compatible ABI. But FreeBSD 10 went EoL on October 31, 2018. And now FreeBSD 11 will itself be EoL on September 30, 2021. Isn't it finally time to drop FreeBSD 10 support and update the standard library's ABI to FreeBSD 12, or at least 11?

[nagisa]

Ultimately users of the platform are best positioned to answer this question. There are probably quite few casual users of FreeBSD developing rustc, so somebody who is actively involved in the FreeBSD community would likely be best positioned to provide some guidance here.

[asomers]

Well, I'm a FreeBSD developer. And I can tell you that we're very proud of our backwards-compatibility, but it doesn't require maintaining new tools for old OS versions. Instead, if a person must use FreeBSD 10, the best way to build new software for it would be to use a snapshot or fork of an old version of the Ports tree. That would include an older version of rustc, one that was known to work with FreeBSD 10. In fact, the ports tree itself quickly drops support for old OS versions (it's in the process of dropping support for 11 right now).

[cemeyer]

I’m also a FreeBSD and Rust developer and would advocate dropping 10 support immediately, and 11 support in a few days, when it is officially EOL.

[delphij]

I'm also a FreeBSD developer and I would second @cemeyer . FreeBSD 10 support should be gone as it's EoL'ed almost 3 years ago, and FreeBSD 11 support would be ended really soon too.

At FreeBSD we are committed to provide backward compatibility for applications compiled for older version of FreeBSD releases by maintaining stable system call interfaces; for C runtime we are providing versioned symbols, so legacy binaries would call the old interfaces, which are usually wrappers of new interfaces. While we are proud that we have provided compatibility shims which worked well for the old binaries, we generally encourage application developers to adopt newer API to take full advantage of the performance improvements and flexibility on newer FreeBSD releases.

[laffer1]

Dropping FreeBSD 10 support is likely fine at this point. I think it's premature with 11, despite the upcoming EOL date.

[bsdimp]

Dropping FreeBSD 10 is fine. There have been no security updates to it in over 3 years, so nobody with any kind of security concerns are still running it. FreeBSD 11 is about to be in the same boat. Unless there's some really compelling application stuck on FreeBSD 11 that can't recompile and redeploy to FreeBSD 12, it would be far more beneficial to update to FreeBSD 12's ABI. Old ABIs are generally only supported on a best-effort basis. Given how far out of date FreeBSD 10 at this point, that best effort decayed to zero long ago.

I believe that community survey has shown that people have completely moved on from FreeBSD 10.

[bapt]

Following the freebsd EOL policy is perfectly fine. FreeBSD 10 support should be dropped now, FreeBSD 11 should be dropped by November 1rst.

[valpackett]

12 is about the only real backwards-incompatible break (64-bit inodes) in a long long time. EoL of everything before that is a great event \o/ Rust definitely should start the move into the ino64 world soon (by the end of the year). IIRC the libc crate already has flags to use newer ABI, so they should become the default.

[dch]

as asomers points out, the FreeBSD ports tree (from which all the packages for end users are built), drops support at the same time. Users who don't stay current with a supported FreeBSD release, will stay on the last rust version that they installed, without a path for upgrade.

[dumbbell]

I agree with the proposal too. It makes sense to drop support for FreeBSD 10 now and 11 once it reaches EOL.

[MikaelUrankar]

Drop support for FreeBSD 10 and 11 please.

[asomers]

@mirabilos could you please explain your objection?

[emaste]

There's a related discussion happening for Go now, although Go's avoidance of libc makes it interesting: golang/go#48164

I agree with dropping FreeBSD 10 support immediately and also agree with moving to 12.x the minimum supported version. However, the official EOL is just over a week from the time I'm writing this comment and I would suggest waiting a short time after that, to allow time for a heads-up on FreeBSD mailing lists or elsewhere.

There are a number of companies and others building products and projects on top of FreeBSD who for better or worse continue to use older, unsupported versions. While it is not reasonable (or possible) to accommodate this case long-term, I'd generally like to avoid forcing them into an urgent and unplanned need to upgrade.

[allanjude]

Specifically, getting to to at least the 12 ABI gets rust onto the good side of the changes to support 64-bit inodes and longer filesystem mountpoints.

[bdrewery]

TLDR, Yes, but just removing everything will cause problems on 12+. Builders need to be updated to 12 first. A lot of these symbols are from 7.0 and were used to support 12...
Edit: Thanks @asomers for #89083 which does that. Most of this is moot then but I'll leave it in case someone wants to understand the full picture here as I linked some of the problems I'm aware of, and the problem is not what it seems.

The issue here is far more complex than it seems and discussed more at rust-lang/libc#570.
This post is long because the premise here is (unintentionally) misleading to those who are not aware of the details. There are symbols bound to versions from FreeBSD 7.0 when symbol versioning was introduced, and from FreeBSD 8 for fstatat(2). There may be some from 10 or 11 somewhere too. If we only drop all of these then Rust will no longer work on FreeBSD 12+.

Isn't it finally time to drop FreeBSD 10 support and update the standard library's ABI to FreeBSD 12

• This only makes sense if the official Rust, and crate, builders are 12, and rustup and git builds on FreeBSD continue to work without external hacks.

It appears the builders are still targeting 11.

• rust/src/ci/docker/host-x86_64/dist-x86_64-freebsd/Dockerfile

  Lines 29 to 32 in 0132f82

  	ENV \
  	AR_x86_64_unknown_freebsd=x86_64-unknown-freebsd11-ar \
  	CC_x86_64_unknown_freebsd=x86_64-unknown-freebsd11-clang \
  	CXX_x86_64_unknown_freebsd=x86_64-unknown-freebsd11-clang++

• rust/src/ci/docker/scripts/freebsd-toolchain.sh

  Line 8 in 0132f82

  freebsd_version=11.4

But FreeBSD 10 went EoL on October 31, 2018. And now FreeBSD 11 will itself be EoL on September 30, 2021.

"10 (or 11) being EOL" is not very relevant here as noted about the use of 7.0 and 8.0 symbol versions. The reasoning for this (rust-lang/libc#937, rust-lang/libc#936 rust-lang/libc#721) was never about 10 or 11. From what I recall it was because the Rust builder (on FreeBSD 10) was generating unversioned symbol usage that broke on 12 with ABI changes there. Rust only knows about the target OS and arch, not release. So the "10/11" compat was for 12. FreeBSD ports had hacks to generate a "working" rust binary but as soon as a crate was used that touched any of the structs from that PR it would blow up. [1] [2] [3] [4]. Using rustup or building rust from git, or using the bootstrap, also would not work as they lacked the patches we were carrying.

Rust's standard library currently uses a FreeBSD 10 compatible ABI

I agree the symbols targetted currently are for "10" (some are 7+) (from libc basically bundling FreeBSD 11 headers), but as far as I know the majority of symbols are used unversioned which is why the problem was present in the first place. That is, I believe that at that time if we purposely bound every symbol to the version of the symbol for that prototype that was copied into the libc header then on 12 it would not have been crashing (unless using an external C library). (Maybe that's a simple compromise regarding #570 but I haven't thought deeply on it.)

Rust should not have any blanket policy of dropping compat/versioned symbols for "EOL" releases - they may be there for a reason that has nothing to do with the specific version. FreeBSD has symbols from 7 and earlier that are still used today and may be bound to at some point. I just want to be sure this is clear as I suspect some of the votes here from fellow FreeBSD devs may be policy votes, rather than informed technical votes. Changing what version is targeted is fine regarding EOL, but symbols are a more nuanced discussion.

Having said all of that, allowing interoperability between rust and external libraries using FreeBSD's current ABI is important and the reasoning for this compat is not there anymore assuming the official Rust builders are upgraded to 12. But someone needs to consider every piece involved in the ecosystem here before rushing a change in because of a date change. 11 being EOL has little to no effect on anything for Rust beyond interoperability.

We really need rust-lang/libc#570 dealt with to have a target containing the FreeBSD release (with proper header sysroots for cross builds or continued FFI use) or libc flags or special crates or whatever. Either way the real problem is the need for investment to solve and maintain the solution, and how that would need to be mimiced for other platforms to be consistent.

[asomers]

@bdrewery, there is already a PR to update the builders: #89083 . That should take care of rustup. And regarding the symbol versions, let me summarize for the benefit of everybody new to this saga:

As we discussed elsewhere, FreeBSD uses ELF symbol versioning, allowing it to occasionally modify libc functions like readdir and stat in a way that is backwards compatible at both the source and binary level. But Rust uses FFI. That's nice for cross-compiling, but it forces Rust to lock its libc to one particular FreeBSD version. For whatever reason, Rust chose 11. The problem with 12+ that bdrewery mentioned is that prior to FreeBSD 12, Rust was linking to symbols such as stat with the default link name "stat". But FreeBSD 12 changed the signature of stat and renamed the old function "stat@FBSD_1.0". That meant that Rust's libc was briefly trying to use FreeBSD 11's stat signature with FreeBSD 12's stat function. Fixing the problem was a matter of specifying the correct link name, as shown in the PRs linked above.

So dropping FreeBSD 10 and 11 support will require removing the FreeBSD 11 link names as well as updating the structure definitions. Most of the new structure definitions are already in libc. The rest can be added quickly.

[valpackett]

FreeBSD 12 changed the signature of stat and renamed the old function "stat@FBSD_1.0"

nitpick: to be clear, 12 introduced the @FBSD_1.5 versions, but the old ones were always (okay, since 7.0) @FBSD_1.0. The "renaming" effect is virtual, just a consequence of unversioned symbols pointing to the latest version.

Rust should not have any blanket policy of dropping compat/versioned symbols for "EOL" releases

I don't think anyone has ever suggested going back to unversioned symbols?

Perhaps this issue should've been called "Update FreeBSD support to use 64-bit inodes, available since FreeBSD 12".

[emaste]

Indeed - the old version in 11.x is stat@FBSD_1.0, and anything linked for FreeBSD 11.x should reference that. You can examine the symbol table of an 11.x binary to see this, e.g.

ref11-amd64% readelf -s /usr/bin//stat | grep -w stat
    13: 0000000000000000    17 FUNC    GLOBAL DEFAULT  UND stat@FBSD_1.0 (2)

the majority of symbols are used unversioned

To be clear, is this now fixed and all references are to the versioned symbols?

[asomers]

To be clear, is this now fixed and all references are to the versioned symbols?

All references in Rust's libc library to symbols that have multiple versions in FreeBSD 11 or later are versioned. References to symbols with a single version, or symbols that haven't had a new version since 11, are unversioned.

[valpackett]

One bonus reason to hurry the transition: the default configuration for riscv64 currently does not enable COMPAT_* flags, so no pre-ino64 ABI there. The riscv64 port is running into this.

[bdrewery]

What's the holdup here? Does someone just need to submit a PR? I pushed in a lot of this compat to get things working but am not a fan of it.

[nikic]

FYI #97944 landed, which switches CI to build rustc for FreeBSD 12 rather than 11. (Not entirely sure how that relates to this issue, but it seemed relevant to mention.)

[asomers]

Now that rustc is using FreeBSD 12 in CI, the next step is top drop FreeBSD 10 support from libc. We'll have to rebuild rustc to test that change, but no other crates. After that's done we can separately drop FreeBSD 11 support. That might require a little more testing.

[mjguzik]

Hello, another FreeBSD developer here.

Anything I can help with to get this sorted out? @asomers?

[asomers]

@mjguzik If I understand correctly, we need approval from one of either @davidtwco @nikomatsakis or @pnkfelix . Then we can:

• Submit a PR to rust-lang/libc that drops support for FreeBSD 10. Testing that will require rebuilding the rust toolchain with the patched libc, which I don't know how to do.
• Submit a subsequent PR to rust-lang/libc that drops support for FreeBSD 11 and changes the ABI to FreeBSD 12. This may break the build for other crates, for example nix-rust/nix. I don't know the best way to deal with that.

[mjguzik]

ok, then bonus question: I want to add copy_file_range, which is only available since 13, so 12 would have to be excluded in some manner. From a grep I found one can use #[cfg(freebsd12)] . Should the older releases not be officially dropped, can this roll with a not freebsd10/freebsd11/freebsd12 check instead (don't know how to write in in rust)?

[asomers]

ok, then bonus question: I want to add copy_file_range, which is only available since 13, so 12 would have to be excluded in some manner. From a grep I found one can use #[cfg(freebsd12)] . Should the older releases not be officially dropped, can this roll with a not freebsd10/freebsd11/freebsd12 check instead (don't know how to write in in rust)?

Ahh, easy question. The answer is, copy_file_range is already available. See rust-lang/libc#3023 .

[asomers]

Ahh, easy question. The answer is, copy_file_range is already available. See rust-lang/libc#3023 .

Err, unless you meant to employ copy_file_range in std::fs::copy. I don't think we can do that in the standard library until Rust drops FreeBSD 12 support. Maybe, if you want to somehow use dlsym to look for the function at runtime.

[mjguzik]

That is what I meant. I thought the cfg thing is an ifdef-equivalent, meaning it would correctly compile the usage when built on 13 and later and omit it otherwise, which maintains all the compat you normally ask for from resulting binaries.

As a side note I have to add the FreeBSD variant supports cross-fs copies, while Linux does not. At the same time there are no fallbacks to splice et al. So this would be a few-liner made separately.

[asomers]

That is what I meant. I thought the cfg thing is an ifdef-equivalent, meaning it would correctly compile the usage when built on 13 and later and omit it otherwise, which maintains all the compat you normally ask for.

It's not quite an ifdef equivalent. Outside of CI, there's nothing that will ever set the freebsd13 flag in a libc build. Due to Rust's heavy emphasis on cross-compiling, nobody's yet come up with a good way to do that in such a widely used crate. See discussion at rust-lang/libc#570 .

[mjguzik]

welp, sigh

Linux support tries to work around stuff by handling ENOSYS at runtime, but we can't do that on FreeBSD due to SIGSYS.

While I'm not fond of this, how about pessimizing binary startup with fetching kern.osreldate. Then io::copy (and possibly many others) can branch on it. Does not solve binary compat, but it is always something. I guess for an initial version I can pessimize it just for code using the copy routine.

EDIT: good news, there is osreldate in the auxv vector, so no need to make any syscalls.

[asomers]

Actually, the situation is both better and worse than that. Linux can optimistically attempt copy_file_range and check for ENOSYS because it makes a direct syscall. With FreeBSD, we need to link to libc.so, so we'll get a link-time failure if you invoke the program on a system without copy_file_range. OTOH, if the link succeeds, then there's virtually no chance that the kernel support won't be present, so we don't need to check for ENOSYS. That's why I think that a dlsym-based approach would work. But since it's unrelated to dropping 10 and 11 support, maybe you should open a separate issue?

[bsdimp]

how about pessimizing binary startup with fetching kern.osreldate.

getosreleasedate() will grab this information from the ELF AUX vector w/o a trip into the kernel for any kernel revision that's relevant for this discussion.

[mjguzik]

how about pessimizing binary startup with fetching kern.osreldate.

getosreleasedate() will grab this information from the ELF AUX vector w/o a trip into the kernel for any kernel revision that's relevant for this discussion.

that's what i wrote:

there is osreldate in the auxv vector, so no need to make any syscalls.

anyhow i'll be posting a new PR few days from now

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

This will be merged soon.

[emaste]

Should this issue be closed now?