set_var/remove_var are unsound in combination with C code that reads the environment

[bluss]

Like the documentation for our set_var (setenv) says, care must be taken with mutating environment variables in a multithreaded program. See this glibc bug #13271 that says getaddrinfo may call getenv.

It looks like we have an unsynchronized call to getaddrinfo and this may cause trouble with glibc/Linux.

Seeing glibc's attitude to setenv in multithreaded programs, set_var seems like a big hazard in general(?).

Discovered as an issue tangential to #27966

cc @alexcrichton

[bluss]

cc @luqmana

[alexcrichton]

If we did take this route I'd want to convert the environment lock to a rwlock to ensure that we could at least have parallel dns queries. Also cc #27705.

[bluss]

Yeah, either way the situation is tricky. We get tangled up in glibc internals.

[alexcrichton]

I found the wording on this page also particularly interesting:

       env    Functions marked with env as an MT-Safety issue access the
              environment with getenv(3) or similar, without any guards to
              ensure safety in the presence of concurrent modifications.

              We do not mark these functions as MT-Unsafe, however, because
              functions that modify the environment are all marked with
              const:env and regarded as unsafe.  Being unsafe, the latter
              are not to be called when multiple threads are running or
              asynchronous signals are enabled, and so the environment can
              be considered effectively constant in these contexts, which
              makes the former safe.

The getaddrinfo function is indeed marked with this tag

[fweimer]

std::env::set_var is just not safe to use in a multi-threaded program, as I commented on issue #24741:

Even without low-level races, std::env::set_var is not quite safe because you can change a variable (such as TZ), do something, and change it back to the original value, without impacting something else in the process which runs at the same time.

This problem would not go away if we provided thread-safe environment access at the libc layer. Therefore, I'm surprised the function isn't marked unsafe.

[steveklabnik]

Triage: as far as I know, this hasn't changed at all. The docs do describe this possible footgun.

[smmalis37]

This issue can also apply to any external crate that calls into something that calls get_env, for example time-rs/time#293 and chronotope/chrono#499. Maybe some way for these crates to observe the lock is called for?

[RalfJung]

Seems to me like this is a soundness issue, and should be marked I-unsound?

[spastorino]

Assigning P-medium as discussed as part of the Prioritization Working Group procedure and removing I-prioritize.

[jhpratt]

Just repeating what I've said on Zulip: To resolve the soundness bug in time & chrono, one option would be to provide a std::env::lock method, which holds a lock on the environment until dropped. It's my understanding this is basically what is happening internally on some functions already in stdlib. Exposing this would allow much safer usage of FFI.

[RalfJung]

The env lock has to be exposed somehow, I agree.

An alternative API would be a function that takes a closure that will be called with the lock held. This is less powerful (which might be a good thing: provide the least powerful API that is good enough).

[RalfJung]

However, talking of the env lock -- also see #64718 for how the current use of the env lock is wrong, so exposing it in that state might be a problem.

[jhpratt]

Sure, that would work as well. End result is the same, if done properly.

[DemiMarie]

Honestly, I don’t believe std::env::set_var can be made safe.

In a pure-Rust program, it is already safe, but in C, setenv is considered thread-unsafe. Therefore, C libraries assume that they can call getenv at any time. Furthermore, adding a call to getenv is probably not considered worthy of a major version bump in the C library, so what was previously a safe call into that library can become unsafe without warning!

Even without the thread safety problems, there are other reasons that std::env::set_var is unsafe. Libraries such as gtk will load plugins from a path specified in an environment variable, and loading a shared library is inherently unsafe. If the shared library is in the system shared library directory, it can be assumed safe, since the OS is considered trusted. But it isn’t safe to (say) load a library from the Downloads folder (which could contain anything).

What might be safe is a function that fails if multiple threads are running, and which only allows setting a limited number of env vars, such as the timezone.

[evanj]

Just to confirm this is not a theoretical issue, here is a Rust program that only uses the standard library and crashes in Linux with glibc with a Segmentation Fault. It calls ToSocketAddrs on one thread, which calls getaddrinfo, and calls set_env on another. https://github.com/evanj/cgogetenvcrash/blob/main/rustsetenvcrash/src/main.rs

[VorpalBlade]

What is the status of this? Specifically with regards to exposing the env lock to users of std, this was talked about in 2020 but nothing much seems to have happened? Was the idea dropped, if so where is the rationale for that documented?

[ChrisDenton]

glibc says calling set_env is unsound in the presence of threads. So in order to address this we need the application to essentially become single threaded while set_env is being called.

The environment lock falls short on this. We can't seriously mandate that everyone acquires a lock before making libc calls, not least because the lock is Rust only.

A more robust course of action would be to simply (and more accurately) mark set_env unsafe and teach how it can be safely used in a cross-platform manner. In this scenario the lock is basically redundant so it could even be considered for removal.

[tarcieri]

I believe this is the current plan:

#116888 (comment)

We discussed this in the libs-api meeting. What we would like to see happen is that this function gets turned into an unsafe function for the 2024 edition, but on older editions only emits a warning if not inside an unsafe block (possibly using deprecated_safe #94978).

[richfelker]

Plan sounds good.

The right thing for applications to do is not modify the environment. If they're modifying it to pass to child processes, the right way to do that is passing an explicit new environment, not modifying your own environment and relying on it getting copied. If they're using the environment to control the behavior of C library code they're calling, that's inherently unsafe and the library should be fixed to accept explicit caller-provided overrides of whatever it was getting from the environment.

[VorpalBlade]

Plan sounds good.

The right thing for applications to do is not modify the environment. If they're modifying it to pass to child processes, the right way to do that is passing an explicit new environment, not modifying your own environment and relying on it getting copied. If they're using the environment to control the behavior of C library code they're calling, that's inherently unsafe and the library should be fixed to accept explicit caller-provided overrides of whatever it was getting from the environment.

@richfelker This doesn't really work if you are implementing a shell though, unless you decide to emulate your internal environment completely (and pass that on to tasks you spawn). That is a niche use case sure, and I'm curious as to what nushell and fish (which was recently rewritten in Rust) do here.

[tbu-]

unless you decide to emulate your internal environment completely (and pass that on to tasks you spawn).

Note that "emulating the internal environment completely" essentially only means keeping a hashmap around and accessing that instead of std::env::var.

I'm curious as to what nushell and fish (which was recently rewritten in Rust) do here.

fish doesn't call std::env::set_var: https://github.com/search?q=repo%3Afish-shell%2Ffish-shell+set_var&type=code
nushell doesn't call std::env::set_var outside of test code: https://github.com/search?q=repo%3Anushell%2Fnushell+set_var&type=code.

[richfelker]

unless you decide to emulate your internal environment completely (and pass that on to tasks you spawn).

Of course this is the correct way to do a shell. Just because ancient unix history did things the sloppy way of modifying the shell process's environment directly rather than keeping its own data structures for key/value map doesn't mean a modern one written in a modern safe language should copy the same bad ideas. It's not even easier to store exports in your own environment, since you already need your own map for non-exported variables, and you can just use the same one (with export flag on each entry) for both.

[tmccombs]

If they're using the environment to control the behavior of C library code they're calling, that's inherently unsafe and the library should be fixed to accept explicit caller-provided overrides of whatever it was getting from the environment.

It is really unfortunate that libc and posix have functions that depend on environment variables, such as localtime_r and anything that uses the locale. I really wish the c and posix standard would make a more multithreading friendly API.

[richfelker]

There is never a reason to set the locale environment vars within a program. If you want to request a specific locale, you pass the name to the locale functions. The environment is only there as a default inherited at execution time.

For time zones, indeed the standard library functionality is rather poor. There have been proposals to modernize it with first class zone objects, but not a lot of progress on making it happen.

[kamulos]

Just for completeness sake: there is also https://github.com/sunfishcode/eyra#why which solves the issue on the level of the libc

[DemiMarie]

For time zones, indeed the standard library functionality is rather poor. There have been proposals to modernize it with first class zone objects, but not a lot of progress on making it happen.

Is this something that could be prototyped in a C library?

[tmccombs]

Sure. It isn't trivial though. On linux, you need to know how to read the timezone files, which is kind of complicated to do, I think that is what chrono does though. I'm not sure if there is a documented and supported way to do it on other major platforms. I think Mac uses the same tzinfo format, but I'm not sure if that is documented to remain stable. I don't know about windows, or mobile oses.