Thread-Local Variables

[tleish]

In Falcon, I want to read a unique value from every request header and then later add it to outbound http requests. With an existing code base, I would have to touch a lot of files in order to read and pass this value to classes which execute outbound requests. This is primarily for logging.

Is there a safe and simple way to accomplish this without modifying lots of code and crossing values between fibers?

Note: With Rails, I've used the RequestStore gem to accomplish thread-local variables per request (which sets Thread.current on a new rack request and clears when the rack request finishes). What's not clear to me is, in Falcon would a similar approach work if a request came in along with an asynchronous outbound http requests. Is this approach thread safe with Falcon?

ADDITIONAL DETAILS: (optional)
I use Falcon as an asynchronous API server. Here's one example of the path a request might take.

Browser->>API Server: Post
API Server->>3rd Party: Post
3rd Party-->>API Server: Response
API Server->>Web Server: Post
Web Server-->>API Server: Response
API Server-->>Browser: Response

At each incoming/outgoing request, I want log a unique request ID so I when searching quest logs, I can trace the request from end-to-end.

[ioquatix]

What's not clear to me is, in Falcon would a similar approach work if a request came in along with an asynchronous outbound http requests. Is this approach thread safe with Falcon?

That gem will not work with Falcon correctly.

What we need is inheritable fiber locals.

This is something I am considering at the moment. cc @eregon

I completely agree that this should (1) be possible and (2) work in a logical way.

The problem right now is there is no native interface for how things get inherited between threads, e.g.

Thread.current[:inherited_x] = 1

parent = Thread.current
Thread.new do
  # Some kind of implicit way to do this:
  Thread.current[:inherited_x] = parent[:inherited_x]
end

The same applies to Fibers.

I believe we may need to introduce some standard way of doing this. A kind of inherited thread local or fiber local.

Threads and/or fibers inherit the scope of the thread/fiber they are created in. This would be limited to specific locals.

Right now, such functionality does not exist.

HOWEVER.

Because of the design of the fiber scheduler, I believe we can monkey patch this.

class Scheduler
  def fiber(...)
    locals = Fiber.current.inheritable_locals
    fiber = Fiber.new{Fiber.current.inheritable_locals = locals; ...}
  end
end

I am currently experimenting with such a design in Ruby 3. The UX is unclear to me (how to define inheritable locals), but I believe we can at least experiment with it.

One idea I've been exploring is having explicit attributes, i.e.

class Fiber
  attr :my_local
end

# Perhaps we could introduce the following:

class Fiber
  inheritable_attr :my_local
end

For your use case, you'd end up with something like

Fiber.current.my_local = 10

Fiber.new do
  # Fiber.current.my_local is 10 here, inherited at point of calling `new`.
end

By this design, we have a generic way for people to define inheritable attributes which are scoped to the task that creates them, and all children (i.e. web requests, and so on) that are performed by them.

[ioquatix]

By the way, just for further reference, this was discussed previously here: socketry/async#52 and socketry/rubydns#42 which are both similar issues in that they require the same underlying functionality.

[tleish]

Ah. I'm wondering if I can use the nested Async.logger to accomplish what I'm looking to do.

Thanks for the references.

[ioquatix]

So.. here is my feeling.

Making the logger a concern of Async was a bad design choice and I will back out of that.

My plan is to remove logger from Task.

This will solve a number of issues, including coupling Async to a specific logger implementation.

My goal is that the console logger, like any other logger, or your metadata, can operate on the same playing field. Instead, we would have something like:

class Fiber
  attr_accessor :async_logger
  inheritable :async_logger
end

[eregon]

State on the Fiber class (as in defining that an attribute is always inherited) feels wrong, because whether to inherit depends on each new Fiber created, some must inherit, some must not. Using the scheduler fiber method seems a good way.

[ioquatix]

Here is a working proof of concept.. a total hack too haha:

require 'fiber'

module Inheritable
	def initialize(**options)
		super(**options)
		
		self.inherit_attributes_from(Fiber.current)
	end
	
	def self.prepended(klass)
		klass.extend(Singleton)
		klass.instance_variable_set(:@inheritable_attributes, Hash.new)
	end
	
	module Singleton
		def inheritable(key, default: nil)
			@inheritable_attributes[:"@#{key}"] = default
		end
		
		def inheritable_attributes
			@inheritable_attributes
		end
	end
	
	def inherit_attributes_from(fiber)
		self.class.inheritable_attributes.each do |name, default|
			value = fiber.instance_variable_get(name) || default
			self.instance_variable_set(name, value)
		end
	end
end

Fiber.prepend(Inheritable)

class Fiber
	attr_accessor :x
	inheritable :x
end

Fiber.current.x = 10

f = Fiber.new do
	puts Fiber.current.x
end

f.transfer

@eregon what do you think?

[ioquatix]

@eregon in your original reply, yes you missed something, these are not thread locals but scoped to the current fiber and any fibres it creates.

Yes, I can make a gem which does this, except for CRuby which may internally create fibres directly rather than calling Fiber.new so that would be the limitation. It might be a good start as a PoC.

[ioquatix]

Regarding existing gems using fiber locals, and it not being enough. Well, this is clearly the responsibility of the user/developer. But I think it's a significant enough use case that we should support it.

[eregon]

except for CRuby which may internally create fibres directly rather than calling Fiber.new so that would be the limitation.

AFAIK CRuby never does that (except the root Fiber, which should never inherit anything).
It seems nicer to use Scheduler#fiber (or some helper method if the code uses that to create the Fiber) rather than monkey-patching Fiber.new.

[tleish]

Just to confirm, is each incoming Falcon request a new Fiber with inherited Fibers? And with the monkey patch setting Fiber.current.x = 10 as a class variable only sets it per Fiber request so it wouldn't leak into other incoming requests?

Am I correct in thinking that the Scheduler monkey patch approach requires Ruby 3?

[ioquatix]

Yeah the scheduler monkey patch would require Ruby 3 but this would work on Ruby 2 as well.

Just to confirm, is each incoming Falcon request a new Fiber with inherited Fibers?

Yes. So this should do what you want.

[ioquatix]

It seems nicer to use Scheduler#fiber (or some helper method if the code uses that to create the Fiber)

1. Fiber scheduler is not used for creating all fibres.
2. Helper method would require explicitly to opt in which isn't always possible.

[tleish]

@ioquatix - the Inheritable example worked great. For my version (Ruby 2.6) I had to modify the initialize.

module Inheritable
  def initialize
    super

    self.inherit_attributes_from(Fiber.current)
  end
 
  #...
end

Everything else from your suggested example appears to be working great. I even did some testing using siege stress tester and logging multiple request IDs to see if I could find any leakage of requests into other requests and after doing 10 seconds of 194 requests per second with a nested async fibers (which included random delays), it seems to work.

Thanks for the help.

If considering this as a permanent solution to the library, I personally think using Fiber.current as a developer friendly since many are already be familiar with Thread.current concept.

[ioquatix]

Thanks for trying it out and reporting back your experiences. It's great to hear that you had a positive experience with the PoC.

Thanks for mentioning the changes required to make it work on the Ruby 2.6.

The problem with Fiber.current and Thread.current is that there is a lot of confusion around which is actually thread local and fiber local. You may already know this, but Thread.current[:x] is actually fiber local.

The decision to use attributes on class Fiber stems from two reasons:

1. It's clear that it's "per fiber" (or per thread on class Thread).
2. It will give you a warning if the same attribute is defined twice. This isn't the case using the "locals" hash.

The way I see this working, is that libraries can use a "namespace scoped" attribute, i.e.

class Fiber
  attr :console_logger
  inheritable :console_logger
end

module Console
  def self.logger
    Fiber.current.console_logger
  end
end

This would be my preferred interface, all things considered. I welcome further discussion on this.

[ioquatix]

I have made a pull request here: socketry/fiber-local#1 - for Ruby 2.x we might need a slight exception to option handling. The other option is this could be a completely separate gem.

[nepthar]

I see this is already answered, but I was looking through here and I thought I'd comment about how Twitter solves this problem. Basically, they have this concept of Locals which is essentially a thread local key-value store that can be loaded and saved. Additionally, keys can be let-style bound to values.

In practice, these are used to solve the problem you propose. They sort of "follow" the chain of continuations always ensuring that, for instance, all code handling the same request is run with the same Locals, and therefore has access to whatever metadata you wish. You can see here, for instance, how locals are saved and restored upon execution.

Anyway, food for thought!

[ioquatix]

Thanks so much for this link. Yes, I believe that you completely understand the use case. That another person has independently come up with the same idea gives me confidence that this is a good abstraction. Thanks so much for giving me some better terminology to describe the problem "following the chain of continuations" clearly describes the problem we are trying to address.

cc @eregon

[nepthar]

Very happy to help. I worked on the team that maintained that code for a few years. I'll take a look at any PRs that come out of this.