Feature Idea: Pooled Instances

[alistairjevans]

An idea came to me about a potentially useful new feature we could add in to Autofac; pooled instances.

Basically, when you resolve a registration configured as pooled, you get an instance from a pool of objects, not a new instance.

In more detail:

• When you configure a registration, you specify PooledInstances() as the lifetime option instead of SingleInstance() or InstancePerLifetimeScope(). You could probably set an optional max capacity or something.
• Behind the scenes, we register a ServicePool that contains pooled instances of the service.
• When a user resolves the service, we give them (Rent) an instance from the pool, which is scoped to the LifetimeScope; when the scope ends, we hand the instance back (Return) to the pool (but don't dispose of the instance).
• Services could implement a provided interface, INotifyPoolEvents, which has a Rent and Return method, called when we retrieve from the pool or put back, to do any necessary resetting. We could conceivably also store additional event handlers somehow to serve a similar purpose without modifying the type.
• When the container (or whatever scope the pool was registered against) is disposed, all the items in the pool are disposed.

Any thoughts? Could give people a nice easy route to use a set of pooled 'something' in their DI?

[alsami]

Interesting idea. Somewhat similar to thread/connection pooling I guess.

What would be the advantage over lets say a singleton, since they somehow would behave like a singleton, wouldn't they? Only thing I can think of now is the fact, that there would be less time spend resolving dependencies, since the second request would just hand back the current existing instance of the pool.

I think it could potentially also be misused very easily. Especially for dependencies that have some sort of resources that should be cleaned up.

Also, how would that integrate with let's say Microsoft DI? They only provide Singleton|Transient|Scoped which for us is Singleton|InstancePerDependency|InstancePerLifetimeScope.

[alistairjevans]

Some resources, like connections, lend themselves quite well to "I have capacity to handle this many concurrent operations, and it's expensive to 'new up' one each time".

You could quite simply manually define an ObjectPool singleton that you resolve, and retrieve an instance of your service, but I'm imagining scenarios where there's a significant existing body of code that always treated the service as either a per-scope dependency, or singleton, and can now pivot to being a pooled resource.

Clean-up is an interesting one, because you wouldn't have Dispose to do that, so another mechanism (Rent/Return methods) would have to fill in the gap.

As for integration with MS DI, this would be one of those scenarios where you have to directly manipulate the container (like if you wanted to do matching lifetime scopes).

When the user resolves using IServiceProvider, it would resolve an instance from the pool.

[tillig]

I dig it. I could see how database connections or whatever would be good here, where previously you may have had to write your own singleton factory to hand out connections. Sort of like what's going on in this document but not bothering the consumer with the actual ObjectPool<T>.

I can see it could be a potential challenge for new folks to get right. We'd have to think about how it would behave with respect to...

• Implicit relationships: Func<T> or Lazy<T> wouldn't pull one from the pool until evaluated, but would Func<T> give you a new one each time? Owned<T> could be interesting - is that the way you "explicitly return a thing to the pool?"
• Max pool limit: If you try to resolve one of these things from the pool but none are available, what's the error?
• Default behaviors: If it's pooled, there's some sort of implicit pool size or it's no different than instance-per-dependency. What's a good "default" pool size?
• Child lifetime scopes: What happens if you have an ObjectPool<MyThing> registered in the root container and then add another in a child lifetime scope registration?
• Captive dependencies: If I resolve a singleton that takes a dependency on a pooled thing, that pooled thing will never go back to the pool.
• Reset policy handling: ObjectPool<T> has a notion of PooledObjectPolicy<T> that explains how a pooled instance can be cleared/reset when returned to the pool so it's ready for the next consumer. That is, if the pooled objects have state (like maintaining a DB connection) some things get "cleared out" and some things stay alive.

I mean, that's just details ("just?" 😆 ) but the idea of handling object pooling as a new lifetime management construct is really interesting. I don't know of any other container that does it, so it'd be a cool differentiating feature.

[alistairjevans]

I think there are three separate sets of issues here.

The first is about how the lifetime of the pool and the objects in the pool maps to existing Autofac notions and implicit relationships.

I think the easiest way to define this lifetime behaviour is to define a few simple rules and let everything else flow from there.

1. Each Pool is a SingleInstance registration.
2. Each Instance of the Service in the Pool (and child deps) is initially resolved at the same scope as the pool (so effectively SingleInstance too), and marked as ExternallyOwned so the Pool controls disposal.
3. When the Pool is disposed, so are all the Instances.
4. When the Service is resolved (at any nested scope, inside Owned/Func/Lazy/whatever) we resolve the Pool, Rent from it, and return that instance.
5. The rented instance is shared at per-lifetimescope (matching scopes supported too I expect).
6. When the scope ends, the Instance is Returned to the Pool.

Okay, so 6 rules. But I suspect we can answer a lot of your questions with those.

The second issue, for Renting/Returning, I propose defining an IPoolPolicy, that can hold custom behaviour for Renting/Returning, including (possibly) allowing services to be resolved from the scope where it is being rented, at the point of rental.

We could define three built-in pool policies:

• one that does nothing (default)
• one that invokes rent/return methods directly on the service interface.
• one that allows delegates to be defined that provide rent/return behaviour.

Lastly, pool capacity, is an interesting one. There's something to be said for just letting the pool grow as needed (if you request more entities, you must need a bigger pool).

I think it would also be good to be able to dynamically shrink the pool...so perhaps defining a 'target max' rather than an absolute max might be better.

[tillig]

I wonder if we could use the standard ObjectPool<T> and associated classes outright for our backing store, so we don't have to own or define that. It could save some troubleshooting and give us ideas about what things need to be handled. Basically just building on this.

I think the rules defined there do answer most of the questions.

I think different pool policies will have to handle pool capacity issues. Like, I might define a maximum pool size on some database contexts to ensure I can effectively limit a given application's connections to that database, like a throttling mechanism. Shrinking it, in combination with a potential hard maximum, would cover most things, I think.

[alistairjevans]

I did think about using the existing ObjectPool, and it would be nice to use the runtime approach, the only questions are whether:

• we are happy to bring Microsoft.Extensions.ObjectPool in as a dependency
• the semantics of the pool match our needs (which I think they do...there is commentary about storing a pool in DI).

[tillig]

Could pooling be written as an extension, so you could add a reference to a package that adds support for such a feature? I agree, possibly adding an extension package as a default dependency if you're not using it is less than ideal; at the same time, I find I don't want to own redundant code if there's something we can use. Perhaps such a thing is too hard for an extension; on the other hand, maybe that tests the extensibility mechanism(s) of v6, to see if it can be done?

[alsami]

Some resources, like connections, lend themselves quite well to "I have capacity to handle this many concurrent operations, and it's expensive to 'new up' one each time".

I dig it. I could see how database connections or whatever would be good here, where previously you may have had to write your own singleton factory to hand out connections

What kind of connections are we speaking of? Thinking about database-connections, which most likely are based on ADO.NET, there is already connection pooling. For instance calling new SqlConnection("some-connection-string") does not actually acquire the connection. In the moment you call connection.Open() the first time, the connection and the pool is being created.

[tillig]

Does it matter? Maybe it's some connection type you've created in your code; maybe it's a connection to a Vault instance for secrets; maybe it's not a connection to a database at all and, instead, it's a set of resources used to perform expensive calculations, like access to a quantum computing resource. I say "database connection" as a placeholder for "any sort of resource for which it's overly expensive to create that resource (or connection to that resource) and/or the resource is finite such that there is a maximum amount of that to spread around the app." I don't mean to try to "sell" the concept solely based on ADO.NET connections, which, as you say, already have some level of pooling built in. I think if you had a need for a pooling mechanism where it wasn't provided by the thing you want pooled - this could be an interesting idea.

[alsami]

like access to a quantum computing resource

You do that on a regular basis? :)

I don't mean to try to "sell" the concept solely based on ADO.NET connections, which, as you say, already have some level of pooling built in. I think if you had a need for a pooling mechanism where it wasn't provided by the thing you want pooled - this could be an interesting idea.

Yes, true. Just thinking that there might be many resources that have a mechanism like this built-in and people might end up using that new scope not knowing it's being provided by the specific resource as well.

[tillig]

people might end up using that new scope not knowing it's being provided by the specific resource as well

This is one of the things I was thinking about when I was mentioning, "I can see it could be a potential challenge for new folks to get right." Knowing when to use it is going to be just as important as knowing how to use it.

But we have a lot of features like that, where it's important to know both when and how to apply it. I don't know that should stop us, just that we'll have to be mindful about explaining it in the docs and examples.

[alistairjevans]

maybe that tests the extensibility mechanism(s) of v6, to see if it can be done?

Like you say, this may be a good test of the new extensibility options.

At the end of the day, what I'm proposing here is basically an Autofac.Integration.ObjectPool package, to make using the ObjectPool more DI.

[alistairjevans]

I'm going to have a go at this locally, see what issues I run into.

[alistairjevans]

Ok, this looks like it will work fairly easily, I've got something working locally, but need more tests and some overloads.

The thing that v6 gives us is Service Pipelines, because I can now store a different object in the Shared Instance lookup than I actually return to the consumer. Without that this would be extremely difficult.

One key point on using the ObjectPool, it doesn't set an upper limit on how many instances it will allow concurrently, it only specifies a limit on how many to maintain. From the docs:

NOTE: The ObjectPool doesn't place a limit on the number of objects that it will allocate, it places a limit on the number of objects it will retain.

So the concept of, I've got X connection slots available, and can never go over that, isn't really part of the ObjectPool behaviour. It just means that it will only pool X connections.

Right now I've got this:

builder.RegisterType<PooledComponent>().As<IPooledService>()
                                       .PooledInstancePerLifetimeScope()
                                       .OnActivated(args => activateCounter++)

// ...other stuff

using (var scope1 = container.BeginLifetimeScope())
{
    scope1.Resolve<IPooledService>();
}

using (var scope2 = container.BeginLifetimeScope())
{
    scope2.Resolve<IPooledService>();
}

// Only 1 instance should have been activated.
Assert.Equal(1, activateCounter);

@tillig, did you want to create a new Autofac.Pooling repo so I can target a PR? I don't think this is an integration for a particular framework per-se, it could apply to anything.

[tillig]

Ooooookay.

Autofac.Pooling repo is live. There is a corresponding team for triage.

I didn't set up anything on AppVeyor because... there's nothing to build yet. 😄 I didn't create a develop branch, set up issue templates, or anything else. But the repo exists and we can start jamming on it.

[tillig]

I'm going to close this issue since the repo is up and running and there's code going in. We can iterate over ideas there as needed.