Improve circular dependency handling (general discussion / design)

[tillig]

We have seen issues and PRs come in about changes desired for circular dependency handling. We had every intention of making some improvements in time for 4.0 but, unfortunately, .NET Core "ate our lunch," as it were.

Much has changed since v4.0 and the .NET Core support was released so we're closing the following PRs so they can be later reimplemented (if desired) and/or used for ideas for improvement:

• Bug in CircularDependency tracking #579: This PR moved the check for circular dependencies to after where the instance lookup occurs. However, there was no unit test associated that could show the issue that this was intended to fix. It appears innocuous, but without understanding the specific case this fixes we declined to take it.
• Better circular dependency handling #575: This PR improved circular dependency handling to always work on properties. It also changed the default behavior of PropertyWiringOptions and broke the IComponentRegistration interface.

There are comments in #575 indicating ideas on better ways to handle improvement of the circular dependency handling. It is potentially non-trivial and breaking.

We invite folks interested in improving or changing the circular dependency handling to comment on this issue and let us know challenges they've seen or improvements they'd like to see. Note a failing unit test or two would help so we can understand in a more concrete fashion the challenges being seen.

[JackGrinningCat]

Hi, I have an circular dependency (A<->B) and expected that A.OnActivated() would give me fully initialized component back as instance pointer with B as property set.
I reread the documentation and i know that 'fully constructed' is not 'fully resolved dependency' here.
I will come up with another solution but as far as I'm concerned i wish for such a feature. But maybe I'm not done with some principle of Dos and Don'ts of DI.

Edit:
Well I solved my problem with an construction dependency that both (A and B) shares (C). As improvement to 'OnActivated' I use IEnumerable. Then any extending methods can be just attached. And it seems to work like a charm.
Well here a thanks for the good work.

class A
{
A( IEnumerable cs)
prop B
}

class B
{
B( IEnumerable cs)
prop A
}

class C
{
MethodForA
MethodForB
}

[bstewart00]

The biggest pain point for me with the current circular detection design is the hard-coded stack limit of 50. I work on a large application with a very deep dependency graph that frequently hits this limit despite no actual circular dependencies.

We've hacked around it so far by "resetting" the stack limit by injecting a Lazy registration in the middle of the stack somewhere. I would like the limit to at least be configurable in the container builder instead of it being arbitrary.

[tillig]

Based on some recent discussion in #924 and issue #927 I was thinking about how we might change the circular dependency detection to remove the need for any sort of hard-coded stack limit.

For example, I'm curious if each resolve operation might carry with it a context that includes the set of services that have already been resolved during that operation. If you see the same service twice in the same operation, it's a circular dependency. Maybe even if you see the same component (concrete class, etc) rather than just the same service then it's circular. The "allow circular dependencies" behavior would then change to basically be "if you see this twice, just don't throw."

I think the new decorator mechanism @alexmg is working on also has a similar tracking thing that was added. Perhaps that whole resolve context thing could be used in a more widespread manner.

Something like that would be a big improvement as far as circular dependency detection but I can see these possible drawbacks:

• Performance overhead: tracking and checking the list of services in the chain would be pretty fast but not free.
• Possibly breaking for edge cases: people can get into some crazy areas where they "work around" the circular dependency detection or inadvertently circumvent it by accident; this would likely catch things that weren't previously caught and may cause a few existing things to break because of it.

[tillig]

IComponentRegistration does carry an Id property with it. I wonder if just tracking the list of GUIDs over the course of the resolve operation would be enough. That would at least keep the memory consumption down - no flowing contexts and things, just a list of GUIDs.

[tillig]

Aggregating some things from other issues to consider as we look at this...

• Circular component dependency detected with manual IEnumerable<T> registration #648: If someone specifically registers IEnumerable<T> that doesn't play nicely with current support for collections and can cause a circular dependency.
• Instance Creation Issue with Property injection and lazy instantiation #718: Use of Lazy<T> and Func<T> defer resolution of values and can break or challenge circular dependency detection.
• InvalidOperationException from Owned<T> lifetime scope circular dependency #927: Use of Owned<T> creates a nested lifetime scope that can break or challenge circular dependency detection.

[bluetentacle]

Is there an easy way of determining how deep a dependency graph is? I could use reflection on the final dependency graph but it's not always reliable. It would be nice to have a way of how many resolutions are performed when I call Resolve. Thanks.

[tillig]

@bluetentacle This may help.

[VonOgre]

I have no idea what the performance implications would be, but could RuntimeHelpers.EnsureSufficientExecutionStack() be leveraged to remove the hardcoded static? I'm assuming it's possible if someone was deep in recursion before calling Resolve(), that the limit of 50 could even be larger than the remaining stack capacity before a StackOverflowException is thrown?

[tillig]

Hmmm, I didn't even know that was a thing, but it appears interesting.

It looks like it just determines if a single function call can be made, where we do have more of a recursion where it's like A -> B -> C -> A in the resolution chain, but there may be some sort of lever there or thread we can pull to be more dynamic about it. Nice find!

[alistairjevans]

It's an interesting method, but I suspect that by the time you get down to the point where that method throws, Autofac should maybe have already stopped?

Is it cheaper than incrementing and checking a counter? (Knowing the C# compiler team, it probably does some native stuff that means it actually is...)

At least it gives us a hard limit though.

[VonOgre]

I hadn't yet looked at the circular dependency detector in any sort of detail, but was looking into how one might be able to measure remaining call stack space and stumbled across that and thought that it (or possibly the Constrained Execution Regions) could be an interesting avenue to pursue 🤷‍♂️

I might poke around a bit more over the weekend, in case there are some hidden goodies to learn about :)

[alistairjevans]

The stack size limit is potentially less problematic than the fact we have to search the stack of in-progress resolves each time we resolve to check for the same registration.

There may be more efficient ways to check that.

[VonOgre]

That definitely makes sense. I only brought up the method as a potential alternative to investigate to the magic number limit before failing, since it seemed that was mostly to ensure that a StackOverflowException wasn't encountered, which is a thread-killing exception. If it can be calculated or discerned on the fly, based on the context where Resolve() is initially called, at least it's more an environmental constraint than a programmatic one.

I wouldn't be surprised if there were more efficient routes to checking whether the overall resolution chain will circle back on itself and is the more worthwhile end to pursue

[alistairjevans]

What are the thoughts on whether there is anything more to do on this? #1148 introduced changes to how stack depth protections are applied, and how circular dependencies are detected when decorators are involved, introducing the segmented stack.

I'd propose that we close this issue, draw a line under the current history of our circular dependency work, and revisit it if/when there is a problem or performance improvement needed.

[tillig]

I agree, I think the v6 changes will address this about as much as we can, at least for now. If someone sees something additional that can be improved we can open a new issue with that specific proposal.