Interception for object materialization (a.k.a. "ObjectMaterialized")

[ajcvickers]

Splitting this out as a separate issue as required on #626 so we can track it independently of other life-cycle hooks.

The ObjectMaterialized event will fire after an entity instance has been created and all it's non-navigation properties have been set.

[SidShetye]

@ajcvickers thanks for breaking this down into even smaller work items. Wasn't sure what granularity @divega was looking for.

Anyway, could this be promoted to 3.0.0 milestone just like #15910 ? The reason being we need hooks at both paths to and from the database - to preserve symmetry and avoid data corruption during data transformations. Having both ObjectMaterialized and SavingChanges is the bare minimum for us to unblock our EF Core customers who've been patiently asking us to support EF Core for a very long time. At our end, we can spotlight the 3.0.0 release as "fully supported" when it eventually comes out.

Thanks

[ajcvickers]

@SidShetye Can you give some details on how you intend to use ObjectMaterialized?

[SidShetye]

@ajcvickers : Sure. We basically use integration points in any ORM to enforce our data security pipeline.

Specifically, for data about to be written to the database we'd use something like SavingChanges in EF that results in encrypted data being sent to the database.

So on the reverse path, we again need our security pipeline to kick in that results in plaintext data being presented to the EF application. ObjectMaterialized in EF 6.x allows us that attach point and we'd like to use it similarly in EF Core too. As you can imagine, without anything like ObjectMaterialized on the reverse path, the data remains in an unusable state. For a visual, check out the very first figure on page 1.

[SidShetye]

@ajcvickers - will this be marked for the 3.0 release? Did my response address your question on usage? Thanks

[ajcvickers]

@SidShetye It's pretty unlikely to go into 3.0 at this point.

[SidShetye]

@ajcvickers Could we please have this for 3.1?

[optiks]

@SidShetye Have you considered using value conversions instead? Presumably you have a mechanism for flagging which properties need to be encrypted?

You could also consider introducing an Encrypted (potentially with an implicit operator) and expose that in your domain.

https://docs.microsoft.com/en-us/ef/core/modeling/value-conversions

[ajcvickers]

@SidShetye We haven't decided what will be in 3.1 yet.

[nphmuller]

With this event we could implement child entity ordering in a way that could simplify our code-base a lot.

We used to have a (pretty unoptimized) implementation that relied on internal EFCore classes. This solution broke when transitioning to EFCore 3.0 (as expected, but it was a risk we where willing to take at the time). For now we will probably order manually per entity type that needs it. But it would be great if we configure this globally for the entity type, which this event would allow.

[EMooreMAC]

We need this feature because our entities track their own changes independently of the DbContext, and our framework needs to know the difference between a property being set on initialization vs later. In EF6, we can set an IsInitializing flag to false after the object is materialized, and our tracking code (in property setters) can ignore property changes that happen during initialization.

I'm open to ideas as to how else we could achieve similar functionality.

[optiks]

I'm open to ideas as to how else we could achieve similar functionality.

@EMooreMAC, I’ve solved this in the past. Give me a few days and I’ll try and get some code for you.

[StevenRasmussen]

Ok.... posting this for anyone else that would like a workaround in the short term until this is implemented. This isn't fully tested so use at your own risk... and with the caveat that the underlying code may change at any time since it is relying on some EF Core internals. Perhaps someone from the EF team could comment on how stable/usable this might be for the short term. That being said... I'm using the code with great success.

Declare an interface for methods to be invoked

interface IMaterialize
{
    void OnMaterializing();
    void OnMaterialized();
}

Implement the interface on your model:

class YourModel : IMaterialize
{
    public void OnMaterializing()
    {
        // Called after the constructor has been called and object created
    }
    public void OnMaterialized()
    {
        // Called after all properties have been populated
    }
}

Create a new implementation of the EntityMaterializerSource class which inherits from the EFCore implementation but overrides one of the methods:

class MyEntityMaterializerSource : Microsoft.EntityFrameworkCore.Query.EntityMaterializerSource
{
    public MyEntityMaterializerSource(EntityMaterializerSourceDependencies dependencies)
        : base(dependencies)
    {
    }

    public override Expression CreateMaterializeExpression(IEntityType entityType, string entityInstanceName, Expression materializationContextExpression)
    {
        var baseExpression = base.CreateMaterializeExpression(entityType, entityInstanceName, materializationContextExpression);
        if (entityType.ClrType.GetInterfaces().FirstOrDefault(i => i == typeof(IMaterialize)) != null)
        {
            var onMaterializingMethod = entityType.ClrType.GetMethod(nameof(IMaterialize.OnMaterializing));
            var onMaterializedMethod = entityType.ClrType.GetMethod(nameof(IMaterialize.OnMaterialized));

            var blockExpressions = new List<Expression>(((BlockExpression)baseExpression).Expressions);
            var instanceVariable = blockExpressions.Last() as ParameterExpression;

            var onMaterializingExpression = Expression.Call(instanceVariable, onMaterializingMethod);
            var onMaterializedExpression = Expression.Call(instanceVariable, onMaterializedMethod);

            blockExpressions.Insert(1, onMaterializingExpression);
            blockExpressions.Insert(blockExpressions.Count - 1, onMaterializedExpression);

            return Expression.Block(new[] { instanceVariable }, blockExpressions);
        }

        return baseExpression;
    }
}

Now simply replace the service while adding the DbContext to the service collection or in the Dbcontext.OnConfiguring:

services.AddDbContext<TestContext>(options =>
{
    options.ReplaceService<IEntityMaterializerSource, MyEntityMaterializerSource>();
}...

// Or...
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
    optionsBuilder.ReplaceService<IEntityMaterializerSource, MyEntityMaterializerSource>();
}

[aktxyz]

wow ... MyEntityMaterializerSource works great :) an official approach will be great but for now this is perfect

[titobf]

@StevenRasmussen thanks so much for that workaround! I have just tried it and it works!

[haiplk]

Hi all,,
Just curious about the override EntityMaterializerSource cannot work with nested select

dbContext.Table1.Select(f=> new CustomModel{ DateTimeField = f.DateTime }).ToListAsync()

Do you all have any idea on it

[EMooreMAC]

@StevenRasmussen just wanted to check back and say that this solution is working for us as well. Thanks a ton, you helped clear a blockage for our large scale migration to EF Core.

[smitpatel]

@haiplk - Your query does not materialize any entity.

[Plasma]

#15911 (comment) Is a good solution, but a proper callback method being called (eg to provide an object authorisation service audit the object) would be ideal.

Are there any plans for this? Thank you

[r-work]

@StevenRasmussen thanks for the workaround, however OnMaterialized gets called as soon as the "parent" entity is materialized before its eager loaded sub collections are loaded/materialized.

So unfortunately this won't work if you want a callback after the "whole" entity is loaded.

[avenmore]

It also seems that the implementation in #15911 (comment) does not call OnMaterialized() for entities fetched with .Include(). Also wish for a real solution.

Edit: My mistake - I had called .ReplaceService() on my OptionsBuilder that I use to create manual connections but not on the add services' options too. It does work for .Include().ThenInclude().

[ajcvickers]

Note from triage: consider configure await when implementing this.

[VahidN]

Is it possible to access the related DbCommand here as well? It will help to write a good caching interceptor based on this new feature.

[ajcvickers]

@VahidN No; this is operating at a different level in the stack.

[maxime-poulain]

Hello,

thanks for this, it is exactly what I needed for my current development.(I am updating the IsTransient property of my entities thanks to this).
I would like just to add that the official documentation is not mentionning that this is possible thanks to IMaterializationInterceptor.

You can find that information however in the following article: What's new in EF 7.

But I guess the above link won't be the one people looking for that kind of behavior where they will be looking at.

If you could update the documentation, I am sure it would be helpfull for many people.

[yokovaski]

The official documentation is indeed lacking a reference to the new IMaterializationInterceptor. Here is the direct link to the announcement with an example of how to use it.