Description
Include your code
Beginning with EF Core 7.0 rc2 when a database entity is generated through a projection and then saved as a new entity, EF Core will throw exception due to "temporary value" of Id property.
Workaround: set Id to default before saving.
Pre-7.0 this was not necessary. If this is expected, it would be nice to add to https://learn.microsoft.com/en-us/ef/core/what-is-new/ef-core-7.0/breaking-changes
var digests = await _context.Users
.Select(u => new DailyDigest
{
User = u,
TimeCreatedUtc = utcNow,
})
.ToListAsync();
foreach (var digest in digests)
{
// next line is necessary in 7.0 rc2, was not necessary in 6.0 and below (down to 1.0 ?)
// digest.Id = default;
_context.DailyDigests.Add(digest);
}
await _context.SaveChangesAsync();Include stack traces
System.InvalidOperationException: The property 'DailyDigest.Id' has a temporary value while attempting to change the entity's state to 'Unchanged'. Either set a permanent value explicitly, or ensure that the database is configured to generate values for this property.
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntry.SetEntityState(EntityState oldState, EntityState newState, Boolean acceptChanges, Boolean modifyProperties)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntry.SetEntityState(EntityState entityState, Boolean acceptChanges, Boolean modifyProperties, Nullable`1 forceStateWhenUnknownKey, Nullable`1 fallbackState)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntry.AcceptChanges()
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.AcceptAllChanges(IReadOnlyList`1 changedEntries)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(StateManager stateManager, Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.<>c__DisplayClass30_0`2.<<ExecuteAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementationAsync[TState,TResult](Func`4 operation, Func`4 verifySucceeded, TState state, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementationAsync[TState,TResult](Func`4 operation, Func`4 verifySucceeded, TState state, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteAsync[TState,TResult](TState state, Func`4 operation, Func`4 verifySucceeded, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken)
Include provider and version information
EF Core version: 7.0 rc2
Database provider: Microsoft.EntityFrameworkCore.SqlServer
Target framework: 7.0 rc2
Operating system: Win11
IDE: Visual Studio 2022 17.4
Metadata
Metadata
Assignees
Type
Projects
Relationships
Participants
Activity
ajcvickers commentedon Nov 8, 2022
@joakimriedel I'm not able to reproduce this--see my code below. Please attach a small, runnable project or post a small, runnable code listing that reproduces what you are seeing so that we can investigate.
joakimriedel commentedon Nov 8, 2022
@ajcvickers I'm terribly sorry for posting a bad issue without repro.
The real project is a huge codebase, five years old and iterated on since the very first version of EF Core, which makes it quite challenging to break parts out in a smaller project and still reproduce. This issue was the only thing (except how hard it was to regenerate compiled model) that I had after migrating from 6.0.10 to 7.0-rc2 (which in itself is amazing! 🙌). I've now spent the last hours trying to reproduce in a smaller project, but without success.
It really bugs me because there is absolutely nothing I can think of in the code that might set or alter
Idin any way, I simply generateDailyDigestobject in the projection, setEmailproperty on it in a loop (after generating an email), and then save it to database. But still I have to setIdmanually todefault intto make it work in 7.0-rc2.Note that your sample has one issue, since it projects from
Usersit's necessary to have at least oneUserin the database:What I've applied from real project so far that I thought might help reproduce;
dotnet ef dbcontext optimize)sqlOptions.UseQuerySplittingBehavior(QuerySplittingBehavior.SingleQuery))HierarchyIdextensionWhat I haven't tried so far, but might affect results;
I'd appreciate any tips or hints on what was changed in 7.0 that you think might affect migrating from 6.0.
ajcvickers commentedon Nov 8, 2022
@joakimriedel Have you tried with 7.0 GA?
joakimriedel commentedon Nov 9, 2022
@ajcvickers Just installed 7.0 GA but this bug still reproduces in real project. I think there's something really fishy going on, perhaps memory corruption or bad caching somewhere? Look at the following verbose logs;
Verbose log
This line is especially troubling since Id 82485 has already been set earlier, why does it change into 82460?
Looks like it is when this entity is about to switch from Added to Unchanged that this exception is thrown.
The DailyDigests will be generated in batches, default batch size is 50. If I change batch size to 1 the bug does not reproduce. It seems only as if I can reproduce this if SaveChangesAsync is called for a large number of items saved to db at once.
joakimriedel commentedon Nov 9, 2022
@ajcvickers while on the topic of logs, I noted that the following log message seems to have reversed property and class name in output string?
Should be
DailyDigest.IdnotId.DailyDigestright?ajcvickers commentedon Nov 9, 2022
@roji Looks like this could be a batching issue related to #28654 and #29356.
@joakimriedel Can you post the SQL generated for the SaveChanges that is being executed when the exception is thrown?
joakimriedel commentedon Nov 9, 2022
@ajcvickers Sure, this is the full log produced when executing the SaveChangesAsync call.
It does indeed look related to batching. I save 50 items here, but somehow it was decided to split into 42 in one batch and 8 in second, and the error seems to occur with the id of the item on the boundary between the two batches.
Full log output
joakimriedel commentedon Nov 9, 2022
@ajcvickers Spent the day trying to get the repro project as similar to the real project as possible, adding generic host, connection pooling and enable multiple active result sets in connection string.
It still only reproduces in the real project but I have at least managed to get a debug log output which is identical to the real project log output, down to a few lines which may or may not have something to do with this bug. See the diff below, in real project when it fails a db reader is closed during detect changes phase.
Related or red herring? You decide.
edit: full diff here
joakimriedel commentedon Nov 10, 2022
@ajcvickers @roji I think I managed to find something interesting today.
I experimented with maxCount (the
.Take(maxCount)in the query) and it seems that I get the exception exactly on 32 items or more. AllmaxCountless than 32 works perfectly.Then I tried reproducing with 32 items after adding the following workaround to the table configuration;
builder.ToTable(tb => tb.HasTrigger("workaround"));This works! 🤯
But the SQL query will be a bit different;
Without HasTrigger
With HasTrigger
Could this difference (missing sort) be the cause of this exception?
I hope you could have a look at this because at this moment I feel not safe about pushing EF7/NET7 to production without resolving this issue.
[-]Potential breaking change with identity column in EF Core 7 when saving entity generated from projection[/-][+]Issue with identity column in EF Core 7 when saving >31 entities generated from projection[/+]roji commentedon Nov 10, 2022
@joakimriedel the difference in SQL between the trigger and non-trigger cases - including the missing sort - are very much by-design, and are a performance optimization introduced in EF Core 7.0 (see this blog post for more information).
However, I don't doubt that there's a bug here; are you now able to submit a reprocible code sample, similar to what @ajcvickers attempted above?
joakimriedel commentedon Nov 10, 2022
@roji I've now spent ~16 hours trying to reproduce this outside of the real project. I've imported almost every aspect that I can think of that might affect the results from the real project into the repro project unsuccessfully. The only thing missing now is the real data, which would be the next step to import into repro project.
As I wrote above, the debug output from repro and real are the same, but it only fails in real project. In real project it is 100% reproducible. 31 items => no exception, 32 items => exception. 32 items w/ HasTrigger => no exception. 32 items where Id is set explicitly to default int => no exception. 32 items in version 6.0.10 => no exception.
As the bug is hard to reproduce in repro project I'm starting to think there is some kind of memory corruption, caching or timing issue that only occurs when full project is loaded and the entity graph is more saturated with data.
roji commentedon Nov 10, 2022
@joakimriedel any memory corruption here is very unlikely... Using actual (or similar) data in your repro is definitely a good step forward, otherwise it's good practice to start from your real project and progressively strip it down until you get a minimal repro.
In any case, I did do various work in this area for 7.0, so it makes sense for there to be a bug here.
63 remaining items