Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Breaking Change in 8.0.4: System.InvalidOperationException: The data is NULL at ordinal 0. This method can't be called on NULL values. Check using IsDBNull before calling. #33547

Open
eisbaer66 opened this issue Apr 16, 2024 · 6 comments · May be fixed by #33559
Assignees
Labels
closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. customer-reported regression type-bug
Milestone

Comments

@eisbaer66
Copy link

File a bug

I have run into this problem when upgrading from 8.0.3 to 8.0.4.
I have a Parent type that contains an optional Child.

public class Parent
{
	public int Id { get; set; }
	public Child? Child { get; set; }
}

The Child contains a simple type, that is saved as a ComplexProperty.

public class Child
{
	public int Id { get; set; }
	public required Complex Complex { get; set; }
	public bool Deleted { get; set; }
}

There is a QueryFilter defined for the Child (simple bool property "Deleted").

public class ApplicationDbContext : DbContext
{
	public DbSet<Parent> Parents { get; set; }


	public ApplicationDbContext(DbContextOptions<ApplicationDbContext> options)
		: base (options)
	{
	}

	protected override void OnModelCreating(ModelBuilder modelBuilder)
	{
		base.OnModelCreating (modelBuilder);

		modelBuilder.Entity<Child> ()
		            .ComplexProperty (x => x.Complex);

		modelBuilder.Entity<Child> ().HasQueryFilter (x => !x.Deleted);
	}
}

The Complex type contains a enum.

public class Complex
{
	public MyEnum Enum { get; set; }
}
public enum MyEnum
{
	None    = 0,
	Enabled = 1,
}

When I now try to query the enum coming from the Parent type though the Child and Complex types, I want the result to be null, if the (optional) Child is not set (is null).

var my_enum = db_context.Parents
                        .Select (p => p.Child != null ? (MyEnum?)p.Child.Complex.Enum : null)
                        .FirstOrDefault ();

But i get a InvalidOperationException instead.

Downgrading back to 8.0.3 restores the expected behavior again.

Include your code

I have created a small solution with one console project to reproduce the problem:
efcore-enum-in-optional-complex-property-with-queryfilter-repro.zip

It creates a sqlite database (deleting any existing "test.db" file beforehand) so you should be able to just run it (again and again) to reproduce the exception.

Stack traces

Unhandled exception. System.InvalidOperationException: An error occurred while reading a database value. The expected type was 'efcore_enum_in_optional_complex_property_repro.Entites.MyEnum' but the actual value was null.
 ---> System.InvalidOperationException: The data is NULL at ordinal 0. This method can't be called on NULL values. Check using IsDBNull before calling.
   at Microsoft.Data.Sqlite.SqliteValueReader.GetInt64(Int32 ordinal)
   at Microsoft.Data.Sqlite.SqliteValueReader.GetInt32(Int32 ordinal)
   at Microsoft.Data.Sqlite.SqliteDataReader.GetInt32(Int32 ordinal)
   at lambda_method31(Closure, QueryContext, DbDataReader, ResultContext, SingleQueryResultCoordinator)
   --- End of inner exception stack trace ---
   at lambda_method31(Closure, QueryContext, DbDataReader, ResultContext, SingleQueryResultCoordinator)
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.MoveNext()
   at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable`1 source, Boolean& found)
   at lambda_method32(Closure, QueryContext)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.Execute[TResult](Expression expression)
   at System.Linq.Queryable.FirstOrDefault[TSource](IQueryable`1 source)
   at Program.<<Main>$>g__Act|0_2(<>c__DisplayClass0_0&) in %path-to-repos%\efcore-enum-in-optional-complex-property-with-queryfilter-repro\efcore-enum-in-optional-complex-property-repro\Program.cs:line 49
   at Program.<Main>$(String[] args) in %path-to-repos%\efcore-enum-in-optional-complex-property-with-queryfilter-repro\efcore-enum-in-optional-complex-property-repro\Program.cs:line 12

Include provider and version information

EF Core version: 8.0.4 (still works in 8.0.3)
Database provider: Microsoft.EntityFrameworkCore.Sqlite 8.0.4 (same error happens with Microsoft.EntityFrameworkCore.SqlServer 8.0.4)
Target framework: .NET 8.0
Operating system: Windows 11
IDE: Visual Studio 2022 17.9.5

@maumar maumar self-assigned this Apr 16, 2024
@PjotrB
Copy link

PjotrB commented Apr 16, 2024

I see that Microsoft.EntityFrameworkCore 8.0.4 is now "unlisted":
https://www.nuget.org/packages/Microsoft.EntityFrameworkCore/8.0.4

Is that because of this issue?

Note that a couple of v8.0.4 packages are still listed, with a reference to Microsoft.EntityFrameworkCore 8.0.4, but they will fail to install or update. IMHO it would be better if all dependent packages would also be unlisted in such a situation.

@dougclutter
Copy link

I see that Microsoft.EntityFrameworkCore 8.0.4 is now "unlisted": https://www.nuget.org/packages/Microsoft.EntityFrameworkCore/8.0.4

Is that because of this issue?

Note that a couple of v8.0.4 packages are still listed, with a reference to Microsoft.EntityFrameworkCore 8.0.4, but they will fail to install or update. IMHO it would be better if all dependent packages would also be unlisted in such a situation.

Agreed.

More importantly to us, where can we find information on why this package was unlisted? Should we hold off on upgrading to 8.0.4? Can we safely use Microsoft.EntityFrameworkCore/8.0.3 with other packages such as Microsoft.EntityFrameworkCore.Relational/8.0.4?

@maumar
Copy link
Contributor

maumar commented Apr 16, 2024

@PjotrB @dougclutter 8.0.4 is back up. Unlisting was not intentional, likely a glitch in our system. We are looking into the details, what exactly happened, but it should be safe to upgrade.

@maumar
Copy link
Contributor

maumar commented Apr 17, 2024

#32911 is the culprit. @eisbaer66 - you can make your scenario work on 8.0.4 by setting the following AppContext switch: "Microsoft.EntityFrameworkCore.Issue32911"

@maumar
Copy link
Contributor

maumar commented Apr 17, 2024

Problem is that upon applying LEFT JOIN operation, Enum value on the complex type is not marked as nullable. We have StructuralTypeProjectionExpression and we mark all the primitive properties as nullable, but for nested structures (like the complex type here) we only mark the shaper itself nullable, but the nested StructuralTypeProjectionExpression that is stored in the ValueBufferExpression is not touched.

In 8.0.3 we were storing all the properties of complex types in the flat structure, along with regular properties, so the nullabilities were set for all of them.

@maumar maumar added this to the 8.0.x milestone Apr 17, 2024
@eisbaer66
Copy link
Author

#32911 is the culprit. @eisbaer66 - you can make your scenario work on 8.0.4 by setting the following AppContext switch: "Microsoft.EntityFrameworkCore.Issue32911"

Can confirm that setting AppContext.SetSwitch("Microsoft.EntityFrameworkCore.Issue32911", true) allows this scenario in 8.0.4 👍

maumar added a commit that referenced this issue Apr 18, 2024
…ption: The data is NULL at ordinal 0. This method can't be called on NULL values. Check using IsDBNull before calling.

Problem was that we changed the way that we store complex type information in the StructuralTypeProjectionExpression, specifically after performing pushdown. We used to extract all the properties from complex types into a flat list, along with regular properties, but that was wrong - we need to uphold the nested structure.
If that STPE with flattened structure would be made nullable, we go through all the properties and mark the  column expressions as nullable (which included properties of a complex type). However, in the new regime, we were not making those changes to columns representing complex type in the nested structure.
As a result, if those columns were later projected (after pushdown), they would seem to be non-nullable, and could cause errors.

Fix is, during MakeNullable() operation for a complex type StructuralTypeProjectionExpression, to peek inside it's ValueBufferExpression and mark all the columns found there as nullable.

Fixes #33547
maumar added a commit that referenced this issue Apr 18, 2024
…ption: The data is NULL at ordinal 0. This method can't be called on NULL values. Check using IsDBNull before calling.

Problem was that we changed the way that we store complex type information in the StructuralTypeProjectionExpression, specifically after performing pushdown. We used to extract all the properties from complex types into a flat list, along with regular properties, but that was wrong - we need to uphold the nested structure.
If that STPE with flattened structure would be made nullable, we go through all the properties and mark the  column expressions as nullable (which included properties of a complex type). However, in the new regime, we were not making those changes to columns representing complex type in the nested structure.
As a result, if those columns were later projected (after pushdown), they would seem to be non-nullable, and could cause errors.

Fix is, during MakeNullable() operation for a complex type StructuralTypeProjectionExpression, to peek inside it's ValueBufferExpression and mark all the columns found there as nullable.

Fixes #33547
@maumar maumar added the closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. label Apr 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. customer-reported regression type-bug
Projects
None yet
4 participants