[release/8.0] Fix to #33547 - Breaking Change in 8.0.4: System.InvalidOperationException: The data is NULL at ordinal 0. This method can't be called on NULL values. Check using IsDBNull before calling. #33692
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ption: The data is NULL at ordinal 0. This method can't be called on NULL values. Check using IsDBNull before calling. Problem was that we changed the way that we store complex type information in the StructuralTypeProjectionExpression, specifically after performing pushdown. We used to extract all the properties from complex types into a flat list, along with regular properties, but that was wrong - we need to uphold the nested structure. If that STPE with flattened structure would be made nullable, we go through all the properties and mark the column expressions as nullable (which included properties of a complex type). However, in the new regime, we were not making those changes to columns representing complex type in the nested structure. As a result, if those columns were later projected (after pushdown), they would seem to be non-nullable, and could cause errors. Fix is, during MakeNullable() operation for a complex type StructuralTypeProjectionExpression, to peek inside it's ValueBufferExpression and mark all the columns found there as nullable. Fixes #33547
AndriySvyryd
approved these changes
May 9, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Port of #33559
Fixes #33547
Description
Problem was that we changed the way that we store complex type information in the StructuralTypeProjectionExpression, specifically after performing pushdown. We used to extract all the properties from complex types into a flat list, along with regular properties, but that was wrong - we need to uphold the nested structure. If that STPE with flattened structure would be made nullable, we go through all the properties and mark the column expressions as nullable (which included properties of a complex type). However, in the new regime, we were not making those changes to columns representing complex type in the nested structure. As a result, if those columns were later projected (after pushdown), they would seem to be non-nullable, and could cause errors.
Fix is, during MakeNullable() operation for a complex type StructuralTypeProjectionExpression, to peek inside it's ValueBufferExpression and mark all the columns found there as nullable.
Customer impact
Queries projecting a non-optional property from an optional complex type via pushdown will fail. Workaround is to manually cast the property to nullable (which seems redundant) and is not always easy/possible without significant query rewrite.
How found
Customer reported on 8.
Regression
Yes, moreover, regression has been introduced in a patch 8.0.4 patch release. (by a high impact, medium risk fix - #33212)
Testing
Multiple tests added.
Risk
Low. Change is straightforward invoking an existing functionality of a StructuralTypeProjectionExpression to make it's subcomponents nullable (like we should have done already). We apply similar logic in other places of the code. Added quirk just to be safe.