Feature/translate json functions #29306 #30010

joelmandell · 2023-01-09T23:26:31Z

I've read the guidelines for contributing and seen the walkthrough
I've posted a comment on an issue with a detailed description of how I am planning to contribute and got approval from a member of the team
The code builds and tests pass locally (also verified by our automated build checks)
Commit messages follow this format: (This I totally forgot)

        Summary of the changes
        - Detail 1
        - Detail 2

        Fixes #bugnumber

Tests for the changes have been added (for bug fixes / features)
Code follows the same patterns and style as existing code in this repo (Kinda... little bit different with unit-tests. But want to get your feedback if that's okay).

roji

Thanks for this PR - it's a good start, see comments below.

Note that there are some other JSON functions supported by SQL Server (ISJSON, JSON_ARRAY, JSON_OBJECT...). Ideally we'd have translations for all of them (and the relevant SQLite ones as well); of course you don't have to do that in this PR, but if you feel like working on those that would be great.

src/EFCore.SqlServer/Extensions/SqlServerDbFunctionsExtensions.cs

src/EFCore.Sqlite.Core/Extensions/SqliteDbFunctionsExtensions.cs

src/EFCore.Sqlite.Core/Query/Internal/SqliteJsonFunctionsTranslator.cs

joelmandell

.

joelmandell

I made some changes now. And think I got them all covered. But little bit uncertain about the propagation of null.

roji

@joelmandell sorry, it's taking long for me to get around to this, here are some comments. Please rebase this PR on the latest main and check out any test failures after that.

src/EFCore.SqlServer/Query/Internal/SqlServerJsonFunctionsTranslator.cs

src/EFCore.SqlServer/Extensions/SqlServerDbFunctionsExtensions.cs

src/EFCore.SqlServer/Query/Internal/SqlServerJsonFunctionsTranslator.cs

src/EFCore.Sqlite.Core/Extensions/SqliteDbFunctionsExtensions.cs

src/EFCore.Sqlite.Core/Query/Internal/SqliteJsonFunctionsTranslator.cs

roji · 2023-03-20T13:01:16Z

test/EFCore.Relational.Specification.Tests/TestModels/JsonQuery/JsonEntityBasicString.cs

+// The .NET Foundation licenses this file to you under the MIT license.
+
+namespace Microsoft.EntityFrameworkCore.TestModels.JsonQuery;
+public class JsonEntityBasicString


Any reason to add this rather than simply using JsonEntityBasic?

JsonEntityBasic has mappings of the columns to classes, and this works only for strings If I understand it correctly.

test/EFCore.Sqlite.FunctionalTests/Query/JsonFunctionSqliteTest.cs

celsowm · 2023-06-25T23:00:53Z

json_value in differents rdbms:

mysql: https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=591207ceddca6fe4f11a3ad8589e7c5b
sqlserver: https://dbfiddle.uk/?rdbms=sqlserver_2017&fiddle=5c70245f5bb7c96d4adcc6b10ebed195
postgres: https://dbfiddle.uk/o46x7dyL
sqlite: https://www.db-fiddle.com/f/tMcVDVmjxb5wSjemg34dy8/2

src/EFCore.Sqlite.Core/Query/Internal/SqliteJsonFunctionsTranslator.cs

…ty type as keyless. (dotnet#32032) Fixes dotnet#24722

[main] Update dependencies from dotnet/arcade - React to xunit breaking changes

…to parent table. (dotnet#32031) Similar to JSON entities, owned entities that are mapped to the same table as their owner should be treated as scalars for the purpose of temporal query validation - they are always in sync with the parent entity, so all operations should be allowed for them, not only AsOf. Fixes dotnet#29156

…1012.7 (dotnet#32044) [main] Update dependencies from dotnet/runtime

…1013.17 (dotnet#32049) [release/8.0] Update dependencies from dotnet/runtime

…1014.1 (dotnet#32050) [main] Update dependencies from dotnet/runtime

…1014.5 (dotnet#32054) [main] Update dependencies from dotnet/runtime

…010.4 (dotnet#32057) [main] Update dependencies from dotnet/arcade

…005.1 (dotnet#32061) [release/8.0] Update dependencies from dotnet/arcade

…1015.3 (dotnet#32060) [main] Update dependencies from dotnet/runtime

…n the base. (dotnet#32041) Fixes dotnet#26218 Fixes dotnet#27404

…1016.9 (dotnet#32067) [release/8.0] Update dependencies from dotnet/runtime

…1016.15 (dotnet#32068) [release/8.0] Update dependencies from dotnet/runtime

)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

… 7.0.12 (dotnet#32063) Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…1017.2 (dotnet#32074) [main] Update dependencies from dotnet/runtime

…otnet#32506) Also fixing exception ordering issue. Fixes dotnet#32505

…1224.2 (dotnet#32670) [main] Update dependencies from dotnet/runtime

Closes dotnet#31083 Fixes dotnet#31407

Instead of on a private list on SelectExpression. Part of dotnet#31049

Closes dotnet#32668

Fixes dotnet#30982 Fixes dotnet#32664

…r only because hardcoded column types etc can be moved from provider to relational/core (dotnet#32706) Also removed some tests from InMemory QBT - those tests were duplicated from SqlServer QBT but have been moved to AdHoc test suites and run for all providers.

joelmandell · 2024-01-03T16:22:18Z

Ping @roji. I do not know if it is to early to continue on this one now? :)

…slator.cs Co-authored-by: Shay Rojansky <roji@roji.org>

Co-authored-by: Shay Rojansky <roji@roji.org>

…/joelmandell/efcore into feature/translate_json_functions

roji requested changes Jan 11, 2023

View reviewed changes

roji added the waiting-for-response label Jan 29, 2023

joelmandell requested a review from roji January 29, 2023 21:38

joelmandell commented Feb 10, 2023

View reviewed changes

ajcvickers assigned roji Mar 19, 2023

roji requested changes Mar 20, 2023

View reviewed changes

joelmandell mentioned this pull request Jul 15, 2023

Relational method for checking whether a path in a JSON exists #31136

Open

Delizald suggested changes Jul 20, 2023

View reviewed changes

src/EFCore.Sqlite.Core/Query/Internal/SqliteJsonFunctionsTranslator.cs Show resolved Hide resolved

dotnet-bot and others added 20 commits October 12, 2023 17:20

Merge in 'release/7.0' changes

e3a58ca

Throw for conflicting explicit relationships when configuring an enti…

8e4513e

…ty type as keyless. (dotnet#32032) Fixes dotnet#24722

[main] Update dependencies from dotnet/arcade (dotnet#32000)

d4e344b

[main] Update dependencies from dotnet/arcade - React to xunit breaking changes

Update dependencies from https://github.com/dotnet/runtime build 2023…

058b875

…1012.7 (dotnet#32044) [main] Update dependencies from dotnet/runtime

Update dependencies from https://github.com/dotnet/runtime build 2023…

5ff7e9f

…1013.17 (dotnet#32049) [release/8.0] Update dependencies from dotnet/runtime

Update dependencies from https://github.com/dotnet/runtime build 2023…

92923bd

…1014.1 (dotnet#32050) [main] Update dependencies from dotnet/runtime

Update dependencies from https://github.com/dotnet/runtime build 2023…

9fe9271

…1014.5 (dotnet#32054) [main] Update dependencies from dotnet/runtime

Update dependencies from https://github.com/dotnet/arcade build 20231…

7310b30

…010.4 (dotnet#32057) [main] Update dependencies from dotnet/arcade

Update dependencies from https://github.com/dotnet/arcade build 20231…

fad6e0f

…005.1 (dotnet#32061) [release/8.0] Update dependencies from dotnet/arcade

Update dependencies from https://github.com/dotnet/runtime build 2023…

2b26705

…1015.3 (dotnet#32060) [main] Update dependencies from dotnet/runtime

Remove the owned navigation on the derived type when configuring it o…

2b0283b

…n the base. (dotnet#32041) Fixes dotnet#26218 Fixes dotnet#27404

Update dependencies from https://github.com/dotnet/runtime build 2023…

afc2414

…1016.9 (dotnet#32067) [release/8.0] Update dependencies from dotnet/runtime

Update dependencies from https://github.com/dotnet/runtime build 2023…

2b73ae7

…1016.15 (dotnet#32068) [release/8.0] Update dependencies from dotnet/runtime

[release/8.0] Don't throw for non-generic collection types (dotnet#32059

87a58cc

)

Merge branch 'release/8.0'

a9e3a82

Bump Azure.Identity from 1.10.1 to 1.10.2 (dotnet#32064)

74aa115

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bump Microsoft.AspNetCore.Identity.EntityFrameworkCore from 7.0.11 to…

80c7aa0

… 7.0.12 (dotnet#32063) Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Merge branch 'release/8.0'

aa44131

Update dependencies from https://github.com/dotnet/runtime build 2023…

3591505

…1017.2 (dotnet#32074) [main] Update dependencies from dotnet/runtime

roji and others added 8 commits December 23, 2023 10:31

Refactor queryable collection support for non-relational providers (d…

5defd88

…otnet#32506) Also fixing exception ordering issue. Fixes dotnet#32505

Update dependencies from https://github.com/dotnet/runtime build 2023…

040fe9a

…1224.2 (dotnet#32670) [main] Update dependencies from dotnet/runtime

Redo SQL tree pruning (dotnet#32672)

651d5b9

Closes dotnet#31083 Fixes dotnet#31407

Track prunability of inner joins on the expression itself (dotnet#32674)

e093b2c

Instead of on a private list on SelectExpression. Part of dotnet#31049

Tiny cleanup in SelectExpression.ApplyPredicate (dotnet#32688)

b9a43a9

Prune SQL Server OPENJSON's WITH clause (dotnet#32673)

2e7ea62

Closes dotnet#32668

Clean up and open table cloning logic (dotnet#32665)

566b241

Fixes dotnet#30982 Fixes dotnet#32664

joelmandell and others added 18 commits January 3, 2024 18:32

wip test JSON_VALUE provider.

c385d2b

DbFunctions: JSON_VALUE & JSON_QUERY support MSSQL.

7ea3a09

Additional test for JsonValue and JsonQuery DbFunctions.

4d8ddea

Added support for json_extract for Sqlite provider (DbFunction).

1422e57

Fixes according to suggestions (i.e. comments, doc)

fc730cc

Fix suggestions for json_extract.

3f2b1fe

Do not infer type mapping. Null-check.

7ae58de

Update src/EFCore.SqlServer/Query/Internal/SqlServerJsonFunctionsTran…

2c6fcb4

…slator.cs Co-authored-by: Shay Rojansky <roji@roji.org>

Update src/EFCore.SqlServer/Query/Internal/SqlServerJsonFunctionsTran…

7f110e3

…slator.cs Co-authored-by: Shay Rojansky <roji@roji.org>

fix formatting

46ec69b

Co-authored-by: Shay Rojansky <roji@roji.org>

fix formatting

7376009

Co-authored-by: Shay Rojansky <roji@roji.org>

make dictionary static

27c7bd0

Co-authored-by: Shay Rojansky <roji@roji.org>

Fix formatting

d5a888c

Co-authored-by: Shay Rojansky <roji@roji.org>

Fix formatting

cb2bbc8

Co-authored-by: Shay Rojansky <roji@roji.org>

Fix formatting

02f9f37

Co-authored-by: Shay Rojansky <roji@roji.org>

wip overload jsonextract.

571cc11

Json_Extract overload. Fix formating. Move tests to correct place.

5b2ae4e

Merge branch 'feature/translate_json_functions' of https://github.com…

ee39973

…/joelmandell/efcore into feature/translate_json_functions

joelmandell requested review from wtgodbe and a team as code owners January 3, 2024 17:36

ajcvickers removed the waiting-for-response label Jan 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/translate json functions #29306 #30010

Feature/translate json functions #29306 #30010

joelmandell commented Jan 9, 2023 •

edited

roji left a comment

joelmandell left a comment •

edited

joelmandell left a comment

roji left a comment

roji Mar 20, 2023

joelmandell Jul 15, 2023 •

edited

celsowm commented Jun 25, 2023

joelmandell commented Jan 3, 2024

Feature/translate json functions #29306 #30010

Are you sure you want to change the base?

Feature/translate json functions #29306 #30010

Conversation

joelmandell commented Jan 9, 2023 • edited

roji left a comment

Choose a reason for hiding this comment

joelmandell left a comment • edited

Choose a reason for hiding this comment

joelmandell left a comment

Choose a reason for hiding this comment

roji left a comment

Choose a reason for hiding this comment

roji Mar 20, 2023

Choose a reason for hiding this comment

joelmandell Jul 15, 2023 • edited

Choose a reason for hiding this comment

celsowm commented Jun 25, 2023

joelmandell commented Jan 3, 2024

joelmandell commented Jan 9, 2023 •

edited

joelmandell left a comment •

edited

joelmandell Jul 15, 2023 •

edited