Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Snowflake: Support semi-structured data #693

Merged
merged 8 commits into from Nov 2, 2022

Conversation

yuval-illumex
Copy link
Contributor

Per snowflake documentation, Colon can be used to query semi-structured data (JSON).

I decided to use the JsonAccess expression (which is used by postgres), but if you think a new expression is needed - I will change.

@coveralls
Copy link

coveralls commented Oct 27, 2022

Pull Request Test Coverage Report for Build 3375944460

  • 22 of 24 (91.67%) changed or added relevant lines in 4 files are covered.
  • 410 unchanged lines in 4 files lost coverage.
  • Overall coverage increased (+0.03%) to 85.974%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/ast/mod.rs 4 5 80.0%
tests/sqlparser_snowflake.rs 9 10 90.0%
Files with Coverage Reduction New Missed Lines %
src/ast/value.rs 7 87.67%
src/ast/data_type.rs 13 88.62%
tests/sqlparser_common.rs 54 96.85%
src/parser.rs 336 83.74%
Totals Coverage Status
Change from base Build 3292360592: 0.03%
Covered Lines: 10935
Relevant Lines: 12719

💛 - Coveralls

tests/sqlparser_common.rs Outdated Show resolved Hide resolved
@yuval-illumex
Copy link
Contributor Author

@AugustoFKL Thank you for your feedback! I change the code to still use the JsonAccess expression (because it's still access to JSON) But with Snowflake syntax support. Appreciate your feedback!

Copy link
Contributor

@AugustoFKL AugustoFKL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@alamb ready for review :)

@yuval-illumex
Copy link
Contributor Author

yuval-illumex commented Oct 31, 2022

I found a bug on JsonAccess, the right value should be Value and not Identifier (Like Postgres).
However, in snowflake the syntax is field:key, without any quote.
I can't find a way for the parse_value function to check if the dialect is Snowflake - only in test. it works fine from the cli, but in the test i'm getting exception, and the code of Snowflake handling is not executed.

@AugustoFKL WDYT? If it seems right, any change for some assistance with the test?

@AugustoFKL
Copy link
Contributor

@yuval-illumex you can get the current dialect by using dialect_of!(self is SnowflakeDialect)

Please try this. The parser is trying to parse a value but is matching it to another expression. TBH there's 2 possible problems:

  1. It's not using your new value support (maybe conflicting with identifiers).
  2. Using KEY is conflicting with our current keyworks.

I'd try to test an expression without possible keywords and with this dialect validation adjust :)

LMK if it goes ok

src/parser.rs Outdated Show resolved Hide resolved
@yuval-illumex
Copy link
Contributor Author

@AugustoFKL Thank you for your response! your #2nd point was correct!

Appreciate your feedback on the current PR.

src/parser.rs Outdated
@@ -3441,6 +3448,9 @@ impl<'a> Parser<'a> {
Some('\'') => Ok(Value::SingleQuotedString(w.value)),
_ => self.expected("A value?", Token::Word(w))?,
},
Keyword::NoKeyword if dialect_of!(self is SnowflakeDialect) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yuval-illumex adding the generic here is feasible?

I'd suggest adding it but, as identifiers have the same exact syntax, I don't know how that would work.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AugustoFKL added :)

Copy link
Contributor

@AugustoFKL AugustoFKL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@alamb ready for your look :)

@yuval-illumex
Copy link
Contributor Author

@alamb @AugustoFKL I added support when the key the user tries to extract from the json is "location". I have a user who tried it.

Copy link
Contributor

@AugustoFKL AugustoFKL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure whether is reasonable or not to nitpick keywords when parsing.

But If no tests break, don't think it's a problem for now...

Copy link
Collaborator

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM -- thank you @yuval-illumex and @AugustoFKL

@alamb alamb merged commit 93a050e into sqlparser-rs:main Nov 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants