Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix interval parsing logic and precedence #705

Merged
merged 6 commits into from Nov 22, 2022

Conversation

sarahyurick
Copy link
Contributor

I originally opened an issue for this on the DataFusion side: apache/datafusion#3944, but found the error to lie in how the queries were being parsed.

These changes allow something like "SELECT col FROM test WHERE d3_date > d1_date + INTERVAL '5 days' AND d2_date > d1_date + INTERVAL '3 days'" to be parsed correctly. Previously, the '5 days' was getting separated from the INTERVAL keyword and lumped into the AND operator.

@coveralls
Copy link

coveralls commented Nov 10, 2022

Pull Request Test Coverage Report for Build 3491640784

  • 82 of 85 (96.47%) changed or added relevant lines in 2 files are covered.
  • 883 unchanged lines in 9 files lost coverage.
  • Overall coverage increased (+0.2%) to 86.341%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/parser.rs 13 14 92.86%
tests/sqlparser_common.rs 69 71 97.18%
Files with Coverage Reduction New Missed Lines %
src/dialect/mod.rs 1 88.46%
src/dialect/snowflake.rs 1 90.0%
tests/sqlparser_mysql.rs 2 99.68%
src/ast/data_type.rs 13 89.71%
src/ast/ddl.rs 16 81.31%
tests/sqlparser_postgres.rs 24 97.27%
tests/sqlparser_common.rs 80 97.16%
src/ast/mod.rs 274 77.47%
src/parser.rs 472 83.85%
Totals Coverage Status
Change from base Build 3413823174: 0.2%
Covered Lines: 11877
Relevant Lines: 13756

💛 - Coveralls

src/parser.rs Show resolved Hide resolved
@@ -363,6 +363,36 @@ impl<'a> Parser<'a> {
Ok(expr)
}

pub fn parse_interval_expr(&mut self) -> Result<Expr, ParserError> {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if you investigated following a similar pattern to NOT which adjusts the precedence based on peeking in the token stream?

sqlparser-rs/src/parser.rs

Lines 1648 to 1660 in 87b4a16

Token::Word(w) if w.keyword == Keyword::NOT => match self.peek_nth_token(1) {
// The precedence of NOT varies depending on keyword that
// follows it. If it is followed by IN, BETWEEN, or LIKE,
// it takes on the precedence of those tokens. Otherwise it
// is not an infix operator, and therefore has zero
// precedence.
Token::Word(w) if w.keyword == Keyword::IN => Ok(Self::BETWEEN_PREC),
Token::Word(w) if w.keyword == Keyword::BETWEEN => Ok(Self::BETWEEN_PREC),
Token::Word(w) if w.keyword == Keyword::LIKE => Ok(Self::LIKE_PREC),
Token::Word(w) if w.keyword == Keyword::ILIKE => Ok(Self::LIKE_PREC),
Token::Word(w) if w.keyword == Keyword::SIMILAR => Ok(Self::LIKE_PREC),
_ => Ok(0),
},

I couldn't find any example of a similar partten in the codebase

https://github.com/search?q=repo%3Asqlparser-rs%2Fsqlparser-rs%20parse_infix&type=code

What do you think @AugustoFKL ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alamb I think we may need to refactor the expression parsing...

With the current structure, we end up with no such good expansion of expressions to parse. And there's still the (big) problem with precedence analysis.

But, for now, and considering the scope of this PR, I think that's OK. Doing it for NOT would be better in another PR, right?

@alamb
Copy link
Collaborator

alamb commented Nov 11, 2022

Thank you for the contribution @sarahyurick

@sarahyurick
Copy link
Contributor Author

Thanks @alamb @AugustoFKL ! Let me know what you think. Should I update anything with the precedence logic, or just keep as is for now?

Copy link
Contributor

@AugustoFKL AugustoFKL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@sarahyurick IMHO that's ok

@alamb alamb changed the title Bug with intervals and logical and/or Fix interval parsing logic and precedence Nov 22, 2022
@alamb alamb merged commit 57083a0 into sqlparser-rs:main Nov 22, 2022
@alamb
Copy link
Collaborator

alamb commented Nov 22, 2022

Thanks again @sarahyurick and @AugustoFKL

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants