Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for empty array literals (ARRAY[]) #532

Merged

Conversation

bitemyapp
Copy link
Contributor

We had SQL queries that contained fragments that looked like this:

,COALESCE(t.location_id_tokenized, ARRAY []) AS location_id_tokenized

We needed to coalesce the values down to empty arrays. This is the recommended syntax on the AWS Athena docs: https://docs.aws.amazon.com/athena/latest/ug/creating-arrays.html

Oddly, the parser assumed all ARRAY keyword literal values would have a comma separated list of values inside of them. This initially threw me off as I assumed it would be a lexing problem. That's one reason there's an AthenaSqlDialect in here. The other reason is that I wasn't sure if I should bolt the ARRAY[] test onto an existing dialect or carve out an Athena dialect. The Athena SQL dialect tests are a hodgepodge of Hive tests with a single array literal test from PostgreSQL adapted to this specific missing feature.

If you want to migrate the test into an existing dialect and nuke the Athena dialect, that's fine by me. Athena SQL is basically Presto+Hive I think, so it's whatever you want to do. This problem is fixed for all SQL dialects regardless because it's in the parser.

Cheers and thanks for the parser, this thing has saved me countless hours.

@Dandandan
Copy link
Collaborator

Hey @bitemyapp , thanks for the PR!

If you want to migrate the test into an existing dialect and nuke the Athena dialect, that's fine by me. Athena SQL is basically Presto+Hive I think, so it's whatever you want to do. This problem is fixed for all SQL dialects regardless because it's in the parser.

Yes please, that sounds like the best way forward.

@coveralls
Copy link

coveralls commented Jul 6, 2022

Pull Request Test Coverage Report for Build 2625673440

  • 12 of 13 (92.31%) changed or added relevant lines in 2 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.07%) to 89.88%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/parser.rs 7 8 87.5%
Totals Coverage Status
Change from base Build 2584755255: 0.07%
Covered Lines: 8873
Relevant Lines: 9872

💛 - Coveralls

@bitemyapp
Copy link
Contributor Author

Yes please, that sounds like the best way forward.

Okie dokie artichokie. I'll push it tomorrow.

@bitemyapp
Copy link
Contributor Author

@Dandandan oh, actually, where should the test specifically for the ARRAY[] syntax go? I nabbed the value-full version from PGSQL but not sure if that's where you actually want it.

@bitemyapp bitemyapp force-pushed the bitemyapp/add-support-for-empty-arrays branch from 2053cd4 to 6f09301 Compare July 6, 2022 21:17
@bitemyapp
Copy link
Contributor Author

Squashed the commits, tests pass locally.

@bitemyapp bitemyapp changed the title Added explicit Athena dialect, support for empty array literals Support for empty array literals Jul 7, 2022
@bitemyapp
Copy link
Contributor Author

Amended the MR title to fit the reality better.

Copy link
Collaborator

@Dandandan Dandandan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good 👍

@Dandandan
Copy link
Collaborator

Dandandan commented Jul 13, 2022

@bitemyapp could you fix the code style issues? Otherwise this is good to go!

@alamb alamb changed the title Support for empty array literals Support for empty array literals (ARRAY[]) Jul 18, 2022
@bitemyapp bitemyapp force-pushed the bitemyapp/add-support-for-empty-arrays branch from 6f09301 to ff82113 Compare July 18, 2022 15:16
@bitemyapp
Copy link
Contributor Author

@Dandandan it should be fixed now. FWIW that license template header option seems to be gone in newer versions of cargo fmt.

@Dandandan Dandandan merged commit f747534 into sqlparser-rs:main Jul 18, 2022
@Dandandan
Copy link
Collaborator

@Dandandan it should be fixed now. FWIW that license template header option seems to be gone in newer versions of cargo fmt.

Thank you @bitemyapp !

@bitemyapp bitemyapp deleted the bitemyapp/add-support-for-empty-arrays branch July 20, 2022 16:39
mobuchowski pushed a commit to mobuchowski/sqlparser-rs that referenced this pull request Aug 3, 2022
Co-authored-by: Chris Allen <chrisa@indeed.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants