Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More restrictive input validation during predicate pushdown filter extraction #1269

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

charlesbluca
Copy link
Collaborator

While running some TPC-H queries using dask-sql + dask-cudf, I noticed some failures around predicate pushdown that seemed to be caused by us incorrectly pushing down blockwise comparisons between columns.

This PR attempts to tighten the logic of _blockwise_comparison_dnf to block this edge case; marking this as draft as I'm not sure if we want to block all RHS getitem calls yet.

cc @rjzamora

@charlesbluca charlesbluca changed the title More restrictive input validation during predicate pushdown More restrictive input validation during predicate pushdown filter extraction Nov 20, 2023
@codecov-commenter
Copy link

Codecov Report

Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (2f068f9) 85.55% compared to head (d841dcc) 85.51%.

Files Patch % Lines
dask_sql/physical/utils/filter.py 0.00% 1 Missing and 1 partial ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1269      +/-   ##
==========================================
- Coverage   85.55%   85.51%   -0.05%     
==========================================
  Files          77       77              
  Lines        4257     4259       +2     
  Branches      758      759       +1     
==========================================
  Hits         3642     3642              
- Misses        446      447       +1     
- Partials      169      170       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@charlesbluca charlesbluca marked this pull request as ready for review November 20, 2023 20:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants