Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The data type of predicate in the row filter should be same in the binary expr #3469

Closed
liukun4515 opened this issue Sep 14, 2022 · 5 comments · Fixed by #3470
Closed

The data type of predicate in the row filter should be same in the binary expr #3469

liukun4515 opened this issue Sep 14, 2022 · 5 comments · Fixed by #3470

Comments

@liukun4515
Copy link
Contributor

@alamb @thinkharderdev
I find a issue caused by this rewrite in my work #3396
if we have two parquet, one has c1,c2, the other has c1,c3.
If the filter is c2 = 1, then it will produce a expr NULL = 1 for the parquet with c1,c3 column.
After I remove the binary type coercion in the physical phase, the binary physical expr of NULL = INT(1) can't be created, because we don't need to support type coercion in creation physical expr.

Originally posted by @liukun4515 in #3380 (comment)

@liukun4515
Copy link
Contributor Author

why not build the build_row_filter from the phy expr?

@liukun4515
Copy link
Contributor Author

If the table schema is c1,c2,c3, the file schema is c1,c3 with filter c2 = lit.

Why not push the predicate = false to the parquet file directly to replace the c2 = NULL

@alamb
Copy link
Contributor

alamb commented Sep 14, 2022

Why not push the predicate = false to the parquet file directly to replace the c2 = NULL

seems like a reasonable thing to me

@liukun4515
Copy link
Contributor Author

If the table schema is c1,c2,c3, the file schema is c1,c3 with filter c2 = lit.

Why not push the predicate = false to the parquet file directly to replace the c2 = NULL

cc @thinkharderdev

@alamb
Copy link
Contributor

alamb commented Sep 15, 2022

I think the more general flow here is to take a predicate like c2 = lit and a schema of c1, c3 and replace c2 with NULL -- which is what the code changed in #3470 is.

Then the predicate is NULL = lit which can be reduced to false if we run the constant evaluator on it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants