Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to DataFusion 17.0.0 #998

Merged
merged 44 commits into from Feb 13, 2023
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
e142cd2
Upgrade to DataFusion 16.0.0
andygrove Jan 18, 2023
0d038b2
fix
andygrove Jan 18, 2023
f03818d
fix todo
andygrove Jan 18, 2023
c5f06b4
Merge branch 'main' of https://github.com/dask-contrib/dask-sql into …
ayushdg Jan 20, 2023
a0fbcbf
Fix clippy style errors
ayushdg Jan 20, 2023
711efb3
Update getOffset and isUnbounded to better handle ScalarValue windowB…
ayushdg Jan 20, 2023
3e97e09
Window Plan: Fix field name order handling since the order now matche…
ayushdg Jan 20, 2023
06c4951
Fix comment location
ayushdg Jan 20, 2023
b4e8b0a
Predicate Filters: Df16 reverts changes in df15 and returns a simpler…
ayushdg Jan 20, 2023
c3c526b
testing
jdye64 Jan 24, 2023
eced31c
Merge remote-tracking branch 'origin/main' into datafusion-16
charlesbluca Jan 24, 2023
68cf931
bump datafusion -> 16.1.0
jdye64 Jan 24, 2023
64f6338
merge update
jdye64 Jan 24, 2023
8e01ac5
Fix 2 failing join pytests
jdye64 Jan 25, 2023
b6e3146
cargo fmt --all
jdye64 Jan 25, 2023
9c2923e
Ok really fix cargo fmt this time
jdye64 Jan 25, 2023
3e7acc8
Update join.rs to accept CAST as a JOIN condition type
jdye64 Jan 25, 2023
dc57fb9
Merge remote-tracking branch 'origin/main' into datafusion-16
charlesbluca Jan 25, 2023
61857bc
Add explicit handling for RexAlias
charlesbluca Jan 25, 2023
345449d
merge with upstream/main
jdye64 Jan 26, 2023
1cf9b5d
Merge branch 'datafusion-16' of github.com:andygrove/dask-sql into da…
jdye64 Jan 26, 2023
2af8d66
Bump DataFusion -> 17.0.0
jdye64 Jan 30, 2023
6875aa3
clippy checks
jdye64 Feb 9, 2023
54d27ce
merge with upstream/main
jdye64 Feb 9, 2023
975b592
Merge remote-tracking branch 'origin/main' into datafusion-16
charlesbluca Feb 9, 2023
1c61577
Fix clippy warnings
charlesbluca Feb 9, 2023
cfc3c21
Add protoc action to workflows for rust and style
jdye64 Feb 9, 2023
62eee30
use arrow re-exported from datafusion
jdye64 Feb 9, 2023
2987e7c
Add protoc action tot conda.yml
jdye64 Feb 9, 2023
d1949fc
Add protobuf install to import / conda build testing
charlesbluca Feb 9, 2023
bd92280
Enable RUST_BACKTRACE=1
jdye64 Feb 9, 2023
955e53b
Add RUST_BACKTRACE=1 to meta.yaml
charlesbluca Feb 9, 2023
5718d6d
Add Rust Toolchain action to conda.yml
jdye64 Feb 9, 2023
4877e5e
Merge branch 'datafusion-16' of github.com:andygrove/dask-sql into da…
jdye64 Feb 9, 2023
f224ab0
Bump setuptools-rust version to 1.5.2 in hopes that helps
jdye64 Feb 9, 2023
ea5f5b8
Enable RUST_BACKTRACE=full
jdye64 Feb 10, 2023
2fae72a
Remove substrait package for now
jdye64 Feb 10, 2023
e324a79
Updates for BinaryExpr in join.rs and comment out FilterColumnsPostJo…
jdye64 Feb 12, 2023
7503e38
Remove protoc github action
jdye64 Feb 12, 2023
41ab6d1
Disable pytest that was using a disabled optimizer rule
jdye64 Feb 12, 2023
477345d
Add libprotobuf=3 dependency
charlesbluca Feb 13, 2023
14b2537
Remove libprotobuf dep
charlesbluca Feb 13, 2023
3c68b53
Remove Rust action from conda.yml
jdye64 Feb 13, 2023
4cadf93
Address review
jdye64 Feb 13, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
108 changes: 71 additions & 37 deletions dask_planner/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 5 additions & 5 deletions dask_planner/Cargo.toml
Expand Up @@ -9,12 +9,12 @@ edition = "2021"
rust-version = "1.62"

[dependencies]
arrow = { version = "28.0.0", features = ["prettyprint"] }
arrow = { version = "29.0.0", features = ["prettyprint"] }
async-trait = "0.1.61"
datafusion-common = "15.0.0"
datafusion-expr = "15.0.0"
datafusion-optimizer = "15.0.0"
datafusion-sql = "15.0.0"
datafusion-common = "16.0.0"
datafusion-expr = "16.0.0"
datafusion-optimizer = "16.0.0"
datafusion-sql = "16.0.0"
env_logger = "0.10"
log = "^0.4"
mimalloc = { version = "*", default-features = false }
Expand Down
2 changes: 1 addition & 1 deletion dask_planner/src/dialect.rs
Expand Up @@ -47,7 +47,7 @@ impl Dialect for DaskDialect {
/// override expression parsing
fn parse_prefix(&self, parser: &mut Parser) -> Option<Result<Expr, ParserError>> {
fn parse_expr(parser: &mut Parser) -> Result<Option<Expr>, ParserError> {
match parser.peek_token() {
match parser.peek_token().token {
Token::Word(w) if w.value.to_lowercase() == "ceil" => {
// CEIL(d TO DAY)
parser.next_token(); // skip ceil
Expand Down