ClickBench: DataFusion / DuckDB comparision scripts

This benchmark compares DataFusion to DuckDB performance with the ClickBench queries aganst the unmodified ClickBench parquet files.

Results

bash setup.sh

Install from crates.io:

cargo install datafusion-cli --version 28.0.0

Or build from source

git clone https://github.com/apache/arrow-datafusion.git
cd datafusion
cargo install --path datafusion-cli

python3 -m venv `pwd`/venv
source venv/bin/activate
pip install duckdb psutil

queres are run with run-datafusion.sh or run-duckdb.sh.

DuckDB:

CREATE=create-single-duckdb.sql bash run-duckdb.sh

DataFusion

DATAFUSION_CLI=./datafusion-cli.413eba1 CREATE=create-single-datafusion.sql bash run-datafusion.sh

More examples in benchmark.sh

Results are written into result.csv

The example python script is hash.py

python3 hash.py

Name		Name	Last commit message	Last commit date
Latest commit History 1 Commits
.gitignore		.gitignore
README.md		README.md
benchmark.sh		benchmark.sh
chart.png		chart.png
create-multi-datafusion.sql		create-multi-datafusion.sql
create-multi-duckdb.sql		create-multi-duckdb.sql
create-single-datafusion.sql		create-single-datafusion.sql
create-single-duckdb.sql		create-single-duckdb.sql
create-view-duckdb.py		create-view-duckdb.py
duckdb.sh		duckdb.sh
hash.py		hash.py
queries-datafusion.sql		queries-datafusion.sql
queries-duckdb.sql		queries-duckdb.sql
result.csv		result.csv
run-datafusion.sh		run-datafusion.sh
run-duckdb.sh		run-duckdb.sh
run-query-duckdb.py		run-query-duckdb.py
setup.sh		setup.sh