Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support index configuration like dbt-postgres #161

Open
jwills opened this issue May 3, 2023 · 1 comment
Open

Support index configuration like dbt-postgres #161

jwills opened this issue May 3, 2023 · 1 comment

Comments

@jwills
Copy link
Collaborator

jwills commented May 3, 2023

DuckDB has indexes like Postgres does, and I suspect that we will discover some use cases and pipelines where the ability to add them in helps significantly with performance.

dbt-postgres includes built-in support for creating indexes on relations, and I suspect that we will want to add the same to dbt-duckdb at some point, so I created a branch that includes the config and tests from the dbt-postgres indexing config ported over to dbt-duckdb.

I'm going to leave it out of the main branch right now though because the current limitations around DuckDB indexes mean that the Postgres-style index configuration doesn't work for a) incremental models or b) seeds. When those issues are fixed in DuckDB (and the corresponding tests pass), we can revisit including this functionality.

@jwills
Copy link
Collaborator Author

jwills commented May 3, 2023

For the time being, assuming your use case and materialization strategy supports it, I recommend using hooks to drop/add indexes from any models in DuckDB that need them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant