Replies: 2 comments
-
Hi! Can you please add a minimal, complete, verifiable example code to your post? That really helps us reproduce the issue to work out what to do. Thank you! |
Beta Was this translation helpful? Give feedback.
-
Thanks @evgri243. I wrote an empty Dask DataFrame using In [1]: import pandas as pd
In [2]: import dask.dataframe as dd
In [3]: df = pd.DataFrame()
In [4]: ddf = dd.from_pandas(df, npartitions=1)
In [5]: ddf.compute()
Out[5]:
Empty DataFrame
Columns: []
Index: []
In [6]: ddf.to_parquet("test_fastparquet.parquet", engine="fastparquet")
In [7]: dd.read_parquet("test_fastparquet.parquet", engine="fastparquet").compute()
Out[7]:
Empty DataFrame
Columns: []
Index: []
In [8]: dd.read_parquet("test_fastparquet.parquet", engine="pyarrow").compute()
Out[8]:
Empty DataFrame
Columns: []
Index: [] How does this differ from your use case? To @bryanwweber's point, if you're able to provide a code snippet we can use to reproduce the issue, that will greatly increase the odds we're able to help debug successfully |
Beta Was this translation helpful? Give feedback.
-
I try to write an empty DataFrame to parquet (simple to_parquet(path)) with fastparquet engine and the resulting DataFrame either has no columns when read with pyarrow or even fails if accessed through fastparquet.
What is the right way to write it? Am I missing something?
Original dataframe:
Read with pandas default reader:
Read wth pandas through fastparquet:
Output folder:
Beta Was this translation helpful? Give feedback.
All reactions