Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Load nullables from dask #9632

Closed
wants to merge 21 commits into from
Closed

Load nullables from dask #9632

wants to merge 21 commits into from

Commits on Nov 2, 2022

  1. Add failing test for use_nullable_dtypes

    Ian Rose committed Nov 2, 2022
    Configuration menu
    Copy the full SHA
    57714ad View commit details
    Browse the repository at this point in the history

  2. WIP use_nullable_dtypes

    Ian Rose committed Nov 2, 2022
    Configuration menu
    Copy the full SHA
    bcd51cf View commit details
    Browse the repository at this point in the history

  3. apply use_nullable_dtypes to meta

    Ian Rose committed Nov 2, 2022
    Configuration menu
    Copy the full SHA
    58659d2 View commit details
    Browse the repository at this point in the history

  4. clean up

    Ian Rose committed Nov 2, 2022
    Configuration menu
    Copy the full SHA
    cd9a6c6 View commit details
    Browse the repository at this point in the history

  5. Raise when using fastparquet

    Ian Rose committed Nov 2, 2022
    Configuration menu
    Copy the full SHA
    32d45eb View commit details
    Browse the repository at this point in the history

  6. Add spark test

    Ian Rose committed Nov 2, 2022
    Configuration menu
    Copy the full SHA
    db09037 View commit details
    Browse the repository at this point in the history

  7. Remove debug code

    Ian Rose committed Nov 2, 2022
    Configuration menu
    Copy the full SHA
    203a1f2 View commit details
    Browse the repository at this point in the history

Commits on Nov 3, 2022

  1. Don't specify storage backend for string dtypes in tests, as that's not 

    available in pandas <= 1.2.0.
    Ian Rose committed Nov 3, 2022
    Configuration menu
    Copy the full SHA
    c77a88e View commit details
    Browse the repository at this point in the history

  2. Speficy use_nullable_dtypes as kwarg

    Ian Rose committed Nov 3, 2022
    Configuration menu
    Copy the full SHA
    f2e1d24 View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2022

  1. Adds handling of nullable dtypes to read_parquet for pyarrow engine

    @hayesgb
    hayesgb committed Nov 6, 2022
    Configuration menu
    Copy the full SHA
    b072d20 View commit details
    Browse the repository at this point in the history

  2. Adds a config option for handling nullable dtypes and string dtypes

    @hayesgb
    hayesgb committed Nov 6, 2022
    Configuration menu
    Copy the full SHA
    9e6e61c View commit details
    Browse the repository at this point in the history

  3. Adds handling of the config to dataframe/io/parquet/arrow.py

    @hayesgb
    hayesgb committed Nov 6, 2022
    Configuration menu
    Copy the full SHA
    970f0f1 View commit details
    Browse the repository at this point in the history

  4. Linting

    @hayesgb
    hayesgb committed Nov 6, 2022
    Configuration menu
    Copy the full SHA
    1b21003 View commit details
    Browse the repository at this point in the history

  5. Fix failing dask_config test

    @hayesgb
    hayesgb committed Nov 6, 2022
    Configuration menu
    Copy the full SHA
    47f7e97 View commit details
    Browse the repository at this point in the history

  6. update config for parquet datatypes

    @hayesgb
    hayesgb committed Nov 6, 2022
    Configuration menu
    Copy the full SHA
    ecec7f8 View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2022

  1. Linting

    @hayesgb
    hayesgb committed Nov 7, 2022
    Configuration menu
    Copy the full SHA
    42b601e View commit details
    Browse the repository at this point in the history

  2. patch schema

    @hayesgb
    hayesgb committed Nov 7, 2022
    Configuration menu
    Copy the full SHA
    5caa375 View commit details
    Browse the repository at this point in the history

  3. Revise test to pull stringtype from config

    @hayesgb
    hayesgb committed Nov 7, 2022
    Configuration menu
    Copy the full SHA
    275d455 View commit details
    Browse the repository at this point in the history

  4. Adds handling of null datatypes to Pyarrow Mapping

    @hayesgb
    hayesgb committed Nov 7, 2022
    Configuration menu
    Copy the full SHA
    e3e0d57 View commit details
    Browse the repository at this point in the history

  5. Linting' 

    git push
q
exit()
    @hayesgb
    hayesgb committed Nov 7, 2022
    Configuration menu
    Copy the full SHA
    f316035 View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2022

  1. Fix dask-schema and add test for np.nan dtypes

    @hayesgb
    hayesgb committed Nov 8, 2022
    Configuration menu
    Copy the full SHA
    7d2df3a View commit details
    Browse the repository at this point in the history