Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Index Monotonic Sort Bug in class DateChunker #974

Open
qcyfred opened this issue Dec 2, 2022 · 0 comments
Open

Index Monotonic Sort Bug in class DateChunker #974

qcyfred opened this issue Dec 2, 2022 · 0 comments

Comments

@qcyfred
Copy link

qcyfred commented Dec 2, 2022

Index Monotonic Sort Bug in class DateChunker (in file date_chunker.py)

If the df's index is not monotonic increasing, arctic will sort the df by index. BUT the variable dates is still not in order.

I suggest arctic to put the code dates = df.index.get_level_values('date') after the if sentence.

def to_chunks(self, df, chunk_size='D', func=None, **kwargs):
    """
    chunks the dataframe/series by dates

    Parameters
    ----------
    df: pandas dataframe or series
    chunk_size: str
        any valid Pandas frequency string
    func: function
        func will be applied to each `chunk` generated by the chunker.
        This function CANNOT modify the date column of the dataframe!

    Returns
    -------
    generator that produces tuples: (start date, end date,
              chunk_size, dataframe/series)
    """
    if 'date' in df.index.names:
        dates = df.index.get_level_values('date')
        if not df.index.is_monotonic_increasing:
            df = df.sort_index()
        # TODO dates won't be sorted, which will cause data store error.
        
      # dates = df.index.get_level_values('date')

Anyway, arctic is an excellent project !

这是我第一次在github上留言。蹩脚的英文。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant