Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix NaN bug in ujson #46628

Closed
wants to merge 4 commits into from
Closed

Fix NaN bug in ujson #46628

wants to merge 4 commits into from

Conversation

Erotemic
Copy link
Contributor

@Erotemic Erotemic commented Apr 3, 2022

This ports the fixes I made in ujson ultrajson/ultrajson#514 wrt reading NaN values to pandas.

@simonjayhawkins
Copy link
Member

cc @WillAyd

@@ -417,6 +417,9 @@ def test_encode_time_conversion_dateutil(self):
def test_encode_as_null(self, decoded_input):
assert ujson.encode(decoded_input) == "null", "Expected null"

def test_decode_nan(self, decoded_input):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you create a test in one of the other modules in this same directory? I think this module was copied directly from ujson but is detached from what an end user would see, as no end user really imports ujson through pandas.

Would be a better test if we can re-create how an end user would come to this through one of the pandas objects

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added a test in pandas.tests.io.json.test_pandas that uses the MWE from the bug report. I think this test should stay here as well, as a similar test now exists in ujson itself.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, we could consider "re-vendoring" ujson now that this bug is fixed in it's main branch as well. But I'm not sure if pandas has other modifications or not (and if their are, they probably should be backported anyway).

@rhshadrach rhshadrach added Bug Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate IO JSON read_json, to_json, json_normalize labels Apr 29, 2022
@github-actions
Copy link
Contributor

This pull request is stale because it has been open for thirty days with no activity. Please update and respond to this comment if you're still interested in working on this.

@github-actions github-actions bot added the Stale label May 31, 2022
@Erotemic
Copy link
Contributor Author

This pull request is stale because it has been open for thirty days with no activity. Please update and respond to this comment if you're still interested in working on this.

This should still be merged and needs review.

@mroeschke
Copy link
Member

Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.

@mroeschke mroeschke closed this Jul 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug IO JSON read_json, to_json, json_normalize Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate Stale
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: Pandas's ujson module incorrectly returns None when it reads NaN.
5 participants