Allow None when nan_as_null=False in column constructor
#15709
Conversation
galipremsagar
changed the title
None when nan_as_null=FalseNone when nan_as_null=False in column constructor
| elif nan_as_null is False and ( | ||
| pd.isna(arbitrary).any() | ||
| and inferred_dtype not in ("decimal", "empty") | ||
| and inferred_dtype not in ("decimal", "empty", "string") | ||
| ): | ||
| # Decimal can hold float("nan") |
There was a problem hiding this comment.
@mroeschke After the changes in this PR, do you think this block might be redundant or does it still capture some error scenarios?
Co-authored-by: Bradley Dice <bdice@bradleydice.com>
Co-authored-by: Bradley Dice <bdice@bradleydice.com>
| pd.isna(arbitrary).any() | ||
| any( | ||
| (isinstance(x, (np.floating, float)) and np.isnan(x)) | ||
| or (inferred_dtype == "boolean" and pd.isna(arbitrary)) |
There was a problem hiding this comment.
Would be good to have (inferred_dtype == "boolean" and pd.isna(arbitrary)) be evaluated outside the loop.
Also what case is this condition trying to catch?
There was a problem hiding this comment.
Done.
Also what case is this condition trying to catch?
It is trying to catch this case:
pd.Series(["a", "b", np.nan], dtype='object')
galipremsagar
added
the
5 - Ready to Merge
| f"Cannot have mixed values with {inferred_dtype}" | ||
| ) | ||
| elif ( | ||
| nan_as_null is False |
There was a problem hiding this comment.
Don't compare to booleans with is.
| nan_as_null is False | |
| not nan_as_null |
There was a problem hiding this comment.
| raise MixedTypeError( | ||
| f"Cannot have mixed values with {inferred_dtype}" | ||
| ) | ||
| elif nan_as_null is False and _has_any_nan(arbitrary): |
There was a problem hiding this comment.
| elif nan_as_null is False and _has_any_nan(arbitrary): | |
| elif not nan_as_null and _has_any_nan(arbitrary): |
There was a problem hiding this comment.
We need this comparison because None is also a supported parameter and it is similar to True behavior.
Co-authored-by: Bradley Dice <bdice@bradleydice.com>
|
/merge |
Description
Fixes: #15708
This PR fixes an issue where we were throwing an error when
Noneis present andnan_as_null=False, this is a bug because of usingpd.isna, this returnsTruefornan,NoneandNA. Whereas we are only looking fornp.nanand notNoneandpd.NAChecklist