Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

REGR: unstack on 'int' dtype prevent fillna to work #38244

Merged
merged 2 commits into from Dec 2, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v1.1.5.rst
Expand Up @@ -20,6 +20,7 @@ Fixed regressions
- Fixed regression in inplace operations on :class:`Series` with ``ExtensionDtype`` with NumPy dtyped operand (:issue:`37910`)
- Fixed regression in metadata propagation for ``groupby`` iterator (:issue:`37343`)
- Fixed regression in :class:`MultiIndex` constructed from a :class:`DatetimeIndex` not retaining frequency (:issue:`35563`)
- Fixed regression in :meth:`DataFrame.unstack` with columns with integer dtype (:issue:`37115`)
- Fixed regression in indexing on a :class:`Series` with ``CategoricalDtype`` after unpickling (:issue:`37631`)
- Fixed regression in :meth:`DataFrame.groupby` aggregation with out-of-bounds datetime objects in an object-dtype column (:issue:`36003`)
- Fixed regression in ``df.groupby(..).rolling(..)`` with the resulting :class:`MultiIndex` when grouping by a label that is in the index (:issue:`37641`)
Expand Down
2 changes: 1 addition & 1 deletion pandas/core/internals/blocks.py
Expand Up @@ -1542,7 +1542,7 @@ def _unstack(self, unstacker, fill_value, new_placement):
new_values = new_values.T[mask]
new_placement = new_placement[mask]

blocks = [self.make_block_same_class(new_values, placement=new_placement)]
blocks = [make_block(new_values, placement=new_placement)]
return blocks, mask

def quantile(self, qs, interpolation="linear", axis: int = 0):
Expand Down
27 changes: 27 additions & 0 deletions pandas/tests/frame/test_stack_unstack.py
Expand Up @@ -1880,3 +1880,30 @@ def test_unstack_group_index_overflow(self):
s = Series(np.arange(1000), index=index)
result = s.unstack(4)
assert result.shape == (500, 2)

def test_unstack_with_missing_int_cast_to_float(self):
# https://github.com/pandas-dev/pandas/issues/37115
df = DataFrame(
{
"a": ["A", "A", "B"],
"b": ["ca", "cb", "cb"],
"v": [10] * 3,
}
).set_index(["a", "b"])

# add another int column to get 2 blocks
df["is_"] = 1
assert len(df._mgr.blocks) == 2

result = df.unstack("b")
result[("is_", "ca")] = result[("is_", "ca")].fillna(0)

expected = DataFrame(
[[10.0, 10.0, 1.0, 1.0], [np.nan, 10.0, 0.0, 1.0]],
index=Index(["A", "B"], dtype="object", name="a"),
columns=MultiIndex.from_tuples(
[("v", "ca"), ("v", "cb"), ("is_", "ca"), ("is_", "cb")],
names=[None, "b"],
),
)
tm.assert_frame_equal(result, expected)