BUG: Regression in pandas 1.2.3 in dataframe.__setitem__

[galipremsagar]

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• (optional) I have confirmed this bug exists on the master branch of pandas.

---

Code Sample, a copy-pastable example

>>> import pandas as pd
>>> pd.__version__
'1.2.3'
>>> df = pd.DataFrame({"a": [1, 2, 3]})
>>> df[[True, False, True]] = pd.DataFrame({"a": [-1, -2]})
>>> df
     a
0 -1.0
1  2.0
2  NaN

Whereas this was fixed in 1.2.2:

>>> import pandas as pd
>>> pd.__version__
'1.2.2'
>>> df = pd.DataFrame({"a": [1, 2, 3]})
>>> df[[True, False, True]] = pd.DataFrame({"a": [-1, -2]})
>>> df
   a
0 -1
1  2
2 -2

Problem description

Setting a column based on boolean values seems to be broken in 1.2.3. This previously was broken and fixed in 1.2.2.

Expected Output

>>> import pandas as pd
>>> df = pd.DataFrame({"a": [1, 2, 3]})
>>> df[[True, False, True]] = pd.DataFrame({"a": [-1, -2]})
>>> df
   a
0 -1
1  2
2 -2

Output of pd.show_versions()

INSTALLED VERSIONS

commit : f2c8480
python : 3.7.3.final.0
python-bits : 64
OS : Darwin
OS-release : 20.3.0
Version : Darwin Kernel Version 20.3.0: Thu Jan 21 00:07:06 PST 2021; root:xnu-7195.81.3~1/RELEASE_X86_64
machine : x86_64
processor : i386
byteorder : little
LC_ALL : None
LANG : None
LOCALE : en_US.UTF-8
pandas : 1.2.3
numpy : 1.19.2
pytz : 2020.1
dateutil : 2.8.1
pip : 20.2.3
setuptools : 50.3.0
Cython : None
pytest : None
hypothesis : 5.29.0
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.11.2
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : 1.0.1
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : None

[jreback]

cc @phofl

[phofl]

I am not sure if this is a Regression, This was caused because of the fix for #39931

I think setitem should align the rhs? I would say the regression was in 1.2, that this stopped aligning

cc @jreback @jbrockmendel

[jorisvandenbossche]

The "new" behaviour was indeed was happened before as well, eg with pandas 1.1.5:

In [3]: df[[True, False, True]] = pd.DataFrame({"a": [-1, -2]})

In [4]: df
Out[4]: 
     a
0 -1.0
1  2.0
2  NaN

In [5]: pd.__version__
Out[5]: '1.1.5'

[phofl]

Yes, we go through iloc here, which aligned in before 1.2 and stopped with 1.2. After this the behavior changed, which I think was wrong then for setitem. In 1.2.3 we restored the previous behavior for setitem again, which is more correct I think

[jbrockmendel]

@phofl @jorisvandenbossche it sounds like the consensus is that the behavior on master (and 1.2.3) is correct and the behavior on 1.2.0 was incorrect?

[phofl]

Is correct imho

[phofl]

If nobody disagrees I would close this, since the behavior is correct again in 1.2.3

[jreback]

as long as we have a test ok to close

[phofl]

test_setitem_boolean_mask_aligning covers this

[jreback]

great

[isVoid]

Hi devs, current behavior is inconsistent between series and dataframe. For series:

In [12]: s = pd.Series([1, 2, 3], dtype="int64")

In [13]: s[[True, False, True]] = [88, 99]

In [14]: s
Out[14]: 
0    88
1     2
2    99
dtype: int64

Is this intended behavior?

[phofl]

Please clarify, your example is correct

[isVoid]

Actually, no issue here. I noticed the example I supplied above has no index on value. When I do s[[True, False, True]] = pd.Series([88, 99]) I get the expected result. Sorry about the confusion!