Skip to content

Commit

Permalink
API: read_stata with index_col=None should return RangeIndex (pandas-…
Browse files Browse the repository at this point in the history
…dev#49745)

* API: read_stata with index_col=None return RangeIndex

* fix comments

* fix comments II

Co-authored-by: Terji Petersen <terjipetersen@Terjis-MacBook-Air.local>
Co-authored-by: Terji Petersen <terjipetersen@Terjis-Air.fritz.box>
  • Loading branch information
3 people authored and twoertwein committed Nov 20, 2022
1 parent 256fad5 commit 4d5dc72
Show file tree
Hide file tree
Showing 3 changed files with 17 additions and 2 deletions.
2 changes: 2 additions & 0 deletions doc/source/whatsnew/v2.0.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -342,6 +342,7 @@ Other API changes
- Passing strings that cannot be parsed as datetimes to :class:`Series` or :class:`DataFrame` with ``dtype="datetime64[ns]"`` will raise instead of silently ignoring the keyword and returning ``object`` dtype (:issue:`24435`)
- Passing a sequence containing a type that cannot be converted to :class:`Timedelta` to :func:`to_timedelta` or to the :class:`Series` or :class:`DataFrame` constructor with ``dtype="timedelta64[ns]"`` or to :class:`TimedeltaIndex` now raises ``TypeError`` instead of ``ValueError`` (:issue:`49525`)
- Changed behavior of :class:`Index` constructor with sequence containing at least one ``NaT`` and everything else either ``None`` or ``NaN`` to infer ``datetime64[ns]`` dtype instead of ``object``, matching :class:`Series` behavior (:issue:`49340`)
- :func:`read_stata` with parameter ``index_col`` set to ``None`` (the default) will now set the index on the returned :class:`DataFrame` to a :class:`RangeIndex` instead of a :class:`Int64Index` (:issue:`49745`)
- Changed behavior of :class:`Index` constructor with an object-dtype ``numpy.ndarray`` containing all-``bool`` values or all-complex values, this will now retain object dtype, consistent with the :class:`Series` behavior (:issue:`49594`)
-

Expand Down Expand Up @@ -596,6 +597,7 @@ Performance improvements
- Memory improvement in :meth:`RangeIndex.sort_values` (:issue:`48801`)
- Performance improvement in :class:`DataFrameGroupBy` and :class:`SeriesGroupBy` when ``by`` is a categorical type and ``sort=False`` (:issue:`48976`)
- Performance improvement in :class:`DataFrameGroupBy` and :class:`SeriesGroupBy` when ``by`` is a categorical type and ``observed=False`` (:issue:`49596`)
- Performance improvement in :func:`read_stata` with parameter ``index_col`` set to ``None`` (the default). Now the index will be a :class:`RangeIndex` instead of :class:`Int64Index` (:issue:`49745`)
- Performance improvement in :func:`merge` when not merging on the index - the new index will now be :class:`RangeIndex` instead of :class:`Int64Index` (:issue:`49478`)

.. ---------------------------------------------------------------------------
Expand Down
2 changes: 1 addition & 1 deletion pandas/io/stata.py
Original file line number Diff line number Diff line change
Expand Up @@ -1725,7 +1725,7 @@ def read(
# If index is not specified, use actual row number rather than
# restarting at 0 for each chunk.
if index_col is None:
rng = np.arange(self._lines_read - read_lines, self._lines_read)
rng = range(self._lines_read - read_lines, self._lines_read)
data.index = Index(rng) # set attr instead of set_index to avoid copy

if columns is not None:
Expand Down
15 changes: 14 additions & 1 deletion pandas/tests/io/test_stata.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,19 @@ def test_read_empty_dta(self, version):
empty_ds2 = read_stata(path)
tm.assert_frame_equal(empty_ds, empty_ds2)

@pytest.mark.parametrize("version", [114, 117, 118, 119, None])
def test_read_index_col_none(self, version):
df = DataFrame({"a": range(5), "b": ["b1", "b2", "b3", "b4", "b5"]})
# GH 7369, make sure can read a 0-obs dta file
with tm.ensure_clean() as path:
df.to_stata(path, write_index=False, version=version)
read_df = read_stata(path)

assert isinstance(read_df.index, pd.RangeIndex)
expected = df.copy()
expected["a"] = expected["a"].astype(np.int32)
tm.assert_frame_equal(read_df, expected, check_index_type=True)

@pytest.mark.parametrize("file", ["stata1_114", "stata1_117"])
def test_read_dta1(self, file, datapath):

Expand Down Expand Up @@ -1054,7 +1067,7 @@ def test_categorical_sorting(self, file, datapath):
parsed = parsed.sort_values("srh", na_position="first")

# Don't sort index
parsed.index = np.arange(parsed.shape[0])
parsed.index = pd.RangeIndex(len(parsed))
codes = [-1, -1, 0, 1, 1, 1, 2, 2, 3, 4]
categories = ["Poor", "Fair", "Good", "Very good", "Excellent"]
cat = pd.Categorical.from_codes(
Expand Down

0 comments on commit 4d5dc72

Please sign in to comment.