BUG: Fix integer overflow in in1d for mixed integer dtypes #22877

[MilesCranmer]

This fixes #22877 raised by @TonyXiang8787. The bug, introduced by #12065, results in integer overflows occurring in the following line:

numpy/numpy/lib/arraysetops.py

Lines 683 to 684 in 2b9851b

	outgoing_array[basic_mask] = isin_helper_ar[ar1[basic_mask] -
	ar2_min]

when mixed dtype input was passed to in1d.

The fix is to simply test for these in advance of the kind='table' method being used:

        #  2. Check overflows for (ar2 - ar2_min); dtype=ar2.dtype
        range_safe_from_overflow = ar2_range <= np.iinfo(ar2.dtype).max
        #  3. Check overflows for (ar1 - ar2_min); dtype=ar1.dtype
        range_safe_from_overflow &= int(ar1_max) - int(ar2_min) <= np.iinfo(ar1.dtype).max
        range_safe_from_overflow &= int(ar1_min) - int(ar2_min) >= np.iinfo(ar1.dtype).min

I also added some unittests to evaluate this behavior.

cc @seberg

[MilesCranmer]

Local tests pass. Ready for review @seberg

[MilesCranmer]

This PR also corrects the bounds of the overflow check. It should have really been <= in #12065, rather than <. I noticed this after adding some new tests.

[seberg]

This is that last thing that would be great to have some fix for in 1.24.1 considering that it returns bad results. I expect this is good, but I need fresher eyes. But if it looks good to others: Maybe we should get it in, and I look at it again later and just follow up if I feel a different approach is better.

[MilesCranmer]

I'm thinking about this proposed change now and while it does fix the problem, it is more conservative than necessary. Recall we are looking for overflows in this calculation:

basic_mask = (ar1 <= ar2_max) & (ar1 >= ar2_min)
outgoing_array[basic_mask] = isin_helper_ar[ar1[basic_mask] - ar2_min]

basic_mask will trim ar1 to only the elements within the range of ar2. Thus, we only technically need to consider min(ar1_max, ar2_max), and max(ar1_min, ar2_min), in these calculations. i.e., the following:

# After masking, the range of ar1 is guaranteed to be
# within the range of ar2:
ar1_upper = min(int(ar1_max), int(ar2_max))
ar1_lower = max(int(ar1_min), int(ar2_min))

range_safe_from_overflow &= all((
    ar1_upper - int(ar2_min) <= np.iinfo(ar1.dtype).max,
    ar1_lower - int(ar2_min) >= np.iinfo(ar1.dtype).min
))

does that make sense?

---

Edit: pushed this change.

[charris]

I wonder why integers larger the uint16 are not tested, are they too big?

[charris]

Thanks @MilesCranmer. @seberg If you see anything that bothers you we can make another PR.

[charris]

Thanks @MilesCranmer. @seberg If you see anything that bothers you we can make another PR.