Skip to content

Commit

Permalink
Merge pull request #17239 from tylerjereddy/treddy_backport_193
Browse files Browse the repository at this point in the history 
MAINT: 1.9.3 backports
  • Loading branch information
@tylerjereddy
tylerjereddy committed Oct 19, 2022
2 parents e7fa417 + ba33e43 commit 25e6b90
Show file tree
Hide file tree
Showing 57 changed files with 1,066 additions and 318 deletions.
103 changes: 103 additions & 0 deletions doc/release/1.9.3-notes.rst
View file
Expand Up @@ -10,10 +10,113 @@ compared to 1.9.2.
Authors
=======

* Jelle Aalbers (1)
* Peter Bell (1)
* Jake Bowhay (3)
* Matthew Brett (3)
* Evgeni Burovski (5)
* drpeteb (1) +
* Sebastian Ehlert (1) +
* GavinZhang (1) +
* Ralf Gommers (2)
* Matt Haberland (15)
* Lakshaya Inani (1) +
* Joseph T. Iosue (1)
* Nathan Jacobi (1) +
* jmkuebler (1) +
* Nikita Karetnikov (1) +
* Lechnio (1) +
* Nicholas McKibben (1)
* Andrew Nelson (1)
* o-alexandre-felipe (1) +
* Tirth Patel (1)
* Tyler Reddy (51)
* Martin Reinecke (1)
* Marie Roald (1) +
* Pamphile Roy (2)
* Eli Schwartz (1)
* serge-sans-paille (1)
* ehsan shirvanian (1) +
* Mamoru TASAKA (1) +
* Samuel Wallan (1)
* Warren Weckesser (7)
* Gavin Zhang (1) +

A total of 31 people contributed to this release.
People with a "+" by their names contributed a patch for the first time.
This list of names is automatically generated, and may not be fully complete.

Issues closed for 1.9.3
-----------------------

* `#3691 <https://github.com/scipy/scipy/issues/3691>`__: scipy.interpolate.UnivariateSpline segfault
* `#5286 <https://github.com/scipy/scipy/issues/5286>`__: BUG: multivariate_normal returns a pdf for values outside its...
* `#6551 <https://github.com/scipy/scipy/issues/6551>`__: BUG: stats: inconsistency in docs and behavior of gmean and hmean
* `#9245 <https://github.com/scipy/scipy/issues/9245>`__: running scipy.interpolate.tests.test_fitpack::test_bisplev_integer_overflow...
* `#12471 <https://github.com/scipy/scipy/issues/12471>`__: test_bisplev_integer_overflow: Segmentation fault (core dumped)
* `#13321 <https://github.com/scipy/scipy/issues/13321>`__: Bug: setting iprint=0 hides all output from fmin_l_bfgs_b, but...
* `#13730 <https://github.com/scipy/scipy/issues/13730>`__: \`scipy.stats.mood\` does not correct for ties
* `#14019 <https://github.com/scipy/scipy/issues/14019>`__: ks_2samp throws \`RuntimeWarning: overflow encountered in double_scalars\`
* `#14589 <https://github.com/scipy/scipy/issues/14589>`__: \`shgo\` error since scipy 1.8.0.dev0+1529.803e52d
* `#14591 <https://github.com/scipy/scipy/issues/14591>`__: Input data validation for RectSphereBivariateSpline
* `#15101 <https://github.com/scipy/scipy/issues/15101>`__: BUG: binom.pmf - RuntimeWarning: divide by zero
* `#15342 <https://github.com/scipy/scipy/issues/15342>`__: BUG: scipy.optimize.minimize: Powell's method function evaluated...
* `#15964 <https://github.com/scipy/scipy/issues/15964>`__: BUG: lombscargle fails if argument is a view
* `#16211 <https://github.com/scipy/scipy/issues/16211>`__: BUG: Possible bug when using winsorize on pandas data instead...
* `#16459 <https://github.com/scipy/scipy/issues/16459>`__: BUG: stats.ttest_ind returns wrong p-values with permutations
* `#16500 <https://github.com/scipy/scipy/issues/16500>`__: odr.Model default meta value fails with __getattr__
* `#16519 <https://github.com/scipy/scipy/issues/16519>`__: BUG: Error in error message for incorrect sample dimension in...
* `#16527 <https://github.com/scipy/scipy/issues/16527>`__: BUG: dimension of isuppz in syevr is mistranslated
* `#16600 <https://github.com/scipy/scipy/issues/16600>`__: BUG: \`KDTree\`'s optional argument \`eps\` seems to have no...
* `#16656 <https://github.com/scipy/scipy/issues/16656>`__: dtype not preserved with operations on sparse arrays
* `#16751 <https://github.com/scipy/scipy/issues/16751>`__: BUG: \`stats.fit\` on \`boltzmann\` expects \`bound\` for \`lambda\`,...
* `#17012 <https://github.com/scipy/scipy/issues/17012>`__: BUG: Small oversight in sparse.linalg.lsmr?
* `#17020 <https://github.com/scipy/scipy/issues/17020>`__: BUG: Build failure due to problems with shebang line in cythoner.py
* `#17088 <https://github.com/scipy/scipy/issues/17088>`__: BUG: stats.rayleigh.fit: returns \`loc\` that is inconsistent...
* `#17104 <https://github.com/scipy/scipy/issues/17104>`__: BUG? Incorrect branch in \`LAMV\` / \`_specfunc.lamv\`
* `#17196 <https://github.com/scipy/scipy/issues/17196>`__: DOC: keepdims in stats.mode is incorrectly documented


Pull requests for 1.9.3
-----------------------

* `#5288 <https://github.com/scipy/scipy/pull/5288>`__: BUG: multivariate_normal returns a pdf for values outside its...
* `#13322 <https://github.com/scipy/scipy/pull/13322>`__: Bug: setting iprint=0 hides all output from fmin_l_bfgs_b, but...
* `#13349 <https://github.com/scipy/scipy/pull/13349>`__: BUG: stats: Reformulate loggamma._rvs to handle c << 1.
* `#13411 <https://github.com/scipy/scipy/pull/13411>`__: BUG: fix out-of-bound evaluations in optimize.minimize, powell...
* `#15363 <https://github.com/scipy/scipy/pull/15363>`__: BUG: fix powell evaluated outside limits
* `#15381 <https://github.com/scipy/scipy/pull/15381>`__: BUG: fix stats.rv_histogram for non-uniform bins
* `#16212 <https://github.com/scipy/scipy/pull/16212>`__: stats.mood: correct for when ties are present
* `#16288 <https://github.com/scipy/scipy/pull/16288>`__: BUG: fix a crash in \`fpknot\`
* `#16318 <https://github.com/scipy/scipy/pull/16318>`__: MAINT: stats: fix _contains_nan on Pandas Series
* `#16460 <https://github.com/scipy/scipy/pull/16460>`__: Fix ttest permutations
* `#16506 <https://github.com/scipy/scipy/pull/16506>`__: MAINT: fix SHGO extra arguments
* `#16521 <https://github.com/scipy/scipy/pull/16521>`__: BUG: Fix error in error message for incorrect sample dimension...
* `#16525 <https://github.com/scipy/scipy/pull/16525>`__: MAINT: stats.ks_2samp: always emit warning when exact method...
* `#16528 <https://github.com/scipy/scipy/pull/16528>`__: BUG: fix syevr series segfault by explicitly specifying operator...
* `#16562 <https://github.com/scipy/scipy/pull/16562>`__: BUG: optimize: Fix differential_evolution error message.
* `#16573 <https://github.com/scipy/scipy/pull/16573>`__: FIX: \`odr.Model\` error with default \`meta\` value
* `#16588 <https://github.com/scipy/scipy/pull/16588>`__: FIX: stats: ignore divide-by-zero warnings from Boost binom impl
* `#16590 <https://github.com/scipy/scipy/pull/16590>`__: MAINT: stats.vonmises: wrap rvs to -pi, pi interval
* `#16630 <https://github.com/scipy/scipy/pull/16630>`__: BUG: eps param no effect fixed
* `#16645 <https://github.com/scipy/scipy/pull/16645>`__: MAINT: Ensure Pythran input for lombscargle are contiguous
* `#16649 <https://github.com/scipy/scipy/pull/16649>`__: Detect integer overflow in bivariate splines in fitpackmodule.c,...
* `#16657 <https://github.com/scipy/scipy/pull/16657>`__: BUG: sparse: Fix indexing sparse matrix with empty index arguments.
* `#16669 <https://github.com/scipy/scipy/pull/16669>`__: FIX: spurious divide error with \`gmean\`
* `#16701 <https://github.com/scipy/scipy/pull/16701>`__: BUG: fix mutable data types as default arguments in \`ord.{Data,RealData}\`
* `#16752 <https://github.com/scipy/scipy/pull/16752>`__: MAINT: stats.boltzmann: correct _shape_info typo
* `#16780 <https://github.com/scipy/scipy/pull/16780>`__: BUG: interpolate: sanity check x and y in make_interp_spline(x,...
* `#16836 <https://github.com/scipy/scipy/pull/16836>`__: MAINT: avoid \`func_data\`, it conflicts with system header on...
* `#16872 <https://github.com/scipy/scipy/pull/16872>`__: BUG: interpolate: work array sizes for RectSphereBivariateSpline
* `#16965 <https://github.com/scipy/scipy/pull/16965>`__: BUG: linalg: Fix the XSLOW test test_sgesdd_lwork_bug_workaround()
* `#17043 <https://github.com/scipy/scipy/pull/17043>`__: MAINT: fix small LSMR problem
* `#17090 <https://github.com/scipy/scipy/pull/17090>`__: MAINT: stats.rayleigh: enforce constraint on location
* `#17105 <https://github.com/scipy/scipy/pull/17105>`__: FIX: special: use intended branching for \`lamv\` implementation
* `#17166 <https://github.com/scipy/scipy/pull/17166>`__: MAINT: stats.rv_discrete.pmf: should be zero at non-integer argument
* `#17176 <https://github.com/scipy/scipy/pull/17176>`__: REL: Prep for SciPy 1.9.3
* `#17190 <https://github.com/scipy/scipy/pull/17190>`__: BUG: special: Fix two XSLOW test failures.
* `#17193 <https://github.com/scipy/scipy/pull/17193>`__: MAINT: update meson.build to make it work on IBM i system
* `#17200 <https://github.com/scipy/scipy/pull/17200>`__: BLD: fix issue with incomplete threads dependency handling
* `#17204 <https://github.com/scipy/scipy/pull/17204>`__: Keepdims incorrectly documneted fix
* `#17209 <https://github.com/scipy/scipy/pull/17209>`__: MAINT: Handle numpy's deprecation of accepting out-of-bound integers.
* `#17210 <https://github.com/scipy/scipy/pull/17210>`__: BLD: fix invalid shebang for build helper script
8 changes: 8 additions & 0 deletions meson.build
View file
Expand Up @@ -45,6 +45,14 @@ if m_dep.found()
add_project_link_arguments('-lm', language : 'c')
endif

if host_machine.system() == 'os400'
# IBM i system, needed to avoid build errors - see gh-17193
add_project_arguments('-D__STDC_FORMAT_MACROS', language : 'cpp')
add_project_link_arguments('-Wl,-bnotextro', language : 'c')
add_project_link_arguments('-Wl,-bnotextro', language : 'cpp')
add_project_link_arguments('-Wl,-bnotextro', language : 'fortran')
endif

# Adding at project level causes many spurious -lgfortran flags.
add_languages('fortran', native: false)
ff = meson.get_compiler('fortran')
Expand Down
2 changes: 1 addition & 1 deletion scipy/_build_utils/cythoner.py
View file
@@ -1,4 +1,4 @@
#!python3
#!/usr/bin/env python3
""" Scipy variant of Cython command
Cython, as applied to single pyx file.
Expand Down
8 changes: 2 additions & 6 deletions scipy/fft/_pocketfft/meson.build
View file
@@ -1,12 +1,8 @@
thread_dep = dependency('threads', required: false)

# mingw-w64 does not implement pthread_pfork, needed by pocket_fft
win_gcc = is_windows and meson.get_compiler('cpp').get_id() == 'gcc'

pocketfft_threads = []
fft_deps = [py3_dep]
if is_windows
if win_gcc
if is_mingw
# mingw-w64 does not implement pthread_pfork, needed by pocket_fft
# Disable threading completely, because of freezes using threading for
# mingw-w64 gcc: https://github.com/mreineck/pocketfft/issues/1
pocketfft_threads += ['-DPOCKETFFT_NO_MULTITHREADING']
Expand Down
15 changes: 8 additions & 7 deletions scipy/interpolate/_bsplines.py
View file
Expand Up @@ -1241,9 +1241,17 @@ def make_interp_spline(x, y, k=3, t=None, bc_type=None, axis=0,

y = np.moveaxis(y, axis, 0) # now internally interp axis is zero

# sanity check the input
if bc_type == 'periodic' and not np.allclose(y[0], y[-1], atol=1e-15):
raise ValueError("First and last points does not match while "
"periodic case expected")
if x.size != y.shape[0]:
raise ValueError('Shapes of x {} and y {} are incompatible'
.format(x.shape, y.shape))
if np.any(x[1:] == x[:-1]):
raise ValueError("Expect x to not have duplicates")
if x.ndim != 1 or np.any(x[1:] < x[:-1]):
raise ValueError("Expect x to be a 1D strictly increasing sequence.")

# special-case k=0 right away
if k == 0:
Expand Down Expand Up @@ -1289,17 +1297,10 @@ def make_interp_spline(x, y, k=3, t=None, bc_type=None, axis=0,

t = _as_float_array(t, check_finite)

if x.ndim != 1 or np.any(x[1:] < x[:-1]):
raise ValueError("Expect x to be a 1-D sorted array_like.")
if np.any(x[1:] == x[:-1]):
raise ValueError("Expect x to not have duplicates")
if k < 0:
raise ValueError("Expect non-negative k.")
if t.ndim != 1 or np.any(t[1:] < t[:-1]):
raise ValueError("Expect t to be a 1-D sorted array_like.")
if x.size != y.shape[0]:
raise ValueError('Shapes of x {} and y {} are incompatible'
.format(x.shape, y.shape))
if t.size < x.size + k + 1:
raise ValueError('Got %d knots, need at least %d.' %
(t.size, x.size + k + 1))
Expand Down
2 changes: 1 addition & 1 deletion scipy/interpolate/_rgi.py
View file
Expand Up @@ -576,7 +576,7 @@ def interpn(points, values, xi, method="linear", bounds_error=True,
if xi.shape[-1] != len(grid):
raise ValueError("The requested sample points xi have dimension "
"%d, but this RegularGridInterpolator has "
"dimension %d" % (xi.shape[1], len(grid)))
"dimension %d" % (xi.shape[-1], len(grid)))

if bounds_error:
for i, p in enumerate(xi.T):
Expand Down
11 changes: 9 additions & 2 deletions scipy/interpolate/fitpack/fpknot.f
View file
Expand Up @@ -21,7 +21,11 @@ recursive subroutine fpknot(x,m,t,n,fpint,nrdata,nrint,nest,
real*8 an,am,fpmax
integer ihalf,j,jbegin,jj,jk,jpoint,k,maxbeg,maxpt,
* next,nrx,number
c ..

c note: do not initialize on the same line to avoid saving between calls
logical iserr
iserr = .TRUE.

k = (n-nrint-1)/2
c search for knot interval t(number+k) <= x <= t(number+k+1) where
c fpint(number) is maximal on the condition that nrdata(number)
Expand All @@ -31,12 +35,15 @@ recursive subroutine fpknot(x,m,t,n,fpint,nrdata,nrint,nest,
do 20 j=1,nrint
jpoint = nrdata(j)
if(fpmax.ge.fpint(j) .or. jpoint.eq.0) go to 10
iserr = .FALSE.
fpmax = fpint(j)
number = j
maxpt = jpoint
maxbeg = jbegin
10 jbegin = jbegin+jpoint+1
20 continue
c error condition detected, go to exit
if(iserr) go to 50
c let coincide the new knot t(number+k+1) with a data point x(nrx)
c inside the old knot interval t(number+k) <= x <= t(number+k+1).
ihalf = maxpt/2+1
Expand All @@ -60,7 +67,7 @@ recursive subroutine fpknot(x,m,t,n,fpint,nrdata,nrint,nest,
fpint(next) = fpmax*an/am
jk = next+k
t(jk) = x(nrx)
n = n+1
50 n = n+1
nrint = nrint+1
return
end
32 changes: 22 additions & 10 deletions scipy/interpolate/src/_fitpackmodule.c
View file
@@ -1,8 +1,3 @@
/*
Multipack project.
This file is generated by setmodules.py. Do not modify it.
*/

#include <Python.h>
#include "numpy/arrayobject.h"

Expand All @@ -15,6 +10,7 @@ static PyObject *fitpack_error;

#define F_INT npy_int64
#define F_INT_NPY NPY_INT64
#define F_INT_MAX NPY_MAX_INT64

#if NPY_BITSOF_SHORT == 64
#define F_INT_PYFMT "h"
Expand All @@ -32,9 +28,20 @@ static PyObject *fitpack_error;

#else

#define F_INT int
#define F_INT_NPY NPY_INT
#define F_INT npy_int32
#define F_INT_NPY NPY_INT32
#define F_INT_MAX NPY_MAX_INT32
#if NPY_BITSOF_SHORT == 32
#define F_INT_PYFMT "h"
#elif NPY_BITSOF_INT == 32
#define F_INT_PYFMT "i"
#elif NPY_BITSOF_LONG == 32
#define F_INT_PYFMT "l"
#else
#error No compatible 32-bit integer size. \
Please contact NumPy maintainers and give detailed information about your \
compiler and platform
#endif

#endif

Expand Down Expand Up @@ -160,7 +167,7 @@ static PyObject *
fitpack_bispev(PyObject *dummy, PyObject *args)
{
F_INT nx, ny, kx, ky, mx, my, lwrk, *iwrk, kwrk, ier, lwa, nux, nuy;
npy_intp mxy;
npy_intp int_max, mxy;
double *tx, *ty, *c, *x, *y, *z, *wrk, *wa = NULL;
PyArrayObject *ap_x = NULL, *ap_y = NULL, *ap_z = NULL, *ap_tx = NULL;
PyArrayObject *ap_ty = NULL, *ap_c = NULL;
Expand Down Expand Up @@ -191,14 +198,19 @@ fitpack_bispev(PyObject *dummy, PyObject *args)
ny = PyArray_DIMS(ap_ty)[0];
mx = PyArray_DIMS(ap_x)[0];
my = PyArray_DIMS(ap_y)[0];
mxy = mx*my;
if (my != 0 && mxy/my != mx) {
/* Limit is min of (largest array size, max of Fortran int) */
int_max = (F_INT_MAX < NPY_MAX_INTP) ? F_INT_MAX : NPY_MAX_INTP;
/* v = int_max/my is largest integer multiple of `my` such that
v * my <= int_max
*/
if (my != 0 && int_max/my < mx) {
/* Integer overflow */
PyErr_Format(PyExc_RuntimeError,
"Cannot produce output of size %dx%d (size too large)",
mx, my);
goto fail;
}
mxy = (npy_intp)mx * (npy_intp)my;
ap_z = (PyArrayObject *)PyArray_SimpleNew(1,&mxy,NPY_DOUBLE);
if (ap_z == NULL) {
goto fail;
Expand Down
2 changes: 1 addition & 1 deletion scipy/interpolate/src/fitpack.pyf
View file
Expand Up @@ -685,7 +685,7 @@ static F_INT calc_regrid_lwrk(F_INT mx, F_INT my, F_INT kx, F_INT ky,
real*8 optional :: r1
real*8 optional,check(0.0<=s) :: s = 0.0
integer intent(hide),depend(mu),check(nuest>=8) &
:: nuest = mu+6
:: nuest = mu+6+2
integer intent(hide),depend(mv),check(nvest>=8) &
:: nvest = mv+7
integer intent(out) :: nu
Expand Down
23 changes: 23 additions & 0 deletions scipy/interpolate/tests/test_bsplines.py
View file
Expand Up @@ -943,6 +943,29 @@ def test_linear(self):
b = make_interp_spline(self.xx, self.yy, k=1, axis=-1)
assert_allclose(b(self.xx), self.yy, atol=1e-14, rtol=1e-14)

@pytest.mark.parametrize('k', [0, 1, 2, 3])
def test_incompatible_x_y(self, k):
x = [0, 1, 2, 3, 4, 5]
y = [0, 1, 2, 3, 4, 5, 6, 7]
with assert_raises(ValueError, match="Shapes of x"):
make_interp_spline(x, y, k=k)

@pytest.mark.parametrize('k', [0, 1, 2, 3])
def test_broken_x(self, k):
x = [0, 1, 1, 2, 3, 4] # duplicates
y = [0, 1, 2, 3, 4, 5]
with assert_raises(ValueError, match="x to not have duplicates"):
make_interp_spline(x, y, k=k)

x = [0, 2, 1, 3, 4, 5] # unsorted
with assert_raises(ValueError, match="Expect x to be a 1D strictly"):
make_interp_spline(x, y, k=k)

x = [0, 1, 2, 3, 4, 5]
x = np.asarray(x).reshape((1, -1)) # 1D
with assert_raises(ValueError, match="Expect x to be a 1D strictly"):
make_interp_spline(x, y, k=k)

def test_not_a_knot(self):
for k in [3, 5]:
b = make_interp_spline(self.xx, self.yy, k)
Expand Down
41 changes: 41 additions & 0 deletions scipy/interpolate/tests/test_fitpack2.py
View file
Expand Up @@ -325,6 +325,33 @@ def test_array_like_input(self):
assert_allclose(spl1([0.1, 0.5, 0.9, 0.99]),
spl2([0.1, 0.5, 0.9, 0.99]))

def test_fpknot_oob_crash(self):
# https://github.com/scipy/scipy/issues/3691
x = range(109)
y = [0., 0., 0., 0., 0., 10.9, 0., 11., 0.,
0., 0., 10.9, 0., 0., 0., 0., 0., 0.,
10.9, 0., 0., 0., 11., 0., 0., 0., 10.9,
0., 0., 0., 10.5, 0., 0., 0., 10.7, 0.,
0., 0., 11., 0., 0., 0., 0., 0., 0.,
10.9, 0., 0., 10.7, 0., 0., 0., 10.6, 0.,
0., 0., 10.5, 0., 0., 10.7, 0., 0., 10.5,
0., 0., 11.5, 0., 0., 0., 10.7, 0., 0.,
10.7, 0., 0., 10.9, 0., 0., 10.8, 0., 0.,
0., 10.7, 0., 0., 10.6, 0., 0., 0., 10.4,
0., 0., 10.6, 0., 0., 10.5, 0., 0., 0.,
10.7, 0., 0., 0., 10.4, 0., 0., 0., 10.8, 0.]
with suppress_warnings() as sup:
r = sup.record(
UserWarning,
r"""
The maximal number of iterations maxit \(set to 20 by the program\)
allowed for finding a smoothing spline with fp=s has been reached: s
too small.
There is an approximation returned but the corresponding weighted sum
of squared residuals does not satisfy the condition abs\(fp-s\)/s < tol.""")
UnivariateSpline(x, y, k=1)
assert_equal(len(r), 1)


class TestLSQBivariateSpline:
# NOTE: The systems in this test class are rank-deficient
Expand Down Expand Up @@ -1209,6 +1236,20 @@ def test_negative_evaluation(self):
[-49.0625, -46.54315]])
assert_array_almost_equal(data_interp, ans)

def test_pole_continuity_gh_14591(self):
# regression test for https://github.com/scipy/scipy/issues/14591
# with pole_continuty=(True, True), the internal work array size
# was too small, leading to a FITPACK data validation error.

# The reproducer in gh-14591 was using a NetCDF4 file with
# 361x507 arrays, so here we trivialize array sizes to a minimum
# which still demonstrates the issue.
u = np.arange(1, 10) * np.pi / 10
v = np.arange(1, 10) * np.pi / 10
r = np.zeros((9, 9))
for p in [(True, True), (True, False), (False, False)]:
RectSphereBivariateSpline(u, v, r, s=0, pole_continuity=p)


def _numdiff_2d(func, x, y, dx=0, dy=0, eps=1e-8):
if dx == 0 and dy == 0:
Expand Down

0 comments on commit 25e6b90

Please sign in to comment.