MAINT: final backports for 1.9.0

.mailmap

@@ -18,6 +18,8 @@
 @endolith <endolith@gmail.com> endolith <endolith@gmail.com>
 @FormerPhysicist <Former@physicist.net> FormerPhysicist <Former@physicist.net>
 @gaulinmp <gaulinmp+git@gmail.com> Mac <gaulinmp+git@gmail.com>
+@h-vetinari <h.vetinari@gmx.com> h-vetinari <h.vetinari@gmx.com>
+@h-vetinari <h.vetinari@gmx.com> H. Vetinari <h.vetinari@gmx.com>
 @ksemb <ksms@gmx.de> ksemb <ksms@gmx.de>
 @kshitij12345 <kshitijkalambarkar@gmail.com> kshitij12345 <kshitijkalambarkar@gmail.com>
 @luzpaz <luzpaz@users.noreply.github.com> Unknown <kunda@scribus.net>
@@ -443,6 +445,7 @@ Shivnaren Srinivasan <shivnaren@gmail.com> srinivasan <shivnaren@gmail.com>
 Skipper Seabold <jsseabold@gmail.com> skip <skip@localhost>
 Shinya SUZUKI <sshinya@bio.titech.ac.jp> Shinya SUZUKI <minasitawakou@gmail.com>
 Smit Lunagariya <55887635+Smit-create@users.noreply.github.com> Smit-create <55887635+Smit-create@users.noreply.github.com>
+Smit Lunagariya <smitlunagariya.mat18@itbhu.ac.in> Smit-create <smitlunagariya.mat18@itbhu.ac.in>
 Sourav Singh <souravsingh@users.noreply.github.com> Sourav Singh <4314261+souravsingh@users.noreply.github.com>
 Srikiran <srikiran@dhcp-v233-179.pv.reshsg.uci.edu> sriki18 <sriki18@users.noreply.github.com>
 Stefan Endres <stefan.c.endres@gmail.com> stefan-endres <stefan.c.endres@gmail.com>

doc/release/1.9.0-notes.rst

@@ -2,8 +2,6 @@
 SciPy 1.9.0 Release Notes
 ==========================
 
-.. note:: Scipy 1.9.0 is not released yet!
-
 .. contents::
 
 SciPy 1.9.0 is the culmination of 6 months of hard work. It contains
@@ -403,6 +401,7 @@ Authors
 *******
 
 * endolith (12)
+* h-vetinari (11)
 * Caio Agiani (2) +
 * Emmy Albert (1) +
 * Joseph Albert (1)
@@ -457,11 +456,10 @@ Authors
 * Isuru Fernando (3)
 * Joseph Fox-Rabinovitz (1)
 * Ryan Gibson (4) +
-* Ralf Gommers (323)
+* Ralf Gommers (327)
 * Srinivas Gorur-Shandilya (1) +
 * Alex Griffing (2)
-* h-vetinari (5)
-* Matt Haberland (459)
+* Matt Haberland (461)
 * Tristan Hearn (1) +
 * Jonathan Helgert (1) +
 * Samuel Hinton (1) +
@@ -512,8 +510,8 @@ Authors
 * Amit Portnoy (1) +
 * Quentin Barthélemy (9)
 * Patrick N. Raanes (1) +
-* Tyler Reddy (174)
-* Pamphile Roy (197)
+* Tyler Reddy (185)
+* Pamphile Roy (199)
 * Vivek Roy (2) +
 * sabonerune (1) +
 * Niyas Sait (2) +
@@ -541,11 +539,10 @@ Authors
 * Will Tirone (2)
 * Bas van Beek (7)
 * Dhruv Vats (1)
-* H. Vetinari (6)
 * Arthur Volant (1)
 * Samuel Wallan (5)
 * Stefan van der Walt (8)
-* Warren Weckesser (83)
+* Warren Weckesser (84)
 * Anreas Weh (1)
 * Nils Werner (1)
 * Aviv Yaish (1) +
@@ -558,7 +555,7 @@ Authors
 * Pavel Zun (1) +
 * David Zwicker (1) +
 
-A total of 155 people contributed to this release.
+A total of 154 people contributed to this release.
 People with a "+" by their names contributed a patch for the first time.
 This list of names is automatically generated, and may not be fully complete.
 
@@ -587,6 +584,7 @@ Issues closed for 1.9.0
 * `#5890 <https://github.com/scipy/scipy/issues/5890>`__: Undefined behavior when using scipy.interpolate.RegularGridInterpolator...
 * `#5982 <https://github.com/scipy/scipy/issues/5982>`__: Keyword collision in scipy.stats.levy_stable.interval
 * `#6472 <https://github.com/scipy/scipy/issues/6472>`__: scipy.stats.invwishart does not check if scale matrix is symmetric
+* `#6551 <https://github.com/scipy/scipy/issues/6551>`__: BUG: stats: inconsistency in docs and behavior of gmean and hmean
 * `#6624 <https://github.com/scipy/scipy/issues/6624>`__: incorrect handling of nan by RegularGridInterpolator
 * `#6882 <https://github.com/scipy/scipy/issues/6882>`__: Certain recursive scipy.integrate.quad (e.g. dblquad and nquad)...
 * `#7469 <https://github.com/scipy/scipy/issues/7469>`__: Misleading interp2d documentation
@@ -788,6 +786,8 @@ Issues closed for 1.9.0
 * `#16540 <https://github.com/scipy/scipy/issues/16540>`__: BLD: meson 0.63.0 and new CI testing failures on Linux
 * `#16555 <https://github.com/scipy/scipy/issues/16555>`__: Building 1.9.x branch from source requires fix in meson-python...
 * `#16609 <https://github.com/scipy/scipy/issues/16609>`__: BUG: \`scipy.optimize.linprog\` reports optimal for trivially...
+* `#16681 <https://github.com/scipy/scipy/issues/16681>`__: BUG: linprog integrality only accepts list, not array
+* `#16718 <https://github.com/scipy/scipy/issues/16718>`__: BUG: memoryview error with Cython 0.29.31
 
 ***********************
 Pull requests for 1.9.0
@@ -1225,6 +1225,7 @@ Pull requests for 1.9.0
 * `#16553 <https://github.com/scipy/scipy/pull/16553>`__: BLD: raise an error with clear message for too-new Python version
 * `#16556 <https://github.com/scipy/scipy/pull/16556>`__: DOC: small tweaks to 1.9.0 release notes
 * `#16563 <https://github.com/scipy/scipy/pull/16563>`__: DOC: Reflect MSVC minimum toolchain requirement
+* `#16570 <https://github.com/scipy/scipy/pull/16570>`__: MAINT: backports before 1.9.0rc3
 * `#16572 <https://github.com/scipy/scipy/pull/16572>`__: MAINT: update bundled licenses for removal of scipy-sphinx-theme
 * `#16581 <https://github.com/scipy/scipy/pull/16581>`__: MAINT: stats: fix skew/kurtosis empty 1d input
 * `#16586 <https://github.com/scipy/scipy/pull/16586>`__: MAINT: stats.truncnorm: improve CDF accuracy/speed
@@ -1235,3 +1236,7 @@ Pull requests for 1.9.0
 * `#16638 <https://github.com/scipy/scipy/pull/16638>`__: DOC: update docs on building with Meson
 * `#16664 <https://github.com/scipy/scipy/pull/16664>`__: MAINT: stats._axis_nan_policy: preserve dtype of masked arrays...
 * `#16671 <https://github.com/scipy/scipy/pull/16671>`__: BLD: update \`meson\` and \`meson-python\` versions for 1.9.0...
+* `#16684 <https://github.com/scipy/scipy/pull/16684>`__: MAINT: optimize.linprog: ensure integrality can be an array
+* `#16688 <https://github.com/scipy/scipy/pull/16688>`__: DOC: a few mailmap updates
+* `#16719 <https://github.com/scipy/scipy/pull/16719>`__: MAINT: stats: Work around Cython bug.
+* `#16721 <https://github.com/scipy/scipy/pull/16721>`__: MAINT: stats.monte_carlo_test: used biased estimate of p-value

meson.build

@@ -4,7 +4,7 @@ project(
   # Note that the git commit hash cannot be added dynamically here (it is added
   # in the dynamically generated and installed `scipy/version.py` though - see
   # tools/version_utils.py
-  version: '1.9.0rc3',
+  version: '1.9.0',
   license: 'BSD-3',
   meson_version: '>= 0.62.2',
   default_options: [

scipy/optimize/_linprog.py

@@ -581,7 +581,7 @@ def linprog(c, A_ub=None, b_ub=None, A_eq=None, b_eq=None,
         warning_message = "x0 is used only when method is 'revised simplex'. "
         warn(warning_message, OptimizeWarning)
 
-    if integrality and not meth == "highs":
+    if np.any(integrality) and not meth == "highs":
         integrality = None
         warning_message = ("Only `method='highs'` supports integer "
                            "constraints. Ignoring `integrality`.")

scipy/optimize/tests/test_linprog.py

@@ -2230,12 +2230,13 @@ def test_mip2(self):
         # source: slide 5,
         # https://www.cs.upc.edu/~erodri/webpage/cps/theory/lp/milp/slides.pdf
 
+        # use all array inputs to test gh-16681 (integrality couldn't be array)
         A_ub = np.array([[2, -2], [-8, 10]])
         b_ub = np.array([-1, 13])
         c = -np.array([1, 1])
 
-        bounds = [(0, np.inf)] * len(c)
-        integrality = [1] * len(c)
+        bounds = np.array([(0, np.inf)] * len(c))
+        integrality = np.ones_like(c)
 
         res = linprog(c=c, A_ub=A_ub, b_ub=b_ub, bounds=bounds,
                       method=self.method, integrality=integrality)

scipy/stats/_resampling.py

@@ -607,6 +607,13 @@ def monte_carlo_test(sample, rvs, statistic, *, vectorized=False,
     null_distribution : ndarray
         The values of the test statistic generated under the null hypothesis.
 
+    References
+    ----------
+
+    .. [1] B. Phipson and G. K. Smyth. "Permutation P-values Should Never Be
+       Zero: Calculating Exact P-values When Permutations Are Randomly Drawn."
+       Statistical Applications in Genetics and Molecular Biology 9.1 (2010).
+
     Examples
     --------
 
@@ -642,7 +649,7 @@ def monte_carlo_test(sample, rvs, statistic, *, vectorized=False,
     >>> print(res.statistic)
     0.12457412450240658
     >>> print(res.pvalue)
-    0.701070107010701
+    0.7012
 
     The probability of obtaining a test statistic less than or equal to the
     observed value under the null hypothesis is ~70%. This is greater than
@@ -695,12 +702,12 @@ def monte_carlo_test(sample, rvs, statistic, *, vectorized=False,
 
     def less(null_distribution, observed):
         cmps = null_distribution <= observed
-        pvalues = cmps.sum(axis=0) / n_resamples
+        pvalues = (cmps.sum(axis=0) + 1) / (n_resamples + 1)  # see [1]
         return pvalues
 
     def greater(null_distribution, observed):
         cmps = null_distribution >= observed
-        pvalues = cmps.sum(axis=0) / n_resamples
+        pvalues = (cmps.sum(axis=0) + 1) / (n_resamples + 1)  # see [1]
         return pvalues
 
     def two_sided(null_distribution, observed):

scipy/stats/_stats.pyx

@@ -171,6 +171,11 @@ def _toint64(x):
 @cython.wraparound(False)
 @cython.boundscheck(False)
 def _weightedrankedtau(ordered[:] x, ordered[:] y, intp_t[:] rank, weigher, bool additive):
+    # y_local and rank_local (declared below) are a work-around for a Cython
+    # bug; see gh-16718.  When we can require Cython 3.0, y_local and
+    # rank_local can be removed, and the closure weigh() can refer directly
+    # to y and rank.
+    cdef ordered[:] y_local = y
     cdef intp_t i, first
     cdef float64_t t, u, v, w, s, sq
     cdef int64_t n = np.int64(len(x))
@@ -189,6 +194,8 @@ def _weightedrankedtau(ordered[:] x, ordered[:] y, intp_t[:] rank, weigher, bool
         rank[...] = perm[::-1]
         _invert_in_place(rank)
 
+    cdef intp_t[:] rank_local = rank
+
     # weigh joint ties
     first = 0
     t = 0
@@ -237,28 +244,28 @@ def _weightedrankedtau(ordered[:] x, ordered[:] y, intp_t[:] rank, weigher, bool
         cdef float64_t weight, residual
 
         if length == 1:
-            return weigher(rank[perm[offset]])
+            return weigher(rank_local[perm[offset]])
         length0 = length // 2
         length1 = length - length0
         middle = offset + length0
         residual = weigh(offset, length0)
         weight = weigh(middle, length1) + residual
-        if y[perm[middle - 1]] < y[perm[middle]]:
+        if y_local[perm[middle - 1]] < y_local[perm[middle]]:
             return weight
 
         # merging
         i = j = k = 0
 
         while j < length0 and k < length1:
-            if y[perm[offset + j]] <= y[perm[middle + k]]:
+            if y_local[perm[offset + j]] <= y_local[perm[middle + k]]:
                 temp[i] = perm[offset + j]
-                residual -= weigher(rank[temp[i]])
+                residual -= weigher(rank_local[temp[i]])
                 j += 1
             else:
                 temp[i] = perm[middle + k]
-                exchanges_weight[0] += weigher(rank[temp[i]]) * (
+                exchanges_weight[0] += weigher(rank_local[temp[i]]) * (
                     length0 - j) + residual if additive else weigher(
-                    rank[temp[i]]) * residual
+                    rank_local[temp[i]]) * residual
                 k += 1
             i += 1
 

scipy/stats/tests/test_resampling.py

@@ -760,6 +760,15 @@ def statistic1d(x):
         assert_allclose(res.statistic, expected_stat)
         assert_allclose(res.pvalue, expected_p, atol=2*self.atol)
 
+    def test_p_never_zero(self):
+        # Use biased estimate of p-value to ensure that p-value is never zero
+        # per monte_carlo_test reference [1]
+        rng = np.random.default_rng(2190176673029737545)
+        x = np.zeros(100)
+        res = monte_carlo_test(x, rng.random, np.mean,
+                               vectorized=True, alternative='less')
+        assert res.pvalue == 0.0001
+
 
 class TestPermutationTest:
 

tools/version_utils.py

@@ -6,9 +6,9 @@
 MAJOR = 1
 MINOR = 9
 MICRO = 0
-ISRELEASED = True
+ISRELEASED = False
 IS_RELEASE_BRANCH = True
-VERSION = '%d.%d.%drc3' % (MAJOR, MINOR, MICRO)
+VERSION = '%d.%d.%d' % (MAJOR, MINOR, MICRO)
 
 
 def get_version_info(source_root):