Update to Python-3.8 #254

qbarthelemy · 2023-06-27T16:10:41Z

Update to Python-3.8

Update code to support SciPy 1.11.0 and scikit-learn 1.3.0.

Update readme for version 0.5, see #253.

qbarthelemy · 2023-06-27T16:41:00Z

scipy.stats.mode for non numeric data is deprecated from SciPy 1.9.0 and is removed in SciPy 1.11.0.

Tests are broken, because transfer learning module uses non numeric extended labels.
See encode_domains documentation: Note that if the classes were integers at first, they will be converted to strings.

@plcrodrigues , any idea?

qbarthelemy · 2023-06-28T05:40:04Z

I solve the problem, replacing scipy.stats.mode by np.unique, which supports non numeric data.

plcrodrigues · 2023-06-28T08:18:46Z

@plcrodrigues , any idea?

This is indeed annoying but doesn't sound too hard to bypass with your np.unique approach (after corrections).

In any case, we should soon take some time to make use of the new sample-props feature from sklearn -- PR #24027 -- in the transfer learning module of pyRiemann. This would avoid recurring to string in the labels.

agramfort · 2023-06-28T09:11:03Z

pyriemann/classification.py

@@ -524,8 +523,8 @@ def predict(self, covtest):
        """
        dist = self._predict_distances(covtest)
        neighbors_classes = self.classmeans_[np.argsort(dist)]
-        out, _ = stats.mode(neighbors_classes[:, 0:self.n_neighbors], axis=1)
-        return out.ravel()
+        out = np.unique(neighbors_classes[:, 0:self.n_neighbors], axis=1)[:, 0]


I doubt this is correct

you would need to use return_counts and get the entry in out that has the biggest count

You are right. Solution for ndarray is a little bit more complicated
https://stackoverflow.com/a/12300214

I will try to find the simplest solution for 2D case.

plcrodrigues · 2023-06-28T08:13:44Z

pyriemann/classification.py

@@ -524,8 +523,8 @@ def predict(self, covtest):
        """
        dist = self._predict_distances(covtest)
        neighbors_classes = self.classmeans_[np.argsort(dist)]
-        out, _ = stats.mode(neighbors_classes[:, 0:self.n_neighbors], axis=1)
-        return out.ravel()
+        out = np.unique(neighbors_classes[:, 0:self.n_neighbors], axis=1)[:, 0]


I'm not completely sure that this will work all the time. The KNearestNeighbor is based on the idea of checking which class is the most common among the K nearest neighbors of the test point, right? Are we sure that np.unique always returns the unique classes of the neighbors in a sorted way? This example makes me doubt it...

You should probably use the output argument unique_counts from np.unique to be sure...

… tests

qbarthelemy · 2023-06-30T16:04:28Z

Last test for Windows Python 3.7 which is OK installed joblib 1.2.0
https://github.com/pyRiemann/pyRiemann/actions/runs/5321719371/jobs/9637189077#step:5:24
5:10

Now, it is KO installing joblib 1.3.1
https://github.com/pyRiemann/pyRiemann/actions/runs/5415226192/jobs/9846294273?pr=254#step:5:24

I try to freeze joblib to 1.2.0, but now, all others tests for Python 3.8 and 3.9 are KO. Any idea?

qbarthelemy · 2023-07-03T11:01:29Z

Last errors are due to scikit-learn 1.3.0, adding a new parameter sample_weight in KMeans ._init_centroids.
However, this parameter is not checked, raising an error when used with default value None.
scikit-learn/scikit-learn#26755

qbarthelemy · 2023-07-03T11:10:36Z

Waiting scikit-learn bugfix, I decide to detect when _init_centroids must be used with a non default sample_weight.
I also clean code, removing support for version before scikit-learn 0.24.

lesteve · 2023-07-05T15:49:06Z

FYI I had a look and the joblib 1.3.1 issue is actually due to loky (default joblib parallel backend). The loky issue is tracked in joblib/loky#411.

Not entirely sure this is going to get fixed in loky. FYI scikit-learn requires Python 3.8+ since scikit-learn 1.1 (released May 2022), so a possible work-around for pyRiemann would be to drop Python 3.7 support in the 0.5 release.

qbarthelemy · 2023-08-07T07:55:14Z

Close-reopen for tests.

qbarthelemy · 2023-08-07T08:01:37Z

Documentation fails, it is probably related to #255. @agramfort, what do you think ?

EDIT: no, it is not related

agramfort

thx @qbarthelemy

qbarthelemy · 2023-08-09T10:21:53Z

For documentation build, I can't solve the dependency problem between sphinx and jinja2.

buid OK
https://github.com/pyRiemann/pyRiemann/actions/runs/5473515554/job/14820290670#step:4:147

build KO
https://github.com/pyRiemann/pyRiemann/actions/runs/5669963085/job/15363815095#step:4:147

build KO
https://github.com/pyRiemann/pyRiemann/actions/runs/5782496088/job/15669426717#step:4:147

build KO
https://github.com/pyRiemann/pyRiemann/actions/runs/5807755983/job/15743206407?pr=254#step:4:135

qbarthelemy · 2023-08-09T12:34:40Z

Doc is now OK!
I let you merge @agramfort , @plcrodrigues , @sylvchev

qbarthelemy added 3 commits June 27, 2023 18:09

update readme for release 0.5

52da888

enforce extended labels to be numeric for classifiers

d4a1bdf

remove pytest warning in test for mean

2bc215c

qbarthelemy added 2 commits June 27, 2023 18:57

fix some errors

6639a68

use np.unique instead of scipy.mode

b25e0e0

qbarthelemy added 2 commits June 28, 2023 07:47

remove ravel

66b0e8d

remove another deprecation warning

357e98c

agramfort reviewed Jun 28, 2023

View reviewed changes

plcrodrigues reviewed Jun 28, 2023

View reviewed changes

qbarthelemy added 2 commits June 29, 2023 19:49

add private function for mode, add tests, and remove warning for mean…

008db99

… tests

freeze joblib version to 1.2.0

50cbf79

qbarthelemy added 2 commits July 3, 2023 13:04

manage case when _init_centroids is used with sample_weight

4bbdb30

update order of imports

6a68a02

qbarthelemy changed the title ~~Update readme for release 0.5~~ Update code to support SciPy 1.11.0 and scikit-learn 1.3.0 Jul 3, 2023

jeremiedbb mentioned this pull request Jul 3, 2023

Error when KMeans ._init_centroids is used with sample_weight=None scikit-learn/scikit-learn#26755

Closed

qbarthelemy added 3 commits July 6, 2023 10:09

improve sklearn version checking

3d13b60

update to python 3.8 - 3.10

0f9a512

correct versions

7a29ac1

qbarthelemy changed the title ~~Update code to support SciPy 1.11.0 and scikit-learn 1.3.0~~ Update to Python-3.8 Jul 6, 2023

update whatsnew

60d28f6

qbarthelemy closed this Aug 7, 2023

qbarthelemy reopened this Aug 7, 2023

qbarthelemy mentioned this pull request Aug 7, 2023

[FIX] Change sum by mean when computing dispersion #258

Merged

agramfort approved these changes Aug 8, 2023

View reviewed changes

fix documentation build

9d404dd

add version min for sphinx

0388349

qbarthelemy mentioned this pull request Aug 11, 2023

speedup pairwise_distance #256

Merged

agramfort approved these changes Aug 11, 2023

View reviewed changes

agramfort merged commit aa8fee7 into pyRiemann:master Aug 11, 2023
10 checks passed

qbarthelemy deleted the release_05 branch August 11, 2023 10:33

qbarthelemy mentioned this pull request Sep 6, 2023

Python3.12: tests fail #262

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update to Python-3.8 #254

Update to Python-3.8 #254

qbarthelemy commented Jun 27, 2023 •

edited

qbarthelemy commented Jun 27, 2023 •

edited

qbarthelemy commented Jun 28, 2023

plcrodrigues commented Jun 28, 2023 •

edited

agramfort Jun 28, 2023

agramfort Jun 28, 2023

qbarthelemy Jun 29, 2023 •

edited

plcrodrigues Jun 28, 2023

plcrodrigues Jun 28, 2023

qbarthelemy commented Jun 30, 2023 •

edited

qbarthelemy commented Jul 3, 2023 •

edited

qbarthelemy commented Jul 3, 2023

lesteve commented Jul 5, 2023

qbarthelemy commented Aug 7, 2023

qbarthelemy commented Aug 7, 2023 •

edited

agramfort left a comment

qbarthelemy commented Aug 9, 2023 •

edited

qbarthelemy commented Aug 9, 2023

Update to Python-3.8 #254

Update to Python-3.8 #254

Conversation

qbarthelemy commented Jun 27, 2023 • edited

qbarthelemy commented Jun 27, 2023 • edited

qbarthelemy commented Jun 28, 2023

plcrodrigues commented Jun 28, 2023 • edited

agramfort Jun 28, 2023

Choose a reason for hiding this comment

agramfort Jun 28, 2023

Choose a reason for hiding this comment

qbarthelemy Jun 29, 2023 • edited

Choose a reason for hiding this comment

plcrodrigues Jun 28, 2023

Choose a reason for hiding this comment

plcrodrigues Jun 28, 2023

Choose a reason for hiding this comment

qbarthelemy commented Jun 30, 2023 • edited

qbarthelemy commented Jul 3, 2023 • edited

qbarthelemy commented Jul 3, 2023

lesteve commented Jul 5, 2023

qbarthelemy commented Aug 7, 2023

qbarthelemy commented Aug 7, 2023 • edited

agramfort left a comment

Choose a reason for hiding this comment

qbarthelemy commented Aug 9, 2023 • edited

qbarthelemy commented Aug 9, 2023

qbarthelemy commented Jun 27, 2023 •

edited

qbarthelemy commented Jun 27, 2023 •

edited

plcrodrigues commented Jun 28, 2023 •

edited

qbarthelemy Jun 29, 2023 •

edited

qbarthelemy commented Jun 30, 2023 •

edited

qbarthelemy commented Jul 3, 2023 •

edited

qbarthelemy commented Aug 7, 2023 •

edited

qbarthelemy commented Aug 9, 2023 •

edited