Replies: 1 comment
-
Unfortunately optuna See #497 How to do multithreading: https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Expected behavior
While training CatBoost model using Optuna, I am trying to make so that all my CPUs are used. But unfortunately (see the screenshot below) not even half of CPUs are used.
My question is: why Optuna cannot load all CPUs? What is the problem?
Code:
What else I tried to modify in my code:
as it was suggested here #2202 , but that did not help either.
Note: I also thought that postgresql and my harddrive may be the factor of slowdown, but this is not so - the diskgraph shows it is used on the speed of 1-2 MB/sec which is far below its possibility, and besides, I have a plenty of hardware to process these requests.
Environment
my requirements.txt (with which I've built the environment prior to installing optuna) was this:
numpy==1.23.5
pandas==1.5.3
pandas_datareader==0.10.0
SQLAlchemy==1.4.49
pandasql==0.7.3
seaborn==0.12.2
redshift_connector==2.0.911
apache-airflow==2.5.3
psycopg2-binary==2.9.7
tsfresh==0.20.1
xgboost==1.7.6
lightgbm==4.0.0
prophet==1.1.4
catboost==1.2
pymc==5.6.0
arviz==0.15.1
shap==0.42.1
ipython==8.10.0
ipykernel==6.19.2
ipywidgets==7.6.5
ipynb-py-convert==0.4.6
notebook==6.5.2
jupyter_contrib_nbextensions==0.7.0
jupyter_client==7.4.4
tornado==6.2
keyring==23.4.0
mlflow==2.5.0
coloredlogs==15.0.1
tenacity==8.0.1
lazyprofiler==0.1.1
boto3==1.28.29
smart_open==6.3.0
Error messages, stack traces, or logs
Steps to reproduce
(source code provided above)
Additional context (optional)
No response
Beta Was this translation helpful? Give feedback.
All reactions