-
-
Notifications
You must be signed in to change notification settings - Fork 965
Optuna GSoC 2022
Optuna is an open source hyperparameter optimization framework to automate hyperparameter search. Optuna provides eager search spaces for automated search for optimal hyperparameters using Python conditionals, loops, and syntax, state-of-the-art algorithms to efficiently search large spaces and prune unpromising trials for faster results, and easy parallelization for hyperparameter searches over multiple threads or processes without modifying code.
Optuna is participating in GSoC 2022 as a member of NumFOCUS.
For coding on Optuna, a solid basis in Python coding will be required. Experience working with Git and github.com will also be useful. If you're interested in applying, we recommend you take a look at the Optuna github repository, and try your hand at some of the Contribution Welcome labelled issues. We will evaluate applications largely based on their contributions to Optuna and other open source projects.
To contact us, please email us at optuna@preferred.jp
Project title: The Next Generation of Hyperparameter Optimization Software
Project title: Maintainable Performance Benchmark Tooling for Optimization Algorithms
Feel free to come up with your own project that did not fall into any of the ones proposed above. We are keen to hear your ideas on how the Optuna ecosystem can be improved.
We are developing the next major version, V3 of the hyperparameter optimization software Optuna. The goal of Optuna V3 is to improve the stability of the software and deliver state-of-the-art algorithms and features to all users, while keeping the development policy open and involving the community. The development of Optuna V3 started last October, but there are still many important and challenging tasks left to be done, for example, improvement of data storage and core algorithms, review testing strategy, re-definition of exception/error specification. We would like you to work on the challenging tasks with the development of Optuna V3.
You will take part in reaching the above goal. The detailed development roadmap for Optuna V3 is available on the GitHub wiki. See here. Further details and progress statuses of each task are described in issues listed at this wiki. You will be dealing with one of the tasks that are not yet completed. Here are some important uncompleted tasks.
- Make a consistent policy for instantiation of storage classes and publicization of private storage accompanying it #2942
- Exception/error specification review #2978
- Change default sampling algorithm #2964
- Review core function testing strategy #2974
You are also free to take whatever tasks you like according to your own interests, or come up with your own.
You will work with the Optuna committers and contributors to develop key items to create the next major version of Optuna. The actual work requires advanced coding skills and communication skills to facilitate discussions. Also, each development item requires its own domain knowledge. For example, RDB, error handling, unit testing, and hyper-parameter optimization algorithms. It is of course welcome to have such knowledge beforehand, but it is also a great opportunity to acquire such knowledge through development.
Minimum
- Software development experience using Python
- High work morale and communication skills
Preferred
- Software development experience in teams
- Knowledge of RDB (MySQL, PostgreSQL, SQLite), error handling and unit-testing
- Knowledge/experience with Bayesian optimization, hyperparameter optimization and black-box optimization
@HideakiImamura, @nzw0301, @himkt
175h
Medium/High
Optuna includes a wide range of black-box optimization algorithms. Maintaining these algorithms is not trivial, because changes are oftentimes difficult to determine whether they are for the better. For instance, altering the default value of a constructor argument or some internal logic might degrade optimization performance without necessarily introducing an obvious bug. Even something as simple as refactoring may have similar consequences. These types of changes are difficult to address using unit tests. They cost both developers and reviewers a lot of time. Here, we would like to work on this problem.
In Optuna, we resort to using Kurobako. It is a command-line tool for benchmarking black-box optimization algorithms. It comes with a set of problems, allowing us to quickly compare different algorithms against those, helping one to determine whether a change should be adopted. Kurobako was initially developed by one of Optuna’s core maintainers, and is written in Rust.
You will make it easier to maintain Kurobako in the Optuna community. We suggest first rewriting Kurobako in Python, alternatively, redesigning it from the ground up. Further, the project may be extended to add new functionality or in some other way improve the user experience. You may also work on the integration with GitHub Actions and Optuna’s CI. Creative ideas are welcome.
You get to work with multiple programming languages and contribute to enhancing the overall experience for many Optuna developers. Development operations are critical for the smooth running of large open source projects, and the skills you can learn in improving the operations of Optuna will be applicable to many other projects. Build relationships with coders in a top tier startup in Japan.
Minimum
- Software development experience using Python
- High work morale and communication skills
- Basic understanding of Rust
Preferred
- Software development experience in teams
- Knowledge/experience with Bayesian optimization, hyperparameter optimization and black-box optimization, and benchmarking of such techniques
175h
Medium/High