New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
support Python3.11 #2418
support Python3.11 #2418
Conversation
When trying to import hydra in Python 3.11, it throws following error: ```python File "/home/user/projects/iterative/hydra/hydra/conf/__init__.py", line 75, in JobConf @DataClass ^^^^^^^^^ File "/home/user/.pyenv/versions/3.11-dev/lib/python3.11/dataclasses.py", line 1221, in dataclass return wrap(cls) ^^^^^^^^^ File "/home/user/.pyenv/versions/3.11-dev/lib/python3.11/dataclasses.py", line 1211, in wrap return _process_class(cls, init, repr, eq, order, unsafe_hash, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/.pyenv/versions/3.11-dev/lib/python3.11/dataclasses.py", line 959, in _process_class cls_fields.append(_get_field(cls, name, type, kw_only)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/.pyenv/versions/3.11-dev/lib/python3.11/dataclasses.py", line 816, in _get_field raise ValueError(f'mutable default {type(f.default)} for field ' ValueError: mutable default <class 'hydra.conf.JobConf.JobConfig.OverrideDirname'> for field override_dirname is not allowed: use default_factory ``` This is due to a recent change in dataclasses in Python 3.11 that disallows mutable to be set as a default value. This is done via check for hashability. See python/cpython#88840. Note that I am trying to add support for Python 3.11 to hydra. This seems like a major blocker. For reproducibility, I used the following command to search for these: ```console rg ".*: .* = [A-Z].*\(.*\)" ```
Hi @skshetry, thanks for working on Python3.11 support! Using The only other option I can think of is to use the decorator We'll eventually need to make the same change for the OmegaConf codebase as for Hydra. |
It might be a good idea to create some regression tests like the following (to make sure that OmegaConf and Hydra can still handle old dataclass pattern when python < 3.11): if python_version < (3, 11):
@dataclass
class X:
y: Y = Y() |
I am hesitant to test that, as it is |
When turning a dataclass into a DictConfig, the if python_version < (3, 11):
@dataclass
class X:
y: Y = Y()
# Will OmegaConf choke on dataclasses defined using
# the old pattern with default values? Currently this works:
cfg = OmegaConf.create(X)
# We should test it to make sure it keeps working (for python 3.6-3.10). I'm worried that future refactoring of |
tests/test_hydra.py
Outdated
|
||
|
||
@mark.skipif(sys.version_info > (3, 11), reason="mutable defaults not allowed in 3.11") | ||
def test_dataclass_with_assigned_mutable_default(): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was not sure where to add this test. Let me know where it fits.
Thanks @skshetry. I'm working on a python3.11 update for omegaconf. I'm planning to revisit this PR after the OmegaConf support is finished. Edit: here's a link to the OmegaConf PR: omry/omegaconf#1032 |
ping @Jasha10 |
Thanks @skshetry |
FYI I've removed |
Motivation
When trying to import hydra in Python 3.11, it throws following error:
This is due to a recent change in dataclasses in Python 3.11 that disallows mutable to be set as a default value. This is done via check for hashability.
See python/cpython#88840. Note that I am trying to add support for Python 3.11 to hydra. This seems like a major blocker.
For reproducibility, I used the following command to search for these:
rg ".*: .* = [A-Z].*\(.*\)"
Have you read the Contributing Guidelines on pull requests?
Yes
Test Plan
If all tests pass, it should be good.
Related Issues and PRs