[Bug?] hydra.runtime.choices seems not to reflect current choice during multi-run

[kanada0727]

🐛 Bug

Description

config value ${hydra:runtime.choices} seems not to reflect current choice under multi-run settings.

Checklist

• I checked on the latest version of Hydra
• I created a minimal repro (See this for tips).

To reproduce

** Minimal Code/Config snippet to reproduce **
For simple reproduction, I prepared a set of files with the following directory structure.

src/
├── config
│   ├── config.yaml
│   └── option
│       ├── option1.yaml
│       └── option2.yaml
└── main.py

The contents of the files are described below.

• src/config/config.yaml

defaults:
  - option: option1
  - _self_
selected_option: ${hydra:runtime.choices.option}
overriden_values: ${hydra:overrides}

• src/config/option/option1.yaml

value: value1

• src/config/option/option2.yaml

value: value2

• src/main.py

import hydra

@hydra.main(config_path="config", config_name="config")
def main(config):
    print(f"option name: {config.selected_option}.")
    print(f"option value: {config.option.value}")
    print(f"overriden values: {config.overriden_values}")

if __name__ == "__main__":
    main()

** Stack trace/error message **
I ran the following multi-run script and got the following logs.

python src/main.py -m option=option2,option1
[2021-11-10 09:04:48,546][HYDRA] Launching 2 jobs locally
[2021-11-10 09:04:48,546][HYDRA]        #0 : option=option2
option name: option1.
option value: value2
overriden values: {'hydra': [], 'task': ['option=option2']}
[2021-11-10 09:04:48,605][HYDRA]        #1 : option=option1
option name: option1.
option value: value1
overriden values: {'hydra': [], 'task': ['option=option1']}

I expect the value of the config name (config.selected_options, retrieved by resolver ${hydra:runtime.choices.option}) to change with respect to overridden option value, as well as config.overridden_values does.
But this value seems not to be changed.

Expected Behavior

python src/main.py -m option=option2,option1
[2021-11-10 09:04:48,546][HYDRA] Launching 2 jobs locally
[2021-11-10 09:04:48,546][HYDRA]        #0 : option=option2
option name: option2.  ## <-- this value is expected to be changed.
option value: value2
overriden values: {'hydra': [], 'task': ['option=option2']}
[2021-11-10 09:04:48,605][HYDRA]        #1 : option=option1
option name: option1.
option value: value1
overriden values: {'hydra': [], 'task': ['option=option1']}

System information

• Hydra Version : 1.1.1
• Python version : 3.8.10
• Virtual environment type and version : pipenv, version 2020.11.15
• Operating system : Ubuntu 20.04.2 LTS

Additional context

Add any other context about the problem here.

[Jasha10]

Hi @kanada0727,

Thank you for the report! I was able to reproduce your example on the main branch.
I agree with you that the current behavior is undesirable.