Handle custom project_dir

[Akasurde]

SUMMARY

Signed-off-by: Abhijeet Kasurde akasurde@redhat.com

ISSUE TYPE

• Bugfix Pull Request

COMPONENT NAME

ansible_runner/runner.py
test/fixtures/projects/project_dir_test/env/envvars
test/fixtures/projects/project_dir_test/inventory/inv_1
test/fixtures/projects/project_dir_test/my_project/debug.yml
test/integration/test_interface.py

[Akasurde]

Closing and re-opening to retrigger the CI.

[jkugler]

Could we also add a test to verify that it works when

1. there is no "project" directory and
2. you specify a custom project_dir (see comment on the original bug for details: project_dir does not work #1218 (comment))

[Akasurde]

Could we also add a test to verify that it works when

1. there is no "project" directory and

2. you specify a custom project_dir (see comment on the original bug for details: [project_dir does not work #1218 (comment)](https://github.com/ansible/ansible-runner/issues/1218#issuecomment-1462576456))

Yes sure. Thanks for the review.

[urbanchef]

If I understand it correctly, this change enforces project_dir to be under private_data_dir even if project_dir is set and is absolute path?

I believe project_dir should not be forced to be under private_data_dir, because then there is no way to work around issue #1143. Consider a case: I have a runner thread that monitors a queue periodically for playbooks to run. When there's a job submitted the runner thread creates a separate unique private_data_dir for each run as well as runs the playbook. This way project_dir remains unchanged, because main ansible playbook code base resides there. The code base consists of dozens of ansible roles and playbooks. Otherwise, I would need to copy all that code into a separate private_data_dir for each playbook run which is clearly not the way I want to go.

One more thing I would like to stress upon is I don't need to reuse anything dumped to filesystem. Be it envs, cmdline, passwords, etc... I handle this programmatically. That is why I also tried to set suppress_env_files=True in settings so that env is not dumped to filesystem. But it didn't work.

Here's the part of runner thread code I use to run playbooks. And it works! I use ansible-runner==2.3.1, ansible==4.10.0 (ansible-core==2.11.12)

settings = {'suppress_ansible_output': True,
                    'suppress_output_file': True,
                    'suppress_env_files': True}

unique_private_data_dir_for_current_run = os.path.join('/opt/ansible/private_data_dirs', f"{unique_identifier}")
ansible_runner_kwargs = {"project_dir": '/opt/ansible/project',
                                 "private_data_dir": unique_private_data_dir_for_current_run,
                                "playbook": playbook,
                                "inventory": json.dumps(inventory),
                                "extravars": run_data,
                                "verbosity": verbosity,
                                "envvars": {'ANSIBLE_STDOUT_CALLBACK': 'mycustomstdoutcallback'},
                                "settings": settings,
                                "cmdline": cmdline,
                                "passwords": passwords
                                 }
result = ansible_runner.run(**ansible_runner_kwargs)

# remove because not needed and some files (passwords) under unique_private_data_dir_for_current_run/env 
# directory may contain sensitive information. For example, ansible vault password. 
shutil.rmtree(unique_private_data_dir_for_current_run)