Use Cookiecutter to generate new experiments as well as workspaces

[marshrossney]

Summary

I realised that the most useful thing this tool could do would be to just semi-automate the task of copying relevant files and information from a workspace to an experiment directory. But if experiment directories could be templated according to user preferences this would be even nicer.

So, let's lean on the widely-used Cookiecutter to generate experiment directories from templates!

Implementation

Basically what happens is

1. Configurations are read and substitutions (e.g. for the directory in which to generate the experiment directory) made.
2. The full configuration dict is passed to cookiecutter.main.cookiecutter via the extra_context argument. This allows the user to refer to configuration parameters in the template. This generates the experiment directory (and templated files)
3. Additional files are copied from the workspace to the newly generated experiment directory. This could have also been achieved in step 2 using a post-gen hook.

Customisation and Cookiecutter 2.0

One thing that can be nice with Cookiecutter is being able to dynamically input parameters at the point of generation. However, the latest stable version of Cookiecutter on PyPI (1.7.3) only allows us to choose between prompting the user for all of the parameters defined in cookiecutter.json, or not prompting the user at all and relying completely on defaults and injected context.

This rules out prompting here, since we do not want the user to have to confirm, nor even be able to modify, the parameters injected by PyREx. Hence, while one is welcome to extend the base experiment template by adding more parameters to cookiecutter.json, these will not be prompted for and will hence take their default values.

It looks like this restriction will be lifted when Cookiecutter 2.0 is released, which introduces private variables in the cookiecutter.json which are never prompted for, but whose values can be overridden using the extra_context parameter.

Unfortunately, it's not clear when 2.0 will be released, and Cookiecutter seems to be suffering from a lack of maintenance. See these issues: #1555, #1636, #1642.