Metadata bugfixes

Bugfixes:

• Fix symlink from metadata to experiment file
• Capture correct cell output as main metric (caused a bug when no output present in last cell)
• Allow setting trials-folder in config

0.5.0: Composite experiments

Composite experiments

This change adds the ability to stitch together steps and lookup dicts from different modules into a multi-step experiment. The intent is to support research workflows that draw from different data creation pipeline, different evaluation criteria, and different intermediate steps, all of which may have scientifically important variants to evaluate.

This applies some additional requirements to the type signature of steps (steps which pass data must return a dict with a "data" key, which is then passed as the first positional argument to the next step). It also allows better static typing of experiment steps.

Old behavior is still accessible, but requires the -m flag on the command line.

Additional features:

• Default trials folder is now repo root
• Remove special "group" CLI argument. Treat group as any eval argument
• Add --version to CLI and __version__ to module
• Add logfile "experiment.log" and source.py to metadata. Given requirements.txt and source.py, it should be possible to reproduce the result of any trial.

Bugfixes:
No longer create kernels that persist after trial

The first evolution

Features:

• explicitly set experiment log level based on whether running in debug
  mode.
• Do not catch and except errors in cells when running in debug mode. This should allow a user's debug session to cause a breakpoint on exception
• Add load_trial_data() to load from a particular hextr
• Add py.typed marker

Fixes:

• Allow calls string eval args, e.g. filename
• Allow empty eval args
• Prevent double-runs of experiments caused by importing main and then
  running it.

Backwards incompatible changes:

• mitosis.run returns hexstr, not experiment metrics

Command Line mitosis

This change makes mitosis available via the command line, so users do not have to implement their own runners from scratch. It also changes how variants are handled in the jupyter notebook - they no longer are sent via pickle, but instead looked up anew. More information is also saved in the trial directory.

Calling mitosis.run() directly may be deprecated in the future.

First "Public" version.

New Features:

• Data, arguments, and freezefile from experiment run are saved in a folder for post-mortem analysis
• Debug mode no longer verifies variant names, checks repo, or touches db.
• Dill is used for saving results, hopefully obviating some errors on certain classes.

Bugs fixed

• Prohibit parameters whose values' string representation is not reproducible. Specifically, this applies to all objects using the default __str__(), which includes a memory location. (StrictlyReproducibleDict and StrictlyReproducibleList class). Previously these were allowed, but any follow-on use of the same parameter would have a different string representation and thus would be rejected by mitosis.
• Make kernel names random. Previously executable path was used to make a kernel name, but spawning multiple experiments simultaneously created a race condition and one of the processes would die.
• Clearer error messages
• Resolved some unicode stuff with a raw string.

New Contributors

• @yb6599 made their first contribution in #11

Full Changelog: 0.1.0...0.2.0