Implement hyperparameter optimization mechanism

Implement a script to start hyperparameter optimization. This should have the following features:

• Parallelization, i.e., run multiple processes at once.

  • Done. We can now use the joblib launcher, specified in config.yaml, launching config.n_jobs processes synchronously. A disadvantage of the synchronous execution is that it waits until all processes are finished before starting the next batch. This wastes processing time. A workaround is pruning by learning to early stop.

• Evaluate average score over n runs, do not rely on a single run.

  • Done, each run is now executed at least sweeper.min_trials_per_param times, as specified in the config.yaml

• Use GPU computing (this is not yet working in hyperopt.py)

• Prune unpromisng runs. E.g. by learning to early stop, i.e., add a parameter that says how many more steps than the so far fastest run may have. I.e, specify a factor x that says stop after x*n_steps_of_fastest_run. Or by using optuna's built-in pruning feature.

• Apply to our RL framework

• Implement mlflow custom logger

• Check how we can define search spaces for parameter lists.

  1. Either this can already be done with list_name.0 and list_name.1, etc. or list_name[0], list_name[1].
  2. Or we have to get rid of the lists.

• Enable asyncronous multiprocessing. Currently, the joblib launcher waits until all trials in a batch are executed, so that it wastes time waiting for the longest runs. These longest runs should either be pruned early (see above) or the framework should not wait for them to finish if resources from previously finished trials are free.

• Implement run_hyperopt.sh shell script. NOTE: Parallelism with a shell script also does not work, it completely messes up the parameter generation because the sweeper does not seem to make new suggestions after a result in a previous process has been obtained. Also, as a consequence, it generates the first n_jobs jobs with the same parameters.

  • Make sure that each process is run at least sweeper.min_trials_per_param times.
  • Make sure that the trials of the same run are stopped in the same subdirectory, and not like now in a subdirectory named after the starting time of the run.