Skip to content

Forward-Model implementation review

Issue Summary: Feedback from @ckv0173 for implementation of Issue #183

1. Training-data in fwd-training-data

Problem: The class name Training_Data is confusing.

Solution:

  1. Change the name of the class to Forward_training_data. The class is in src/utils/fw_utils.py.
  2. Change the import and usage of this class in src/custom_algorithms/cleansac_fw.py and src/custom_algorithms/cleanppo_fw.py

2. Forward model training code in fwd-model class

Problem: The code for executing the training steps of the fwd-models is currently mostly implemented in the custom algorithms which want to implement the fwd-models (cleansac_fw.py & cleanppo_fw.py). It would be better if it is just one function which has to be implemented there. Data_Loader collect_training_data aufruf sollte den intern in fwd Modell passieren.

Where: learn() in cleansac_fw.py and clean_ppo.pythere is a section with a comment 'Forward model training'

Solution:

  • write a new function in src/utils/fw_utils.py which can be called and where the logic is executed or just do it in forward_model.train().
  • Change the name of the cofig field 'retrain_every_n_steps' to 'train_every_n_data'.
  • Change the call 'self.fwd_training_data.collect_training_data(self._last_obs, action, new_obs)' to a function of the forward model
  • The forward model needs a reference of the fwd_training_data

3. Getters for shapes in fwd-model

Problem: The observation shape and action shape is needed in the fwd-model. Right now this is done with getters in the Actor class in cleansac_fw.py and in the Agent class in cleanppo_fw.py.

Solution: Think of solutions to simply this, so the implementation of a fwd-model in a algorithm is simpler.

4. Changes in the wiki

  1. for all the above the wiki should be edited according to the changes.
  2. At the reference of the documentation of pytorch there is written "Pycharm" instead of "Pytorch" in the section "Save your forward model"
  3. The heading of the section "Inference" should be changed, that it is easier to understand what this part is about for people not familiar with inference.
  4. Explain the difference between save_state_dict() and `save_model()
Edited by Leon Kruse