Hydra `policy_kwargs` issue - SB3 expecting `List` and not `ListConfig`
There is a problem with passing the policy_kwargs
to the SB3 algorithms.
It is straightforward to define it in hydra:
However, the get_actor_critic_arch()
method of SB3 cannot convert it because it is of type ListConfig
and not List
. I already tried to convert the list, but assigning it to cfg.algorithm.policy_kwargs.net_arch
converts it again to a ListConfig
.
I guess this prevents us from running hyperopt for the hidden layers and anything that involves a list of parameters. If we do not like to change the original SB3 code and maintain an own version of that function, we need to find another solution. I am not sure if this is worth a pull request for the original SB3 code, as I can imagine that many people nowadays use hydra.