The following charts show the performance of cleanppo (left) and SB3s' ppo (right) on FetchReach-v2. cleanppo converges much slower. Find out why and improve the performance of cleanppo.
cleanppo
ppo
FetchReach-v2