Fix Error In Custom Gym Environment För Reinforced Learning With Python
I have a Python program with a custom Open AI Gym environment.
After maybe 50 000 – 200 000 time steps I get an error.
I have tried on different computers and I tried to reinstall Anaconda environment but the problem is still there. (So I can make the error happen on multiple computers)
I want help finding and fixing that problem so the program can run for millions of time steps.
Error:
------------------------------------
| time/ | |
| fps | 258 |
| iterations | 25300 |
| time_elapsed | 489 |
| total_timesteps | 126500 |
| train/ | |
| entropy_loss | -0.054 |
| explained_variance | 0.0318 |
| learning_rate | 0.0007 |
| n_updates | 25299 |
| policy_loss | 29.4 |
| value_loss | 2.14e+07 |
------------------------------------
Traceback (most recent call last):
File "C:\anaconda3\envs\GymSB3Oanda\lib\site-packages\spyder_kernels\py3compat.py", line 356, in compat_exec
exec(code, globals, locals)
File "c:\_sb3withoanda\_newdeal20221202.py", line 717, in
model.learn(total_timesteps=nSteps)
File "C:\anaconda3\envs\GymSB3Oanda\lib\site-packages\stable_baselines3\a2c\a2c.py", line 191, in learn
return super(A2C, self).learn(
File "C:\anaconda3\envs\GymSB3Oanda\lib\site-packages\stable_baselines3\common\on_policy_algorithm.py", line 250, in learn
continue_training = self.collect_rollouts(self.env, callback, self.rollout_buffer, n_rollout_steps=self.n_steps)
File "C:\anaconda3\envs\GymSB3Oanda\lib\site-packages\stable_baselines3\common\on_policy_algorithm.py", line 169, in collect_rollouts
actions, values, log_probs = self.policy.forward(obs_tensor)
File "C:\anaconda3\envs\GymSB3Oanda\lib\site-packages\stable_baselines3\common\policies.py", line 592, in forward
distribution = self._get_action_dist_from_latent(latent_pi)
File "C:\anaconda3\envs\GymSB3Oanda\lib\site-packages\stable_baselines3\common\policies.py", line 610, in _get_action_dist_from_latent
return self.action_dist.proba_distribution(action_logits=mean_actions)
File "C:\anaconda3\envs\GymSB3Oanda\lib\site-packages\stable_baselines3\common\distributions.py", line 274, in proba_distribution
self.distribution = Categorical(logits=action_logits)
File "C:\anaconda3\envs\GymSB3Oanda\lib\site-packages\torch\distributions\categorical.py", line 66, in __init__
super(Categorical, self).__init__(batch_shape, validate_args=validate_args)
File "C:\anaconda3\envs\GymSB3Oanda\lib\site-packages\torch\distributions\distribution.py", line 56, in __init__
raise ValueError(
ValueError: Expected parameter logits (Tensor of shape (1, 3)) of distribution Categorical(logits: torch.
Size()) to satisfy the constraint IndependentConstraint(Real(), 1), but found invalid values:
tensor(, device='cuda:0')