Fix error in custom Gym environment för reinforced learning with Python

I have a Python program with a custom Open AI Gym environment.
After maybe 50 000 – 200 000 time steps I get an error.
I have tried on different computers and I tried to reinstall Anaconda environment but the problem is still there. (So I can make the error happen on multiple computers)
I want help finding and fixing that problem so the program can run for millions of time steps.


Error:
------------------------------------
| time/ | |
| fps | 258 |
| iterations | 25300 |
| time_elapsed | 489 |
| total_timesteps | 126500 |
| train/ | |
| entropy_loss | -0.054 |
| explained_variance | 0.0318 |
| learning_rate | 0.0007 |
| n_updates | 25299 |
| policy_loss | 29.4 |
| value_loss | 2.14e+07 |
------------------------------------
Traceback (most recent call last):

File "C:\anaconda3\envs\GymSB3Oanda\lib\site-packages\spyder_kernels\py3compat.py", line 356, in compat_exec
exec(code, globals, locals)

File "c:\_sb3withoanda\_newdeal20221202.py", line 717, in <module>
model.learn(total_timesteps=nSteps)

File "C:\anaconda3\envs\GymSB3Oanda\lib\site-packages\stable_baselines3\a2c\a2c.py", line 191, in learn
return super(A2C, self).learn(

File "C:\anaconda3\envs\GymSB3Oanda\lib\site-packages\stable_baselines3\common\on_policy_algorithm.py", line 250, in learn
continue_training = self.collect_rollouts(self.env, callback, self.rollout_buffer, n_rollout_steps=self.n_steps)

File "C:\anaconda3\envs\GymSB3Oanda\lib\site-packages\stable_baselines3\common\on_policy_algorithm.py", line 169, in collect_rollouts
actions, values, log_probs = self.policy.forward(obs_tensor)

File "C:\anaconda3\envs\GymSB3Oanda\lib\site-packages\stable_baselines3\common\policies.py", line 592, in forward
distribution = self._get_action_dist_from_latent(latent_pi)

File "C:\anaconda3\envs\GymSB3Oanda\lib\site-packages\stable_baselines3\common\policies.py", line 610, in _get_action_dist_from_latent
return self.action_dist.proba_distribution(action_logits=mean_actions)

File "C:\anaconda3\envs\GymSB3Oanda\lib\site-packages\stable_baselines3\common\distributions.py", line 274, in proba_distribution
self.distribution = Categorical(logits=action_logits)

File "C:\anaconda3\envs\GymSB3Oanda\lib\site-packages\torch\distributions\categorical.py", line 66, in __init__
super(Categorical, self).__init__(batch_shape, validate_args=validate_args)

File "C:\anaconda3\envs\GymSB3Oanda\lib\site-packages\torch\distributions\distribution.py", line 56, in __init__
raise ValueError(

ValueError: Expected parameter logits (Tensor of shape (1, 3)) of distribution Categorical(logits: torch.Size([1, 3])) to satisfy the constraint IndependentConstraint(Real(), 1), but found invalid values:
tensor([[nan, nan, nan]], device='cuda:0')