Transformers for Time Series Forecasting - Rainfall prediction

The entire code for Ranifall prediction using time series forecasting is available. The prediction is done using 2 models - transformers and RNN. The output is that RNN is performing better than transformers. The code needs to be fine tuned such that transformer performs better than RNN.