Debugging and Training 2 combined models as one on PyTorch (Python-3)

I am working on combining a U-Net (model) with a Vision Transformer (model) to achieve the task of removing weather-based noise from input images (rain, snow, fog). My method trains a little bit and stop after few initial epochs then reports error in computing the first PSNR and SSIM metrics. I can't seem to figure out the problem.
At this point I need help to make the model train successfully and achieve good results.
Freelancer with good knowledge of Python3, PyTorch, and deep learning will be very useful.