Transformer with FL

Hello, I used the transformer and Federated learning methods FedPer and FedAvg in the field of HAR. In this way, I used only the Encoder part of the transformer and instead of applying positional encoding to the input, I applied it to the attention matrix. Then I came to the FedAvg and FedPer methods, on the client side, I used a transformer to train the model, and on the server side, I used a transformer to aggregate the weights. I considered the number of epochs to be 100 and 2 learning rates, one 0.01 and the other 0.001, and the batch sizes of 16, 32, and 64. I measured the model's accuracy in these lr and different batch sizes. Now where is the problem? The problem is that my model achieved excellent results of 99 in the MotionSense and PAMAP2 datasets. Still, in the UCI-HAR dataset, the model performed poorly and reached the maximum result of 91, which is not acceptable. I want to make the model work on the UCI-HAR dataset as well as the other two datasets, if it doesn't work better, use other datasets such as WISDM or Opportunity datasets. So my goal is that the model on UCI-HAR or these two other datasets in learning rates of 0.01 and 0.001 as well as batch sizes of 16, 32, and 64 achieves an accuracy of 99.