improving java automation project design

I have some listeners which I need for test, each listener has its own base class to ensure we don't start listeners that we don't need for some tests. But this has resulted in many base classes. Would like an hour of a developers time to discuss how we can improve this and create some POC.