Help me resolve hostnames of clients on vpn.

I'm using openvpn and I can connect machines with ip address just fine, but I cant connect with hostnames. I found some instructions about how to set up resolvconf to update when connecting to the vpn, but this is not enough. I think I also need to set up dnsmasq correctly but this is a little bit outside of my abilities. Any help would be appreciated.