If playing to a click live is unfair because drummers of old didn't have access to it, there are a long list of things that should also be counted as unfair:
- Playing on modern drum heads is unfair, because in the old days drummers had to use animal skin.
- Playing the hihat with sticks in unfair, because in the old days it was on the floor, and only played with a pedal
- Playing with microphones is unfair, because in the old days drummers had to be able to make themselves heard without them
...and so on. If you go far enough back in time, drummers had to use rough pieces of wood or their hands to play on hollowed out tree trunks. Is it unfair that we have access to more finely crafted instruments?
Advances happen all the time. You're free to choose whether or not to take advantage of them, but using them does not make you a worse player. In the end, the music is what counts, and if you produce better music with the help of certain tools and gadgets, more power to you.
I play to a click for a couple of songs live, where we play to a prerecorded backing track (some keyboard patches and background vocals), and the songs sound much better than if we were to only play it using what we have available to use live as a band. Playing live to a click was a challenge at first, but like any skill it gets easier and more comfortable with practice, and now I don't even think about it.