I've worked with plenty of people who do this. Heck, there are some well-established MDs in this country whose tempo memory is abysmal. If you've done the material enough the band will usually just slide in at the right tempo, but if you're subbing/depping, it can be a real nightmare as a drummer because you sometimes get the sideways looks when you're 10 BPM too bright or slow based on the count-off.
Of course, any count-in is a blessing if it's in the right ballpark. I'm used to only getting two beats or even an upbeat. Depending on the context, often it's only a nod or a cue and we're off. There are some old school MDs who do everything that way. You're simply expected to know the material well enough to hit the tempo every time. Again, if you've done the stuff a lot, it's no problem. When you're depping for someone, it can be a bloody nightmare.
I've done what Bo does - i.e. keep trucking along at the counted-off tempo until someone realises there is a problem, and that can work in some circumstances. But you have to be careful. Sometimes you need to be sensitive and adaptive, so over time I try to figure out the tendencies of the person doing the count-in and adjust to that. I have one guy I've worked with on and off for years both on ships and on land, and he counts
everything off a little quick (unless he's got a metronome handy) and tends to push when he plays. We know this, so we ignore the count off and tend not to follow him when he starts to rush.
And while we expect concentration and focus out of everyone on stage, if you've played the same material every night for several months or years, it's understandable that the concentration is going to wane now and again. So if I know an MD is having one of those nights when he's counting everything a little slow, I just whisper to the bass player that we need to stay on top with a wink. In the end, the MD/conductor is starting the tempo, but he can wave his hands all he wants, if the rhythm section is driving in one direction, the band will follow. Half of them aren't looking at the MD anyway.
Perhaps that's arrogance, or perhaps it's just taking to heart what JR Robinson said, that "the drummer is the leader of any modern band." Tempo and feel are our province. Other guys get to do harmony and melody, but we have to deal in tempo, rhythm and subdivision. So whether I'm the actual leader or not, I try to find a way to take an active role in the process of getting the right tempos, etc. Every situation varies and you have to be diplomatic and not step on toes, but we need to take an interest. It's best for the music -- which is the most important thing -- and for me and my reputation (such as it is) in the end.
