If someone else wants to count in, I'm all for it. If it's too slow or too fast, then we can get out the metronome and see what's what. Clicks are a pain to learn to play with, but they end a lot of arguments!
I'll usually be the last one to cue the ending, unless the song structure is pre-determined, because that's when the soloists are doing their thing, or the vocalist is vamping, or someone is trying to play something cool. I'll watch whomever is riffing, shredding, Coltraning, whatever, and wait for that person to finish. When they look back, then it's time to bring it home.
Mid-song, though, I've found it's better to be boss than defer. If you know you have solid time, why not? Besides, the players who do have good time appreciate it, and they tend to be the ones you want to work with anyway.