For me the issue is always the toms. It's too easy for them to sound like isolated drums, like an 80's drum machine.
kicks and snares are usually OK.
Overall, for me, the big issue with drum modules is the obsession with letting users dial in any modification - room size, pitch of drum, amount of drum wires, amount of drum damping etc....
In a real recording you decide all that upfront and don't really change it after the fact. In VSTs you can alter many of those things, but likewise, the further you depart from the default sound of the drum, the more it sounds weird, unreal.
In programs like EZdrummer and Superior I generally scroll through until I find a sound that I like. I don't select an 'ok' sounding drum and heavy edit it. People tend to want to heavily edit their drum module kit pieces - and knowing that, the module makers build in all kinds of editable factors which compromise the drum sound in the first place - like impulse rooms instead of ACTUAL room recordings.