Another factor that a lot of people don't realize is that a drum that is tuned very loosely, yet played fairly hard, will have the tension rods loosen more quickly than with the heads tensioned tighter. It's not noticed as much on toms since they're not played as much as the snare, also with very few rimshots. But y'know how the tension rods get harder to turn the more you tighten them? Obviously it's because the head is getting stretched more so there's more tension on the head, and more resistance [from friction] in the tension rod threads in the receivers and also at their flange/washer/hoop contact points. But it also works in reverse- a more tightly tensioned head's hoop will pull upwards on the rim & rods harder, keeping said resistance, helping prevent the subsequent loosening of the rods. So, a loosely tuned drumhead will lose tension faster than a tighter one.
Rimshots speed detuning along at any tension because, instead of the rim briefly having increased upward force from the head's hoop due to the head's playing surface being pushed downward with every stroke, a rimshot literally moves the rim downward briefly, so for a few milliseconds, there's no [or at least reduced] upward tension against the top of the tension rods, so they vibrate loose little by little. The head also gets stretched a little more at rimshot locations because the rim gets pushed downward a little more with every rimshot, which leads, slowly, to the stuff in the 1st paragraph.
Um, I can't provide any sort of an actual physics explanation, but these are the layman's terms as it was explained to me a few years ago via email exchange by [no, I'm not kidding here], a physics professor who was also an avid drummer.