I'm more metal than punk, I expect I do alot more hard work at faster speeds than the average punk drummer.
Playing fast is what I got into drumming for to begin with, I was trying to go as fast as I could as soon as I got my first kit 10 years ago. Yeah it's hard work physically, but just like anything... if you do it alot it starts to come naturally, and when it starts to become 'easy' you feel quite a sense of power, you feel like your right at the front leading the charge. something I don't think you can get from playing slower stuff, or 'laying back'.
But, like me, if you ONLY play as fast as you can for the first few years of your drumming, then the slow or quiet songs will be the ones you dread, because you don't have a feel for slow groove and you're gonna sound loose and amateur.
These days I can do slow and fast (also loud and... a-little-less-loud, still working on that), I've spent the last few years trying to expand my skill set, so I can groove well in many different styles, but playing at breakneck speeds is still the first thing I want to do when I sit down at the kit and what I find the most exciting to play, just as breakneck speed music/drumming was what I found most exciting to listen to before I was a drummer and when I was starting out.