I have done both, and to me it's not predicated by much more than to have available the sound palette I need for the music I'm playing.
For years I was one of those guys who dug Rush and Porcupine Tree and tricky music, and so my kits were large. In fact, for some years I had my kit set up similarly to Neil's because in private that's what I enjoyed playing most. There was also a band where I was asked to specifically do "proggy" stuff, and so I brought most of the kit when we gigged. I downsized as appropriate for musicals and other for-hire gigs. Having a 9-piece wraparound kit would have made no sense to back a choral society singing swing classics, for example. When I moved to my new home, I similarly downsized my kit to what made sense to play in each situation.
And now, I'm in playing situations where my new acquisition, the 4-piece Ludwig, makes all the sense in the world. It also allows me to really look at different aspects of my playing, which as of late has become much more groove-oriented and less "busy".
It wasn't to be more or less "free", although I do miss having something to the left of my hi-hats when I play a smaller kit for whatever reason.