Although drums are percussion instruments, I tend to think of it this way:
If you only play drums (set, snare, congas, etc), you are a drummer.
If you play all percussin instruments (have studied and know everything from set, snare, congas, etc, to marimba, timpani, and even little stuff like knowing proper technique for triangle, etc), then you are a percussionist.
By that definition, I am a percussionist. When I am talking business, etc, I refer to myself as a percussionist...but in daily conversation, talking to band mates, etc, it's easier just to say drummer.