Maybe I'm just rationalizing my ineptitude, but think about how many hours you can devote to drumming. It took me a long time to realize that my playing goals need to match the amount of time I'm spending on drumming. If I can't devote myself drumming, fulltime, for a few years, I can't set the same goals as someone who can.
From what I've seen, there are three ways to deal with that.
1) Develop an incredibly huge ego and convince yourself you're a genius.
2) Have fun faking it and considering yourself a bit of a harmless fraud.
3) Learn to approach music with humility.
I've spent a few years working on #3, but now I'm giving up and going back to #2; "I'm not a drummer, but I play one on stage." That works best for me.