Wait so what did he say when you went over and talked to him? If you are nice and trying to work around his schedule then most people don't mind if you play your drums. Doesn't he work? You should be able to play when he is at work. If he's still complaining then put some pads on your drums, or maybe ask your parents to go talk to him, perhaps he would be more understanding talking to an adult. Some people just suck though about the noise. I tried to have band practice in the garage at my house once. We got about halfway through a song before a neighbor comes knocking. He said he couldn't even hear his tv, which was total bs because he lives two houses away and the rest of my family is inside the house watching tv.
As far as you getting in trouble for being too loud though, I doubt it. If you live in an HOA they may have different noise requirements, but most cities I really doubt you are being loud enough to warrant a ticket. Even if the guy called the police all they can do is tell you to quiet down. If the guy is being such a stickler about the noise, go ahead and play and let him be pissed off.