First off, rudiments are not at all like musicial notes. Someone looking at a piano for the first time would have to be told where middle C is, and what scale is based on that note, and what modes and chords are based upon that scale. Without knowing any of that you could not functionally play the piano. I realize that's arguable but it should be obvious what I mean. A drum is not a piano or a trumpet.
Someone looking at a drum for the first time would simply have to hit the drum. So there's that.
Secondly, if, as the argument seems to be going, everything a drummer plays is a rudiment, whether or not that drummer has any knowledge of rudiments, then why protest? Roy Haynes has proudly said that he doesn't know one rudiment from another, and I don't see where that's held him back.
Really, the question is like asking someone to come up with any object that isn't made up of molecules. What was being played on drums existed before the organization of rudiments. It just took someone to sit down and do the organizing and to give those resulting organized systems their names, and now we have the rudiments.
Anyway that's my story and I'm sticking to it.