I think your first point is the thing that really pisses me off about music today. I was listening to this interview with Lez Zeppelin, and they said the treat The Led like it were a canonic work, like Beethoven. The reason why bands like The Zep and The Stones, ELP, Crimson were so good was because they could take, 'steal,' music but do something unique and interesting with it. Jazz was is fundamentally genre messing, and when you go back to Bach, Mozart and the Beethoven, they were doing the same thing.
It can be silly:
I think that you are right that it is the lyrical content of the Motown tune that separates it from bubble gumminess. Its 1967. Any song with 'Free' or 'Freedom' is making a statement whether it wants to or not. What if SS, was, "Freedom, oh, sweet sweet Freedom, here comes a bright new sun, and it leaves me wanting more." :) A song I always liked from that era was Build me up Buttercup.(You can't write a song like Eight Miles High and say you were innocent enough to think no one would take it as a drug reference.)
I think that the other important facet is that popular music is about the songs, and the lyric is on par, and often more important than what is happening musically, "Blowing in the Wind." And the visual can be as important as in musical theater, dance or musical film. That may be unnerving for musicians. If you can making living doing what you like like Bruford, Dejohnette or Billy Cobham, and make a living from it, my hat goes off to you. But I know quite a few jazz drummers who have taken gigs doing R and B and hip-hop. One of my favorite jazz drummers did hip hop a tour with this guy I worked with years ago.
You and I both have a pop sensibility. I love that live version of Down Let the Sun Go Down on Me, where the crowd goes wild when George Michael introduces Elton. GM has a great voice for pop It's going to leave me wanting more after a song or two. But we can also recognize a sense of musical accomplishment.
The reason why I mentioned "Alabama" specifically is because this is a piece that transcends music. It certainly transcends aesthetics. It was written after a racially motivated church bombing in Alabama left four young girls dead. The tune is more than music: moving, haunting, edgy and transcendental. The ability that the Coltrane Quartet had to just sit on that edge is breath taking and in this piece, it is fully realized. Knowing the narrative and listening to the piece, would one say that they are not moved by it?