Once, while I was in high school, I listened to a Red Hot Chili Peppers song, and listened for how well they stayed on the beat. We had been using a metronome in my guitar lessons, and I think I was working on sticking to steady tempo. But at the time I don't think I believed it was that important. If you're feeling the rhythm of the music, who cares if your bpm varies?
So I listened to a Chili Peppers song, and kept time in my head, to see how they varied the tempo. To my surprise, they stayed dead on. Dead. In a lot of songs, the tempo varies less than one might imagine, even when they're rocking the fuck out. When it actually does vary, it can be very evocative.
This reminds me of something I read in a guitar magazine once: while the audience will forgive the occasional bum note, they'll really notice it (consciously or not) when you unintentionally vary the tempo. So you have to develop a sort of mental metronome, or better yet, a kinesthetic one.