Well...sailing CAN be a lazy sport. My wife and I just sold our 23.5' mono that was very relaxing to tool around on. Unless we were trying for max performance, it was comfortable cruising all the way.

It's all a matter of perception. If the general public were exposed, and by that I mean close-up, of what small, high-performance cats/skiffs are capable of, I think sailing would be regarded more highly as the real athletic activity it is.