Scott,
Of course we're biased since we went the cat route rather than the kite route. I'm only a few years older than you, but have found more fun in catsailing because I've always like boat racing because there's a challenge when you've got a bunch of boats in a small area trying to work each other over.
Windsurfing (slalom racing)is great in big wind. Sailboard racing (triangles) can be fun, but raking that 10.0 around all day has it's disadvantages..
Imagine, even in 5kts, the cat can move pretty quick, and you don't have to hold the sail by yourself or stand in one place the whole time..
I would imagine that a windsurfer, by skipping over the water, would be faster than a cat (small displacement hull) on a broad reach, but I think overall, the cat will move better, especially when you can't get your windsurfer on a plane.
I had thought about kitesailing, but I'm not sure how you would race with one of those in tight quarters...
So, compromise by getting a cat WITH a kite (spinnaker).