Hi Tudy,
I'm a fairly new cat sailor - I started this summer. For me, at least, fitness was not an issue, but of course men have a little more upper body strength. It is more strenuous if you have ocean winds and are trying to race. If it's too strenuous for you, you can learn to depower the boat and sail slower and more relaxed.
But you really do need to learn how to sail the thing, and I think that works best if you learn from someone else. The few times I have been able to sail with someone else, I learned a lot.
I did have experience with a Sunfish before I sailed a cat. Do you have experience sailing any kind of sailboat?
Jonathan