I'd much rather stay between the hulls if I can.
I totally agree with you Pete, singlehanded is a different story I think, and certain precautions should be taken.
I've only lost my boat once, and that was really scary even though it was in the middle of a regular buoy regatta. I swam like a bastard for about 30 seconds and realized there was no fricken chance I was going to catch it in 15mph winds. The whole time I kept thinking, TURTLE!! TURTLE!!! come on you SOB TURTLE!!!!!. It also made me realize that a whistle is a very important piece of equipment when singlehanded and I really wished I had one at the time. That reminds me, I still need to go buy that whistle.