Comfy life jackets cost $$$ but are well worth it.
I have an MTI side-zip that is very nice. Mainly we have used Extrasport kayak vests in the past. The more pockets the better. The higher it sits, the better for having a harness and sitting on the deck.
When I lived on St. Johns I had a couple deep water experiances. If the water isn't rough, I would not swim with a jacket on. However I would keep it with me. It is always a good idea to keep a few feet of thin rope in a pocket so you can make a leash for the jacket and tow it behind you as you swim.
In fresh water, it is a different story...