Agreed. I don't believe neoprene absorbs water, however, the fabric that is often bonded to it when used in wetsuits can absorb water.
There are probably lighter foams that could be used, 1/2" neoprene is going to get pretty heavy, but the flexibility of neoprene would be nice. I would also be concerned about weather/UV degredation.

sm