Todd,
I found that problem with the Barz with the rubber gaskets without holes in them. I switched to the gaskets with holes in them and it was no longer an issue.
I also found it's important not to overtighten the strap on the barz. They will still stay on... I ended up paddling through the surf two days at tybee and bashing the lenses against the hull and they didn't go anywhere or scratch.