I wouldn't race with it on your body, though.
Hobie Class Rules:
18.1 b) Weight and water jackets for the purpose of adding personal ballast are not permitted, shall not be considered clothing or equipment, and shall not be allowed for meeting minimum crew weight.
Try explaining to a Jury that it wasn't being used to increase your weight (especially when it works just as well attached to the trampoline).