I have done that race single handed but the boats do get pretty spread out on the way back and the middle of Raritan bay is still pretty far offshore if you have any trouble. IMHO distance races are better suited for double handed boats.
Sounds like your interested in trying your hand at competative sailing in which case OD fleet racing is the acid test. Find a crew and join in.