Weight is an issue. The boat probably sails best with a combined crew weight of 310 lbs. Over 350 you can pretty much forget being competitve.
You're welcome to sail with me but I'm 235 so do the math.
PM me here if you'd like.
Jesus Pete – Don’t be another of the whinny butt people blaming their poor performance on their weight.

Yes weight matters, but it is not something that will make you non-competitive. Heavy teams have an advantage in a blow, light teams have an advantage in light stuff and in a drifter it is about luck and local knowledge. Either way it is measured in boat lengths and tends to average over time. Non-competitiveness comes from the driver.