Hey Mike -
Keep in mind that by making that contact, you break a rule. If there's no damage or injury, you can't be penalized, but you still break a rule and the protest committee will likely say so. Also remember, if you're the overtaking boat less than two boatlengths to leeward, you'd probably get tossed, not the windward boat. It rarely pays to pass someone close to leeward - you're too limited in what you can do.
Best bet - stay with your puff, keep the boat moving, stay clear of the slower guy. You're going to pass him clean in the conditions described if you can resist the magic gravitational magnetic "come to me" pull that seems to occur. I'm guilty - and learned my lesson.