PTP said - "unless you are within 2 boat lengths of the mark at which point I think you have to give room to the port boat."

Wrong. At a windward mark, the port tack boat has no right of way even within the two boat length circle. Read Rule 18.3.