The overtaking boat complains I slowed down.
Tough! - you're entitled to. He has to keep clear. (assuming - if you slowed deliberately - that he has been given time and opportunity)
If you haven't slowed deliberately, then he just has to cope - it's part of racing.