We only sail in saltwater at our club, but have raced some in mixed water and sweetwater.
We "never" wash our mylar sails, and they last just as long as our competitors. Dacron with a soft finish however I see as a different matter. The same goes for lines on the boat. These benefit from washing regularly when sailing in salt water. We also take care to wash/clean all blocks and travellers with roller or needle bearings. Likewise it is nice to clean the boat and trampoline now and then.