It did wear the car when I did it, but you just use the other hole and double life of the car. I got a few years out of them. Since they are relatively cheap I didn't really care. I did not notice any issues with excessive friction.
When you swapped holes did you flip the car?
