If the holes in the aluminum tubing are enlarging than I would say you have way too much slop in the connections. It's causing rubbing and lateral pull on the bolts. Does the stock system have springs in the connections? I crank mine down pretty tight (springs nearly fully compressed) so that the rudders don't swing loosely and they also stay square to the crossbar. I can still smoothly control the helm and I don't need to put too much pressure on the hiking stick to maintain course. Also, the Murray's kit contains the shaped plastic guides which keep the connection perfectly square. If anything, these have worn out every three seasons or so, but the kit is pretty cheap so getting a new one is no pain on the wallet.

Also, is the steel bolt threaded all the way through? It shouldn't be. That'll wear on the inner gudegon lines for lowering/raising the Prindle rudders and wear on the aluminum. They should be smooth-shanked in the middle.

I've never seen a knuckle or u-joint system for the connection. Every Hobie, Tornado, Nacra, Inter, Prindle I have ever seen has always had a single-pivot connection between crossbar and gudgeon arm.