Shouldn't be too heavy while sailing straight, just a little bit of weather helm. Just to cover all bases, are your rudders all the way down, ie. vertical?

When tacking/gybing I had a lot of pressure on the rudders. Now that I've built new rudder boxes the pressure has decreased significantly while manoeuvring. It has to do with the pivot point between the tiller and the tie-bar. The hinge/universal joint should be about 2 inches in from the centreline of the rudder blades.