Mitch,

It sounds like your rudders are either the wrong size or you do not have them adjusted in the vertical position correctly. As you get water pressure or resistance against them, the lever arm (in this case the rudder) loads up and by creating a moment, wherby you are countering it by pulling (or pushing) on your tiller. It sounds like the same thing you would experience when you can't put you rudders down all the way in shallow water and try to steer. You don't need it too far out to create a very large force working against you. I'm sure that it works eponentially too according to your speed.

Just my 2 cents.