Had that once, then Daniel van Kerckhof stepped on board and showed me how it is done.

You sail is too full, too much draft. Or you squaretop is not twisted off enough.

The bound up feel comes from having a draggy tip vorted; squartop is not twiste off enough

The excessive heeling comes from a mainsail that is too full.

Remedy, reduce mainsail draft and increase twist in the top. Square top leech should be about 200 mm of the centreline, sometimes even more (Glenn ashby advice).

Reducing draft : derotate the mast. 60 degrees sound like a whole lot to me even on a teardrop shaped mast. Pull on downhaul more while letting off some mainsheet tension. A beneficial side effect of these actions is that the leech will twist of more. Exactly what you are looking for. When lucky the top of the mast will bend off to lee and have the top twist more still. Best would be if this addition twist is so great that you can add some mainsheet tension again to bring it back in and thus flattening the mainsail even further. Now you have decreased draft a little more.

Boat should not be at all flighty anymore and it should quite suddenly pick up speed when nearing the optimal trim. It may even be the case that you'll need to come of the trapeze, not enough power to keep you out. However, you are going faster just the same. Trapezing is not a must, do it only when the optimal trim requires you to trapeze out. If you are not trapezing in optimal trim but everybody else is then take a hard look at your prebend settings, you may have too much of it, making the middle part of the leech fall away to much and basic losing power that way. DON'T try to keep yourself out by pulling the head in to far. The result is a flighty boat and slowing yourself down, either way you'll be less competitive.

I found that when trapezing is marginal on these boats (new mainsails) that then it is better to sit in, hike out and do alot of mainsail sheeting in the gusts. It is better to open up the sail and shed excessive power that way then the be on the trapeze and close up the sail to keep you out thus adding alot of aerodynamic drag, slowing you done.

Hopes this helps

Wouter