Boomless sails are great, but you need to put some thought into settings [which is what you ARE doing]

Once you find an 'average' position for the traveller, you will move it forward a tad as you get over powered to let the leech twist off a bit and increase tension on the foot which will,in turn, derotate the mast a bit.
When you are looking for more power, let the traveller back a bit to make the leech more 'hooky' and take tension off of the foot to let the mast over-rotate.
As jake says - sheet like hell.
When I sailed boomless [6.0] and before crews union allowed them to sheet, I jammed the tiller between my aft foot and back beam so that I had both hand on the sheet for a moment.