My problem with the mast rotation was the clew position, with it further in, the sail flattens and the sail shape is correct. My new problem is that I have no "tension" on the square top and it inverts.

I did not use the inner hole before because of this, but even with the outer main clew hole the square top inverts as soon as it blows more than 5-7 or I use any down haul.