There ar three "golden rules" for the shaping of a mainsail that should always be remembered as "the basics"
! the luff tension controls principally the top third shape of the sail
2 the mainsheet controlls the bottom third of the mainsail shape and the "straightness"of the leech (not the roach that is totally dependent on the battens))
3 the middle third is a "static set" that is predetermined by the amount (or lack there of) of the diamond tension.
These are the basic "set up" controls for the mains'l.
When sailing, all of these control effects can be "tuned" and adjusted by the inter action of the mast rotation the mainsheet traveller, and the foot outhaul. The effect from the combinations of all these controls is enormous so there is a fourth, and most important rule of all - never make radical and/or multipull control changes together as you will never be able to assess accurately the results of the changes but instead only make small incremental single control changes and assess fully its effect before going on to the next "experiment"
Regarding the "flick off" of the square top sail, less rotation of the mast induces easier and sooner twist off of the head of the sail.