I posed your question to a very well exercised, experienced and respected yacht designer. Here's what he said:
Tom - There is no real formula for length vs. mast height. In truth, because the beam and weight of a cat determine the stability, the relationship between sail area and stability is what you want to look at...and there is no formula for that either. The boat really needs to be designed for the sailing area and the skill of the crew. A boat rigged to go offshore in the big winds around Australia will be a boring dog, sailing in the generally light winds of the US east coast. A boat rigged to be fun sailing for a performance sailor like me would be terrifying to a retired couple.
In my opinion, the question is: How much boat do you want to take care of?
All those extra square and cubic feet of space and volume that a catamaran provides need to be maintained - either with your hands or your wallet.
Having taken care of large multihulls for others, the biggest boat that I would want to personally have would be 42-45' long - small enough for one couple to take care of and handle without professional crew, yet large enough for two couples or a small family to cruise aboard.
Now, with the funds available, I might stretch that up to 52' but, doing so, I would stick with the 45's interior accomodations and just stretch the hulls out. That way, instead of having a relatively heavy (for its length) cruising multihull, I would have a relatively light one, with a longer waterline and little weight in the ends, for an easier motion in a seaway. Weight, by the way, is the single most important factor in multihull performance.
As for the aspect ratio of the mainsail (mast height vs. boom length), it is a fallacy to think that a shorter rig with a longer boom is safer.
I once raced a tall rigged Stiletto 30 against a 36' Newick trimaran with a low aspect ratio rig in pretty big wind going from Miami to Nassau. After the start, we were ahead and to weather. From that vantage point I was able to watch how the same puffs were affecting each mainsail. On the Stiletto with the tall skinny mainsail, the leech would stretch when a puff hit, causing the top of the sail to twist off and depower automatically. On the Newick, the load on the leech of the lower aspect mainsail was much less, so it did not stretch. Instead, what stretched was the body of the mainsail, causing the sail to get more full right when you need for it to get flat. The result was that they had to reef and we did not...and we sailed away from them...
The maximum drive in a sail is in the forward part. So, by having tall, skinny sails, you are increasing the luff length and maximizing the drive while minimizing the parasitic drag. That's why you see gliders with long skinny wings, rather than short fat ones.
As a yacht designer, I've played around with the question: "How long a boat is possible with a rig limited to the standard ICW bridge height of 65'?".
For a performance oriented cruising boat, that came out around 39' LOA. By increasing the boom length a moderate amount and adding overlap to the headsails, I was comfortable up to about 42' LOA, with very little loss of performance.
Your 45' requirement limits the LOA to about 30-33'.
Keep in mind that, in order to have adequate under bridgedeck clearance, the base of the mast is going to be at least 5' off the water. that only leaves 40' of mast - less than a standard rig Stiletto 30, once you deduct the masthead instruments.