On every boat I have sailed there is always a crossover point in thewind range where your VMG (spped to next mark) is better by wiring downwind.

the issue with a singlehander is of course control, but once you learn how to deal with that you will be faster to wire it.

Sitting in so you can "crash" gybe faster is not really giving you the best VMG...