More beam = more righting momentum = more speed

Nowhere more noticeable than on the upwinds particularly when punching of a start line.

Other factors such as sailing skill will always come into play, but reduction in beam is a guaranteed disadvantage. That you cannot hide that or argue against.