Speed is only one tool for following how you do, good part with it is that it's still there when others have pulled away and you are sailing behind. It doesn't take lot of time to look at it sometimes and combine this knowledge with how others are doing around you then it starts to be more useful. This data can be used for finding better trim between races which I see quite important for inexperienced sailors like me.

Anyway I understand your points on VMG and shifts but I still don't think that they are that important for good sailors mostly because sifts can be detected with land marks or compass and the VMG is only needed for sailors that don't know how their boat should feel.