Around here the pecking order is by age of the activity.
Fishermen, both diving and pole, surfers, sailboards, then kite surfers. Cats rate near boards or kites. The smartest sailors and surfers stay well away from fishermen. i personally am constantly asking my crew to look for divers.
As Rick implies, fisherman are a strange breed. There have been stories of conspiracies to screw with sailors(and surfers of old) by fishermen. The guy could well have been out trolling for an accident the way some do in traffic. He obiviously wasn't planning on hooking a fish if his pole wasn't hooked to the boat.