Stitch a circular fibreglass plate to the end, with a fair hole in the center for the retrieval line. Easy to make some extra holes in it, for less weight and drainage. This will help with your problem of pulling to far.

Take care with the edges of the hole your retrieval line will go trough, a metal fitting might be best there.