I have to do this also this weekend. I have found a sailmakers page that tells how to do it. I am afraid it is in german:

http://www.sobstad.ch/Deutsch/numeros.htm

In a few words, they say you should use lots of solvent like gasoline or turpentine where the old glue ist left over. Once the glue has turned smoother, you carefully use a scraper to get rid of it.