Take everything you can carry including traveller block et.
Tie cheap tarp over it and write your phone number on the cover.
This lets anyone who wants it removed know whose it is and stops the preying eyes. Also, anyone who wants it has the same problem you have about removing it. So only an outright vandal is against you.
Also, If others usually leave them.........are they old timers and are you just a paranoid newbie??