Now that I think about it, you could make your own hubs to mimmick the set up I described in my last post. All you need is a flange and a short piece of pipe to make that delrin bearing section. You would then attach the wheel to the flange. You may have to do some welding but it could work.

Let me know if what I said makes sense.