Totally understandable, but I would still stand by media blasting as the best/ fastest way to get all of the old paint and rust off of the rims. I'm sure a wire wheel on your drill would work fine, but some left over paint or rust might be hidding in a crevice somewhere and you might have to end up repainting the wheels in a couple of years, but if they are just for off road that might not be so bad.
"The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong, it usually turns out to be impossible to get at or repair."
-Douglas Adams, Mostly Harmless