I would have to agree with Jay; there has to be a wire somewhere that is shorting out and when you move the fuse box around. I've had my fuse box off a couple of times and I can't really remember where the big wiring harness comes out of it, but it has to be grounding out on the fender well somewhere.
The only other thing that I could think of is that maybe one of the fuses (be it one of the individual ones or one of the big main ones) has something loose in it. It might not look blown, but maybe some of the contact are a little wonky making it lose continuity. 
Those are my guesses.
"The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong, it usually turns out to be impossible to get at or repair."
-Douglas Adams, Mostly Harmless