Are you sure the jack is tightly set against the base and the bar going over it? That could make a noise if rattled a bit.
I'm not sure what the prevsious owner may have done underneath the panels, but it is possible that the metal mounting bracket frames behind the panels aren't screwed down tight enough... including the speaker mounts.
To get behind the other panel to the speaker is another "chore", unless you just want to pry it out enough to stick your arm back there. I would take off the grab handle on the upper panel by the roof. If I remember correctly, there is a round screw cover on the upper panel by the roof that needs to be popped off to get to the screw. Most of taking this panel off is popping the plastic plugs out... although if you REALLY want to get in there, you'll need to take off the front seatbelt... but to do most of the work back there, there really isn't a need.