You're not missing anything. If you go below a 16" rim, some of the components (Brake Caliper, Suspension Arm) that are nested inside the rim will no longer have clearance. Picture trying to fit your size 11 foot into a size 8 shoe. If you push the rim outboard (by either offset or spacers) then the clearance issue is reduced or negated.
The only reason I'm able to run the 15" rims is because God blessed me with proper offset.