"Should I remove the drivers side as well"

Absolutely. It prolly won't bother the TOD but the pumpkin may not like it.