It could be that your tires in front have conformed to front suspension variations, same with the rear = then when you swaped them out the differences in the tires are causing things to go squirrelly. When you go in for an alignment check to see if your rears are straight (I don't think much can be done to straighten them, but it would explain variations in the rear). I'm assuming the rotation was front to back - most tires these days can only be mounted in one direction, so you're not supposed to cross rotate them.

I get an alignment every year whether I think it's needed or not - but it helps to have a lifetime Firestone alignment deal. Out of the last 5 or 6, it's been off each time except the last (about 5 months ago) - everything was fine that time.

-- John