I'm not really sure WHAT results it would have. If the shims were gone, it seems like there would be some play not normally there and a person would likely hear something knocking back and forth when making direction changes.
I'm simply trying to guess what could be causing that tire to at some times look normal and at some times look like it's sticking out farther than the other side as you've described.
On a front wheel drive car I have, I seem to recall c-clip retainers having to be used when installing the cv-joints. Maybe the one on your passenger side has come off?