"Well, I have to take apart my drivers side to see if thats where my noise is anyhow"

But you don't need to remove the outer race (green cup) & that is where it begins to get very painful.