Sounds more like a wheel bearing to me. But if it was that bad I'd think it would make obvious grindy noises. Of course, the same could be said about the diff, too.