No problems my friend.
There is a definite difference driving in TOD and just 2wd.
It takes me less than 100 yards to realise I left the lever in TOD last time I drove.
Noise level is worse in TOD (still not bad, but definitely different), and VX corners differently.
I think there would be a slight difference in fuel consumption, but have never tested it. Free wheeling hubs should make even more difference, but I haven't got around to finding any yet.
PK