at least the VX should prepare you for the rough ride you'll get in the Elise, the noise should be the same as well
seriously though, the elise would be super fun but not terribly practical, I would still get that and keep the VX over getting a RAV4, that way you'll have TWO cool cars instead of being left with a boring one, and yes, I realize the newer RAV4's are surprisingly fast but I doubt they will ever be seen in a concours...
I bet you'll love the gas mileage of the Elise as well!