It sounds like it is too close, to me. Find something to drive on like a ramp to test your flex out a bit. If you place a rear tire on a tall curb or cinder block it will force a front wheel a little higher into the wheel well by greater body weight transfer. You could even use a floor jack to force some flex to see where you're at as a start tho. In my experience the worst rubbing usually comes from reversing.