For one thing, your check engine light will always be on, so it won't be able to alert you to any new problems that may arise.
There is also the potential that not having EGR for a long period of time could put your engine outside of the operating parameters it was designed for. This could be something as simple as running lean on the freeway for a couple hours and burning a valve. It could do nothing. For this engine, which was designed with EGR in mind, I would leave it on there. That's coming from a guy who is also in a non-emissions state and have run vehicles without catalytic converters or emissions pumps.