Did you check the big nut, 23mm I think, under the EGR? Its on the passengers side. I had mine come loose and could hear an exhaust leak when ever the engine was under a load.
I also had both flanges break at the pipe, but both where incredibly obvious something was wrong. Bad noise all the time.
I have only one time, on a different car, had a cat come apart. Started as a rattle in the muffler when ever I hit a bump. I could tell by shaking the muffler. But it eventually blocked the pipe and the car wouldn't do anything above an idle.
Good luck
Dan