Alright I figured out how to isolate it, just needed a better/thicker pry bar than the screw driver I had at the time. Didn't see any marks on my new to me/used fan when I was installing it, the entire mounting area was blueBut everything seems to be working fine, no longer sounds like a semi and fan is spinning a LOT more freely than it was.
Thanks for the responses but I felt like a moron for asking once I saw how easy it was with a larger 1'-6" SQUARE prybar. :embarrass Just stick it under one of the mounting bolts and wedge against the center of fan then crack one at a time and once they were cracked they were easily taken off.