sounds to me like you might need to replace a rotor as well.
about 4 years ago my right rear brakes went from being fine to having no pad at all. it ruined my rotor and i had to replace that. while they were being done the tech notived that the caliper was not retracting. they did a quick fix on it and it was fine for about two years. then the same problem happened again. so i replaced everything myself and i haven't had a problem since. it's not too had to do your self at all. just as long as you know how to do brakes you should be fine.