Yeah when it's just sitting there with a cold engine it's relying solely on pressure from the spring and the smooth surface of the seals to keep water from leaking past but when the seals start to go, bearings get a little play in them, etc. that spring pressure alone won't get the job done and water sneaks past the seals and out the weep hole. When it's running, however, there is additional pressure from the wet side which helps seal things up - it pushes on the impeller forcing the faces of the hard seal together tighter and also pushes on the rubber between the shaft and the rotating part of the hard seal, making it less likely water can sneak between the shaft and the rubber seal. If you want to see it leak faster, take the radiator cap off and let it run with the cooling system at atmospheric press. - that way you have the second worst case scenario for leakage - a spinning shaft but no pressure to help the seals. The worst case of course is when the seals get so bad pressure doesn't help, it hurts. Then you're blowing out water while driving, which can get you stranded on the side of the road, overheated, warped heads, etc. A pump will last quite some time in that first phase of failure but when it starts leaking under pressure - hosing down the bearings with hot water, the rumbling begins and things fall apart pretty fast after that. You don't know when the leaking started, so you have no idea how close you are to the catastrophic phase...

Just in case the seller tried to "fix' the leak with some of that Stop Leak stuff, you might want to consider adding some cooling system cleaner a few hundred miles before you take the vehicle in for service, then (provided your temp gauge stays in normal range - i.e., you haven't been experiencing overheating probs) a day or two before service, drain the system and replace with plain water to flush everything out. Ordinarily you wouldn't want to run plain water since the seals depend on coolant for lubrication but you're replacing the pump anyway so it doesn't matter...