OK, might be a stupid question, but do you have the "Auto" button activated? If the Auto button is activated (the little green light should be lit, but yours could be burned out) AND the fan control is set to Auto, the climate control will try to keep the cabin at whatever temperature the slide is set to, and it will turn the blower fan on and off to achieve this, as well as kick the compressor on/off as necessary.

Usually when you activate the Auto button the A/C button will light up with it, but it is possible to press the A/C button to turn the A/C off, but still leave the Auto button active.