Don't know if you've solved your problem yet, but I had the exact same problem a while back and it turned out to be a dime or nickel (can't remember) that had gotten lodged in there and was completing the connection to turn on the light. Same as you, if I pushed it down, it would go off but then come back on.