Drafts Can Blow Out A Pilot Light If your fireplace, water heater, or furnace pilot light goes out ... Then, use a brush — such as an old toothbrush — to gently scrub away any corrosion.