When a fuel cell stack is in an abnormal state or out of order and issues an abnormality signal, a part of reformed fuel from the reformer is bypassed through the first bypass conduit to the fuel gas exhaust conduit and diluted with an inert gas such as nitrogen gas from the exhaust fuel gas dilution conduit.