We investigate the response of photonic crystal (PC) atoms with Kerr nonlinearity to the excitation of ultrashort optical pulses. Surprisingly, it is found that PC atoms with sharp resonant peaks in frequency spectrum (or with large quality factors) can respond to ultrashort pulses very rapidly when Kerr nonlinearity is introduced. More interestingly, the response time under high-power densities can be even shorter than the duration of the pump pulse. It is revealed that the nonlinearity induced broadening of dynamical frequency spectrum is responsible for the ultrafast response time, implying the possibility of ultrafast all-optical switching.