This paper addresses the small sample size problem in linear discriminant analysis, which occurs in face recognition applications. Belhumeur, Hespanha and Kriegman (IEEE Trans. on PAMI, Vol. 19 (7) (1997), 711-720.) proposed the FisherFace method. We find out that the FisherFace method might fail since after the PCA transform the corresponding within class covariance matrix can still be singular, this phenomenon is verified with the Yale face database. Hence we propose to use an inverse Fisher criteria. Experiment results suggest that this new approach works well.