In this paper, we address the facial expression recognition (FER) problem using kernel canonical correlation analysis (KCCA). Following the method proposed by Lyons et al. [7] and Zhang et al. [8], we locate 34 points manually from each facial image as the landmark locations and then convert these geometric locations into a labeled graph (LG) [7] vector using Gabor wavelet transformation method to represent the facial image. On the other hand, a semantic expression vector consisting of the semantic ratings of each facial image is used as the semantic expression representation. Learning the correlation between the LG vector and the semantic expression vector is performed by KCCA. According to this correlation, we can estimate the associated semantic expression vector of a given test image and then perform the expression classification according to this semantic expression vector. Moreover, we propose another efficient algorithm for KCCA, which can avoid regularization to the Gram matrix. The experimental results on the Japanese Female Facial Expression (JAFFE) database and the Ekman's "Pictures of Facial Affect" database illustrate effectiveness of the KCCA method in facial expression recognition problem.