Real-time face tracking and 3D reconstruction from video is a challenge in computer graphics and computer vision. We develop a system to track face by integrating image pyramid algorithm and Kalman filter. First, Facial features of the first frame are acquired automatically by Plessey corner detector according to FDP in MPEG-4 and FACS based on anatomical knowledge and general 3D model. And then corner disparity is obtained by image pyramid algorithm and Kalman filter. Kalman filter is employed to resolve the tracking speed and occlusion problems. Finally, 3D model is reconstructed by SfM (Structure from motion). Experimental results show the high prospect of this algorithm, and the two issues of real-time face tracking and modeling are resolved in our algorithm.