This paper presents an improved incremental learning technique for SVM, namely redundant incremental SVM (RISVM), for pattern classification problems. Through adding some non-support vectors (say, redundant vectors in the sense of contribution to the final solution) at each incremental step, the RISVM algorithm can achieve similar performance to the SVM in batch (or non-incremental SVM) but result in less support vectors for the same quality of pattern classification, and also it can provide better generalization performance in comparison with other incremental techniques for SVM. The bispiral problem and five widely used benchmark data sets are employed to verify the method, and the simulations support the feasibility and effectiveness of the proposed approach.