The representation of load dynamic characteristics remains an area of great uncertainty and has become a limiting factor for power system analysis and control. The random nature of the load makes load modeling a very difficult problem, which becomes even more challenging when the field measurements increase and the recorded dataset becomes large. This paper proposes a novel concept of modeling load based on support vectors (SVs) of load data. A three-stage procedure to find SVs of the recorded load dataset is presented. Then the load model is built on the SVs. Although the model is derived from only a small subset of the original dataset, it has a strong generalization capability to describe dynamics of the whole dataset. However, the computational burden on the modeling process is much relieved since only a small subset of data is involved. The proposed method also answers the question on how data should be grouped and how many load models should be built as data are accumulated. This paper infers that, although the data space where the load varies seems indefinite and big, its characteristic can be captured and modeled in a much smaller subspace. The presented method is shown to be effective by the case study on Hushitai substation.