In China, the irregular high-rise buildings are beconung more extensive. Considering the irregularities and complexities of these buildings, it is significant to verify the safety and rationality of their seismic design through effective means including small scale dynamic model tests, refined numerical analysis and large scale joint tests. The target building of this paper is Shanghai International Design Center (SHIDC), which has two different-height connected towers designed by Japanese Architect Tadao Ando. The hybrid system of reinforced concrete core walls in conjunction with perimeter steel frames is out of Chinese code in the plan and elevation layouts. Thus, scaled model was carefully tested on the shaking table to obtain the weak position, dynamic characteristics, inter-story drift, and shear force of the structure. A linear dynamic analysis by ETABS was performed to compare, check, and support the experimental results. On the basis of those results, the seismic performance of the complex irregular structure was evaluated and the suggestions to the same type of structural design were given.