In the current research field of non-fullerene solar cells, the device performance of all polymer solar cells (all-PSCs) lags significantly behind those based on polymer donor and molecule acceptor. To further improve the device performance of all-PSCs, the small molecule electron acceptor ITIC was introduced for the first time to the all-PSCs based on PTP8/P(NDI2HD-T). The combinative polymer/molecule acceptor P(NDI2HD-T)/ITIC with only a small amount of ITIC, 15 wt% in acceptors and 6 wt% in total, can significantly improve the device PCE from 6% to over 7%, which is among the highest-efficiencies for reported all-PSCs. The improved device performance can be attributed to the broadened absorption and optimized blend morphology, which can further enhance charge carrier generation and balance charge carrier mobility in the ternary blend. More importantly, we discovered that this strategy can be successfully applied to all-PSCs based on other polymer donors like widely spread P3HT or PTB7. Therefore, our work not only reveals the detailed effect of polymer-small molecule acceptor alloy in all-PSCs, but also suggest that the ternary cell strategy of polymer donor/polymer:molecule acceptor may become a general and facile approach to further boost the performance of current all-PSCs.