All‐polymer solar cells (all‐PSCs) are attractive as alternatives to fabricate thermally and mechanically stable solar cells, especially with recent improvements in their power conversion efficiency (PCE). In this work, efficient all‐PSCs with near‐infrared response (up to 850 nm) are developed using newly designed regioregular polymer donors with relatively narrow optical gap. These all‐PSCs systems achieve PCEs up to 6.0% after incorporating fluorine into the polymer backbone. More importantly, these polymers exhibit absorbance that is complementary to previously reported wide bandgap polymer donors. Thus, the superior properties of the newly designed polymers afford opportunities to fabricate the first spectrally matched all‐polymer tandem solar cells with high performance. A PCE of 8.3% is then demonstrated which is the highest efficiency so far for all‐polymer tandem solar cells. The design of narrow bandgap polymers provides new directions to enhance the PCE of emerging single‐junction and tandem all polymer solar cells.