Background: Approximately 4% of epidermal growth factor receptor (EGFR)−mutated non-small cell lung cancer (NSCLC) present EGFR exon 20 in-frame insertions. These mutations confer intrinsic resistance to available EGFR tyrosine kinase inhibitors (TKIs), as they result in steric hindrance of the drug-binding pocket. Studies have shown that chemotherapy is superior to currently approved EGFR TKIs as a first-line or second-line treatment for EGFR exon 20 in-frame insertions. Nevertheless, the difference in the efficacy of different chemotherapy regimens in patients with EGFR exon 20 in-frame insertions remains unclear. Methods: Data from advanced NSCLC patients harboring EGFR exon 20 in-frame insertions from an independent cohort (Jenn-Yu Wu, 2019) was used to analyze the objective response rates (ORRs), disease control rates (DCRs), progression-free survival (PFS), and overall survival (OS) of different chemotherapy regimens. Results: Thirty-five advanced NSCLC patients harboring EGFR exon 20 in-frame insertions treated with chemotherapy were used to analyze, of which 7 received gemcitabine plus cisplatin (GC), 22 received pemetrexed plus cisplatin (PC), and 6 received taxol plus cisplatin (TC). ORRs were 31.82% for the patients received PC, 14.29% for GC, and 16.67% for TC. DCRs were 72.73% for the patients received PC, 42.86% for GC, and 16.67% for TC. Median PFS was numerically longer in patients who received PC chemotherapy (6.10 m; 95 % CI: 3.46–8.75) than GC (4.20 m; 95 %CI: 2.40–6.00; P = 0.14) or TC (3.40 m; 95 %CI: 1.96–4.84; P = 0.06). Median OS was significantly longer in patients who received PC chemotherapy (22.20 m; 95 % CI: 8.87–35.53) than GC (9.00 m; 95 %CI: 7.46–10.54; P = 0.04), and numerically longer than patients who received TC (8.10 m; 95 %CI: 0.00–22.62; P = 0.19). Conclusions: These results suggest that pemetrexed plus cisplatin may be the preferred chemotherapy for patients harboring EGFR exon 20 in-frame insertions, but further validation is needed in large clinical studies.