Background: Accurate identification of pCR to neoadjuvant chemotherapy (NAC) is important for determining appropriate surgery strategy and guiding the resection extent in breast cancer. However, a non-invasive tool to predict pCR accurately is lacking. Our study aims to develop ensemble learning models using longitudinal multiparametric MRI to predict pCR for each molecular subtype of breast cancer. Methods: In this study, all patients underwent pre-NAC and post-NAC MRI examinations, and we collected multiparametric MRI sequences. Post-operation pathologic results were used to determine the pathologic complete response (pCR). We extracted 14676 radiomics and 4096 deep learning features and calculated a set of additional delta-value features. Then in the primary cohort (n = 409), inter-class correlation coefficient test, U-test, Boruta and the least absolute shrinkage and selection operator (LASSO) regression were used to select the most optimal feature set for each subtype of breast cancer. Finally, 20, 15 and 13 features were selected to construct the models based on 5 specific algorithms with cross-validation method for predicting pCR in HR+/HER2-, HER2+ and TNBC subtypes. The ensemble learning strategy was used to integrate the single-modality models outputs and improve the prediction performance. The diagnostic performances of models were evaluated in the three external cohorts with lager sample size of patients (n = 343, 170 and 340, respectively). Results: A total of 1262 patients with breast cancer from four centers were enrolled in this study, and pCR rates were 10.6% (52/491), 54.3% (323/595) and 37.5% (66/176) in HR+/HER2-, HER2+ and TNBC subtype. After model development, the Multi-Layer Perception (MLP) neural network yields the best diagnostic performances in all subtypes and feature sets. In the three subtypes, the stacking model integrating pre-, post- and delta- models yielded the highest AUCs of 0.959, 0.974 and 0.958 in the primary cohort, and AUCs of 0.882-0.908, 0.896-0.929 and 0.837-0.901 in the three external validation cohorts, respectively. In all subtypes, the stacking model had accuracies of 85.0%-88.9%, sensitivities of 80.0%-86.3%, and specificities of 87.4%-91.5% in the external validation cohorts. Conclusions: Our study established a novel tool to predict the responses of breast cancer to NAC and achieve excellent performance. The models could help to determine post-NAC surgery strategy for patients with breast cancer.