Background: HER2 amplification or overexpression is negative prognostic factor that occurs in circa 15% of primary invasive breast cancers, with positive results opening up eligibility for HER2 targeted (e.g. trastuzumab) combination therapy. However, HER2 intratumoral heterogeneity has been proposed as an explanation for the development of resistance to anti-HER2 targeted therapy. The aim of this retrospective study was to test the prognostic value of a HER2 heterogeneity index on ER+ breast cancer H&E whole slide images (WSIs). Methods: ER+ breast cancers slides (n=60 cases) of known HER2 status (immunohistochemically assessed over the 0 to 3+ range, with HER2 2+ borderline cases validated by fluorescence in situ hybridization) were digitized on an Aperio AT3 scanner. A subset of these patients (n=29) was accounted for by HER2 amplified/borderline cases. A HER2 heterogeneity index was tested and compared to standard clinical HER2 status in these cases and related to progression free survival (PFS). Survival analyses were performed using Kaplan-Meier (KM; with median cut-off) and Cox Proportional Hazards (as a continuous variable) models. Results: The HER2 heterogeneity index performed better in terms of prognostication than clinical HER2 status alone in relation to PFS (KM P=0.019; HR 8.06, P=0.01 vs. KM P=0.02; HR 6.75, P=0.048). These differences were more pronounced in the HER2 amplified/borderline tumor subset, where the HER2 heterogeneity index outperformed clinical HER2 status (KM P=0.04; HR 6.38, P=0.001 vs. KM P=0.10; HR 0.64, P=0.20). Conclusions: These findings highlight the merit of identifying HER2 heterogeneity from H&E slides alone and establish this index as a robust prognostic factor in determining PFS in ER+ breast tumors. Future work will extend this investigation into ER- tumors and evaluate patient-specific tumor responses to anti-HER2 targeted therapy.