Background: Low response rates and toxicities from immune checkpoint inhibitors (ICI) highlight the need for improved ICI response prediction tools. Increasing evidence shows the tumor microenvironment (TME) plays a critical role in therapy response; however, current ICI response prediction methods often use a single tumor characteristic (i.e., tumor mutational burden [TMB]). Here, we present the BostonGene Tumor Immunity Portrait, an AI-driven molecular predictor based on the TME subtype combined with TMB status to identify ICI responders in non-small cell lung cancer (NSCLC), melanoma, and gastric cancer (GC). Methods: We collected a meta cohort from 26 publicly available datasets (n = 2,077) composed of NSCLC, melanoma, and GC patients that received ICI therapy. A transcriptomic-based TME subtyping approach was used to classify the TME as fibrotic (F), immune-enriched, fibrotic (IE/F), immune-enriched (IE), or desert (D) (Bagaev, Cancer Cell, 2021). TMB status (low < 10; high > 10) was determined by whole exome sequencing. A logistic regression model was used to generate a score of 0, 1, 2, or 3 based on the sum of points assigned to the TME subtype (F = 0 points, IE/F and D = 1 point, IE = 2 points) and to the TMB status (high = 1 point and low = 0 points). Each score corresponded to one of four Tumor Immunity Portrait types: 0 - Immunosuppressive; 1 - Resistant; 2 - Neutral; 3 - Immune Primed. Results: Tumor Immunity Portrait types were associated with increasing ICI response rates in each cancer type. Immune Primed type (IE + TMB-High) response rates in melanoma and GC were 76.2% and 100%, respectively. Immunosuppressive type (F + TMB-Low) was associated with ICI non-response in melanoma (p = 0.003) and NSCLC (p < 0.001). The Tumor Immunity Portrait accurately predicted ICI non-response, shown by the negative predictive value (NPV) of the Immunosuppressive type (F + TMB-Low) for NSCLC (100%), melanoma (92.3%), and GC (96.1%). Conclusions: Using an integrated approach combining the tumor’s characteristics and TME, the BostonGene Tumor Immunity Portrait provides accurate prediction of ICI non-response in 3 cancer types, warranting further validation for use in personalized treatment decision making.