Background: While Immune checkpoint inhibitors (ICI) have revolutionized the field of oncology, the benefits have come at the cost of serious side effects known as immune-related adverse events (irAEs). Approaches that can predict patients’ susceptibility to irAEs are key to their early detection and management. In the present study, we investigate the association between irAEs reported during ICI therapy across multiple cancer types and markers of tumor immune response. Our primary objective is to explore potential biomarkers for assessing patients’ risk of irAEs. Methods: 472 patients were evaluated who had tumor immune profiling performed paraffin embedded formalin fixed archival tumor biopsy samples using Omniseq Immune Report Card (IRC) and subsequently underwent ICI therapy. The IRC consisted of enumeration of tumor infiltrating lymphocytes (TILs) by immunohistochemistry (IHC) and TIL-associated genes by RNA-Seq, PD-L1 expression by IHC, and tumor mutational burden (TMB) by DNA-Seq. irAE type and grade were determined based on retrospective chart review. Fisher’s exact test was used to determined statistically significant associations between immune markers and irAE development. Results: Patients with lung (55%), ovarian (9%), and melanoma (5%) cancers constituted the majority of the cases. The median age of patients was 61, with 56% being female and 44% male. Most patients underwent treatment with (94%). irAEs developed in 36% of patients, with 2% of patients developing high-grade irAEs (Grade 3 or 4). Skin (11%), thyroid (10%), and GI (9%), were the most commonly affected organ systems. Increased TILs were associated with increased risk for any irAE (p = 0.04). A stronger association was noted in patients who underwent anti-PD-1/L1 monotherapy (p = 0.01) and/or in cases of lung cancer (p = 0.01). Interestingly, subanalyses by gender showed a statistically significant correlation between increased TILs and risk for any irAE in males (p = 0.006) but not in females (p = 0.63). High PD-L1 (defined as > 70% by IHC) was also significantly associated with increased risk for any irAE (p = 0.03). Subanalyses by gender and age again showed a similar association in females (p = 0.0002) and/or patients < 65 years (p = 0.04). high TMB and any irAE in female patients (p = 0.01) and in breast cancer cases (p = 0.03). On multivariate analysis, TILs by IHC appeared to be the strongest predictor of irAEs (p = 0.03). Conclusions: The tumor immune microenvironment (TME) has been shown to influence response to ICI, yet its association with irAEs has not been well studied. Our analysis sheds light on potential TME predictors for irAE in patients receiving ICI therapy. Further studies are needed to deepen our understanding of immune toxicity and to develop tools for identifying patients who are at risk.