Background: Conventional anticancer drugs have been developed to improve overall survival (OS) and/or progression-free survival (PFS) in the palliative intent setting. However, long-term survival is often limited by acquired biologic resistance. In contrast to conventional anticancer drugs, ICIs putatively preserve long-term survival in a small subset of patients. However, in the absence of a gold-standard definition, long-term survival improvement has not been adequately quantified. Long-term survival rate difference has been proposed as a method of quantifying long-term survival. However, there a paucity of literature demonstrating statistical improvements in survival rate. Therefore, the objective of the study was to quantify the improvement in long-term survival rate in OS and PFS improvement in ICIs compared with non-ICIs. Methods: The FDA Hematology/Oncology Approvals & Safety Notification web page was searched for RCTs cited in oncology drug approvals between 01-2011 and 06-2019. The most recent OS and PFS updates were used when available. OS and PFS curves were digitized to reconstruct pseudo individual patient data as per established methods. Survival rate improvement at 1, 2 and 3 years were calculated as differences between the experimental arm and control arm. Survival rate improvements across RCTs at 1, 2 and 3 years were pooled using a random-effects model and the results were compared between ICI and non-ICIs. Subgroup analyses were conducted for melanoma and non-small cell lung cancer. Results: We identified 130 RCTs (22 ICIs, 108 non-ICIs) for analysis. The absolute OS rate improvement was 9.5% in ICIs and 4.9% in non-ICIs at 1-year (difference = 4.6%, p < 0.001), 10.3% in ICIs and 4.9% in non-ICIs at 2-years (difference = 5.3%, p < 0.001), and 10.2% in ICIs and 5.1% in non-ICIs at 3-years (difference = 4.6% p 0.006). The absolute PFS rate improvement was 10.3% in ICIs and 17.7% in non-ICIs at 1-year (difference = -7.3% p < 0.001), and 8.9% in ICIs and 15.5% in non-ICIs at 2-years (difference = -6.6% p < 0.001). Conclusions: Long-term OS survival rate improvements, while statistically significant were only modestly higher in ICIs compared to non-ICIs. PFS survival rate differences were statistically lower in ICIs compared to non-ICIs. While ICIs represent an important shift towards long-term survival, greater emphasis should be placed on rigorously evaluating long-term survival rate improvement and reporting of future novel therapies.