Background: The COVID-19 pandemic and associated policies such as lockdowns spurred the widespread adoption of telemedicine throughout the U.S. Telemedicine has played a particularly important role for oncology patients by reducing potential exposure to infection and improving access for patients with poor performance status. However, as policymakers consider extending payment for telemedicine into the post-pandemic era, it is important to understand the impact of telemedicine on downstream care utilization in this population. We conducted a retrospective cohort study investigating the relationship between telemedicine use in oncology and subsequent outpatient oncology encounters, emergency department (ED) visits and hospitalizations at a large academic health center. Methods: We studied all outpatient oncology encounters occurring between 2018 and 2022 at the University of California, Los Angeles (UCLA) Health system, including in-person visits and telemedicine (audio-visual & audio-only) visits. We used multiple linear regression to predict the number of outpatient visits, ED visits, and hospitalizations within 30 days of an index visit based on visit modality, adjusting for demographic and clinical characteristics including patient age, race, ethnicity, sex, insurance, distance to clinic, distance to the nearest UCLA hospital, hospice referral, palliative care visits, median visit interval, appointment length, whether the visit occurred off the patient’s chemotherapy schedule, and the month and year of visit. Results: Our cohort included 62,815 patients with 672,427 outpatient encounters, of which 623,890 (92.7%) were in-person visits, 40,392 (6.0%) were video visits, and 8,145 (1.2%) were telephone visits. Overall, patients on average had 0.90 downstream outpatient visits, 0.16 ED visits, and 0.18 hospitalizations within 30 days of an index outpatient encounter. In our adjusted analyses, telemedicine encounters were associated with less outpatient utilization (270 fewer downstream visits [95% CI: 241 to 299] per 1000 encounters; p < 0.001). Telemedicine was also associated with greater ED visits (42.1 more ED visits [95% CI 18.5 to 65.8] per 1000 encounters; p < 0.001) and hospitalizations (54.8 more admissions [95% CI 33.6 to 76.1] per 1000 encounters; p < 0.001). Conclusions: Oncology patients who had a telemedicine visit rather than an in-person visit were less likely to have a follow-up outpatient encounter but more likely to have an ED visit or hospitalization within 30 days. Our findings suggest that telemedicine has largely played a substitutive role in the outpatient setting, but patients may require higher levels of care after these visits when compared to in-person visits. Additional research should further evaluate the effectiveness of telemedicine in oncology and define appropriate contexts for telemedicine use in this population.