Background: Colorectal cancer is the second most common cancer in Hong Kong. Radiotherapy has been established as an indispensable component during treatment of rectal cancer. Radiation-induced adverse effects are common for radiotherapy patients, severely affecting the quality of life of both the patients and their caregivers. Although previous studies have demonstrated the impacts of radiation-induced adverse effects on the quality of life of rectal radiotherapy patients, little were studied about the impacts on the patients’ and their direct caregivers’ psychosocial health. As there was no previous similar study done in local context, this research serves as a pilot study for improving the understanding of psychosocial needs and quality of life of rectal radiotherapy patients and their direct caregivers in Hong Kong. Methods: This study was a prospective and observational research. Rectal radiotherapy patients and their direct caregivers were recruited from Princess Margaret Hospital, Hong Kong from 2022 to 2023. The primary endpoint was the questionnaire assessment of psychosocial needs and quality of life of rectal radiotherapy patients and their direct caregivers before and after radiotherapy treatment. Questionnaires in different versions for the patients and their direct caregivers before radiotherapy treatment, 3-month and 6-month post-radiotherapy courses were distributed. Results: Twenty-four subjects were recruited, including 13 patients and 11 caregivers. The results show that rectal radiotherapy patients worried about their health in the future. Their direct caregivers’ emotional and intimate lives were affected. Nevertheless, the patients’ health and quality of life improved throughout the post-treatment period. For the caregivers, their feelings of anxiety and depression improved throughout the follow-up period. Conclusions: Most of the results are consistent with previous studies. This study provides a knowledge foundation and research direction for future studies. More future studies with a larger subject number are highly encouraged to solidify the knowledge on psychosocial needs and to improve the care of the patients and caregivers.