ePRO system

Patients used a web-based ePRO-monitoring tool on their own mobile device or private computer. The tool included symptom questionnaires based on CTCEA (clinician-based common terminology criteria for adverse events) with graphic display and side effect alert. By a messenger service patients were able to send requests and information to the clinic. On this basis, once a week patient received a symptom questionnaire and a subsequent symptom report was sent to the clinic. The questionnaire collected information on the patient’s general well-being, disease symptoms, and treatment side effects. In addition, the application provides the option to record symptoms in the form of a diary and to send additional reports or contact the clinic directly. The responsible physicians as well as the patients themselves have access to a graphic summary of all reported symptoms when logging on to their account.

Whenever a patient reports a symptom classified as severe by the application, an adverse event message (AEM) is sent to the health care team and to the patient, respectively. Depending on the severity of the symptom, the automated response to the patient would either instruct them to contact the clinic during working hours, or to seek immediate medical help via the emergency department. At the outpatient clinic, the nurses involved are responsible to check the incoming ePRO reports and AEM on a daily basis. In case of an AEM, the nurses are supposed to contact the patient and to inform the treating physician if considered necessary.

Data collection

Patients’ data was pseudonymized and entered into an electronic database using Microsoft Excel. Baseline parameters included age, gender, duration of disease, treatment protocol (first-line versus treatment of recurrence), and ECOG status.

In the further course, patients were followed up for 6 months. Number and causes of AEM were recorded into the database at monthly intervals. For each AEM, information on the subsequent reaction of the clinic (i.e., text message, phone call, unplanned visit at the outpatient clinic or hospital admission) was obtained. In case of several reactions to one AEM (i.e., phone call followed by a visit at the outpatient clinic), all of them were documented. In cases of reactions referring to more than one AEM (i.e., symptoms connected to each other such as fever and weakness), the respective AEM were summarized.

Patient adherence was defined as the percentage of questionnaires submitted by the patient in relation to the number of questionnaires sent to the patient. For each patient, adherence to the ePRO system was determined both on a monthly interval as well as overall adherence regarding the whole study period.

At the end of the study period, perceptions of both patients and the involved medical staff were assessed in the form of a structured interview. The interview was conducted face-to-face or via telephone following a questionnaire (see Table ​Table2).2). In addition, patients and staff were asked to describe their impressions in their own words. A descriptive analysis of all obtained data was carried out via Microsoft Excel. Percentages are presented rounded without decimal numbers.

A semi-structured literature search was conducted using MEDLINE via PubMed. The chosen search terms were “epro oncology” combined with the terms “implementation,” “adherence,” “benefits,” “barriers,” and “feasibility,” respectively. References of the identified papers were also considered if relevant (“snow balling”). Randomized-controlled trials and implementation studies focused on ePRO implementation in oncology facilities were taken into account and systematically checked for benefits, barriers, feasibility, and perceptions of patients and staff.

Adherence

Of the 11 patients, three patients stopped using the ePRO system in the course of the study period after a mean time of 1.7 months. Reasons were concerns about data security in two cases, although one of these two patients additionally mentioned dissatisfaction with the ePRO system as an additional reason. One patient stopped using the ePRO system after being referred to a different treatment site.

Patients received a median of 4.5 questionnaires per month and a median of 26 questionnaires per patient in the course of the study period. Overall adherence was found at a median level of 95% (per protocol analysis), the median monthly adherence remained on a constant high level, showing no tendency to decline within 6 months (see Table ​Table11 and Fig. ​Fig.22).

Open in a separate window

Fig. 2

Adherence of 8 patients continuing use of ePRO tool until study end

Adverse event messages

Overall, the medical team received 59 AEM; 64% of patients received at least one AEM. The mean number of AEM per month and patient was 0.92, with a range of 0–5.33 alerts per month.

When investigating the causes of AEM, we found pain to be the most common cause, with almost 50%, followed by cardiopulmonary symptoms and gastrointestinal problems (see Fig. ​Fig.33).

Open in a separate window

Fig. 3

a Causes of adverse event messages in percentages (%) b Reactions to adverse event messages in percentages (%)

Following a received AEM, the involved medical staff would in most cases decide to either send a text message or to call the patient (see Fig. ​Fig.3b).3b). Only about 15% of AEM resulted in an unplanned visit at the outpatient clinic and 10% led to hospital admission.

Patients’ point of view

Of the patients, 60% judged the required time investment as adequate, although the number of questions was perceived as too high by 60% of patients. However, 80% of patients were content with usability of the app and 70% reported no or few technical issues. Assistance in using the ePRO system was needed by 30% of patients (see Table ​Table22).

All patients evaluated the automated message as not helpful. Five patients remembered to have been contacted by the nurses or doctors, out of whom three patients thought this was helpful (60%).

In general, patients did not have the impression that using the ePRO tool had a positive impact on their oncological treatment (90%), nor did they feel that their symptoms received more attention by the treating physician (80%). Nevertheless, most patients (60%) stated that they would recommend the ePRO tool to other patients.

When asked for open feedback, most patients gave a positive overall statement. Some mentioned the diary function as helpful, and thought that the ePRO tool gave them an overview of their symptoms and provided them with a sense of security. The option to contact the outpatient clinic via text message was also appreciated.

On the other hand, several patients criticized the redundancy of questions and felt that the answering options did not represent their symptoms accurately. When asked about their overall impressions, two patients answered that it was “a good idea in theory, but has not yet delivered in practice”.

Perceptions of the medical team

Judgement of the medical team was overall negative and differed from patients’ perception in several aspects (see Table ​Table2).2). Usability was rated unsatisfactory by all staff members, and 57.1% (4/7) reported frequent technical problems. In the opinion of 86%, the system did not display patients’ symptoms in a clear structure and did not provide them with an overview; 86% (6/7) complained about an increase of workload.

AEM were evaluated as mostly inappropriate and unreasonable by 86% (6/7), and no positive impact on treatment was perceived (86%). None of the involved staff members agreed that they would recommend the ePRO system to colleagues.

Nevertheless, the medical team still expressed their belief that ePRO systems in general could offer benefits to the oncological care of outpatients (4/7; 57%). In an open question, the following possible benefits were mentioned: easement of communication (4/7), gaining a better overview on patients’ complaints (4/7), earlier detection of severe events (2/7) and prevention of hospital admissions (1/7). In addition, one of the physicians suggested benefits in regard to pandemic situations.

Uptake and adherence

General attitude of patients regarding the ePRO technology is a key factor to successful implementation. In our study, we found the majority of patients agreed to participate in the ePRO procedures. Among patients fulfilling all legibility criteria, only three patients denied participation (3/14, 21.4%), which is consistent with refusal rates found in other ePRO trials (Judson et al. 2013; Wu et al. 2016; Taarnhøj et al. 2020).

However, language barrier caused 11.1% of our patients to be excluded. This topic is rarely addressed in literature, even though it is an often stated exclusion criteria (McCann et al. 2009; Judson et al. 2013; Wu et al. 2016; Basch et al. 2016; Sztankay et al. 2019). If mentioned, language barrier accounts for very few patients to be excluded (Wintner et al. 2015; Klagholz et al. 2018). One possible explanation to this might be disparities in the population structure, with our study site located in a very multicultural area in Berlin. Offering an ePRO system translated to a wider range of languages might be necessary in places with higher cultural diversity, though admittedly this presents a challenge.

Among the eight patients continuing until the end of study we found a high-level adherence without any tendency to decline, but three patients dropped out for reasons related to the ePRO tool. High adherence is congruent with previous studies measuring adherence of cancer patients using ePRO tools (Benze et al. 2019; Friis et al. 2020). In some studies, adherence tended to improve with longer intervals of symptom questionnaires (Judson et al. 2013; Denis et al. 2014). However, the “ideal” intervals might depend on the respective setting. For example, Denis et al. found higher monthly adherence among lung-cancer patients in a follow-up situation, using ePRO as a tool for earlier detection of relapse. In this context, close monitoring of severe toxicities is no longer necessary, and patients might prefer not to be reminded of their cancer more than necessary.

Overall, both recruitment and adherence of patients seemed feasible at our facility, but we also identified several barriers that limited general use (e.g., language and lack of IT). Moreover, patients expressed satisfaction with usability of the application, appreciated having a better overview of their symptoms and felt reassured. Again, these findings are similar to previous research, receiving almost entirely positive feedback from patients (McCann et al. 2009; Denis et al. 2014; Wintner et al. 2015; Wu et al. 2016; van Eenbergen et al. 2019). In conclusion, these findings lead to the impression that barriers of ePRO implementation are not primarily to be found on the patients’ side, but lack of perceived benefit certainly jeopardizes patient commitment needed for this tool.

We express our gratitude to the cancer society of Berlin (Berliner Krebsgesellschaft) for funding this study. In addition, we would like to thank all participating patients, doctors, and nurses for their participation and engagement.