Abstract

Background

Patient-reported outcomes (PRO) regarding symptoms and quality of life (QOL) have become an important part of cancer clinical trials and routine care, as there is strong evidence that clinicians’ ratings alone do not present a complete picture of cancer patients’ symptom burdens [1–5]. Studies emphasise the additional benefit of the use of PROs (improved symptom management [6], better identification of intimate or psychosocial problems [7], enhanced communication between health care professionals and patients [7–9]) and the administration of validated questionnaires via specialized software helps to overcome common implementation barriers to the routine use of PROs (e.g., administrative obstacles, concerns that electronic assessments might use more resources than they save) [10, 11]. Electronic PRO (ePRO) assessment is generally well accepted by patients [6, 12] can be conducted in an efficient and timely manner, and both in clinical settings and in patients’ homes. Web-based home monitoring offers insight into the “black box” of a patient’s condition between hospital visits, a parameter that is systematically overlooked by QOL assessment restricted to the hospital setting. The large number of by now available specialized software solutions for ePRO assessment, predominantly developed within English speaking countries and offering a variety of data collection and educational features [13, 14], mirrors the growing interest in ePRO assessments. However, ePRO assessments require a certain level of patients’ computer literacy; therefore, patients’ habits and attitudes regarding the use of new technologies (devices and technologies, which can be summed up under the umbrella term “new media”, e.g., computers, smart phones, and the internet) are particularly important. Devices for internet access and internet infrastructure are becoming increasingly affordable and easy to manipulate. Nevertheless, some groups (e.g., older, computer-illiterate, or socioeconomically disadvantaged patients) might be put at a disadvantage when data collection is conducted mainly via new media (i.e., computer-based assessments via internet-ready devices), as familiarity with such devices seems to play an important role for implementation of ePRO and patient recruitment [15, 16]. Though, available statistics suggest that actual existing problems with lacking familiarity with or open-mindedness to computer and internet-ready devices might vanish in the course of time. For instance in the Austrian general population in 2014, about 40 % of persons older than 65 years access the internet on a regular basis (a slightly more than tenfold increase since 2002), roughly a third of these using mobile devices [17]. Nearly as many cancer patients aged 66 to 99 years reported the availability of their own internet connection and at least occasional use of the internet [12]. Given the actual technical development, this number can be expected to further increase. Nevertheless, the feasibility and user-friendliness of used software solutions for ePRO assessment need to be evaluated for optimization of applied systems.

Methods

Sample and procedure

Cancer patients at various departments of Medical University of Innsbruck (urology, neurology, nuclear medicine, internal medicine) and at the Department of Internal Medicine at Kufstein County Hospital were approached before their consultation or treatment application by a study nurse, who explained participation in the electronic QOL assessment (at the hospital or at home; see Fig. 1 for a flow chart of the procedure), and obtained patients’ written informed consent. The requirements for patients’ participation included minimum age of 18 years, diagnosis of oncologic disease, no severe cognitive impairments, and German-speaking. The study was approved by the ethics committee of Medical University of Innsbruck.

Open in a separate window

Fig. 1

Flow chart describing procedure for clinic-ePRO and home-ePRO

Electronic data capture

The Computer-based Health Evaluation System (CHES) is a specialized software for electronic PRO data collection, result calculation, and presentation for use in clinical practice as well as for merging clinical and PRO data for research purposes [18]. We used CHES for both clinic-ePRO in the hospital setting and home-ePRO in patients’ homes; the assessment procedure is outlined above. A study nurse gathered sociodemographic and clinical data from the hospital records and entered them in CHES to link with associated QOL data stored in the CHES database.

The CHES software works with a database stored at each particular hospital server, which patients and physicians access by different means. Physicians can use CHES via their work desktop (for display of patient data as longitudinal or cross-sectional reports, preparation of individual questionnaire sets, and/or making data available for patients online). Patients are equipped with login data for access to questionnaires via a website or an iOS app (see Figs. 1 and ​and2).2). Consequently, both physicians and patients use the same software but are accessing different features. For a more detailed description of the features of CHES, please refer to Holzner et al. [18].

Open in a separate window

Fig. 2

Screens for login to the CHES home-ePRO (left) and for questionnaire completion (right)

Results

Patient characteristics

Clinic-ePRO patients

At Medical University of Innsbruck, between February and September 2012, 113 patients (mean age 45.1 years, SD 14.4, range 22 to 81), most of whom (88.5 %) were attending an outpatient follow-up appointment, completed the evaluation form directly after clinic-ePRO assessment (see Table 1). Most common diagnoses were testicular cancer (67 %), glioma (15.7 %) and neuroendocrine tumours (12.8 %). Data on the number of patients approached for evaluation completion are not available.

Table 1

Patient characteristics

		Clinic-ePRO at the hospital	Home-ePRO at home
		(N=113)	(N=45)
Age	mean (SD, range)	45.1 years	58.7 years
		(SD 14.4, 22 to 81)	(SD 10.4, 29 to 74)
Sex	male	83.2 %	48.9 %
Diagnosis	gastrointestinal tumours	0.9 %	31.2 %
	glioma	15.7 %	-
	gynaecological tumours	0.9 %	33.3 %
	lung cancer	0.9 %	22.2 %
	neuroendocrine tumours	12.8 %	-
	testicular cancer	67.0 %	-
	other	1.8 %	13.3 %
Treatment	chemotherapy	5.3 %	100 %
	follow-up	88.5 %	-
	other	6.2 %	-
Device	iPad2	-	71.9 %
	own laptop or PC		28.1 %
Institution		Medical University of Innsbruck	Kufstein County Hospital

Open in a separate window

Home-ePRO patients

Of the 166 patients approached for participation at Kufstein County Hospital, 60 declined routine ePRO assessments at home; 55 chose autonomous data entry via the internet (home-ePRO; inclusion rate of 33.1 %); and 51 chose weekly phone calls. Owing to the particular nature of the phone interviews, these were excluded from evaluation. Regarding age, the two samples of refusing patients and those who chose phone calls were comparable but significantly older than patients participating in home-ePRO (F=13.642, p<0.001, 68.7/67.6 years [SD 10.5/9.2] versus 58.7 years [SD 10.7]). The groups did not differ in terms of patients’ sex, though a tendency that more men rejected ePRO and more women chose phone calls (χ2=4.778, p=0.092) became apparent.

The most frequent reason for refusal of QOL monitoring was skepticism about QOL assessment itself or reservations about the electronic device (46.6 %). Other reasons were exclusive agreement to clinic-ePRO at the hospital (25 %), poor general condition (13.3 %), logistical problems or errors in implementing protocol (6.7 %), speech problems (3.3 %), and in some rare cases, language barriers, technical barriers, or treatment discontinuation (1.7 % each) (see Fig. 3).

Open in a separate window

Fig. 3

Patient flow for home-ePRO evaluation

Of 55 patients participating in home-ePRO, four (7.2 %) died before evaluation, two (3.6 %) refused to evaluate the home-ePRO system, and four (7.2 %) did not respond to repeated mailings. In total, between March and August 2013, 45 patients (inclusion rate: 81.8 %, 27.1 % of all eligible patients) completed the home-ePRO evaluation form. The sex ratio was nearly balanced (51.1 % female); mean age was 58.7 years (SD 10.4, range 29 to 74), and pathology types were gynecological (33.3 %), gastrointestinal (31.2 %), lung (22.2 %), and lymphoma (13.3 %). The most frequent assessment mode was the iOS app (71.9 %). Detailed information on patient characteristics is shown in Table 1.

Patients’ basic habits regarding their personal use of the internet

Clinic-ePRO patients

Four fifth (81.4 %) of the 113 clinic-ePRO patients stated to use the internet at home, most of them several times per week (66.4 %). Laptops (53.1 %), PCs (50.4 %) and smart phones (45.1 %) were the most often used internet-ready devices. Tablet PCs were used by only one fourth of patients (26.5 %) (multiple answers allowed for each patient). Nearly one fifth (18.6 %) stated not to pursue any internet activity (i.e. no internet use at all) and 11.5 % chose at least one activity. The majority of patients were quite familiar with the internet, as 28.3 % chose two to three activities and 41.6 % reported to use the internet for four or more activities. Reading and writing e-mails (82.2 %) was the by far most common activity on the internet, next to searching the internet for information (74.3 %), online banking or booking (72.6 %), entertainment (67.1 %), online shopping (45.1 %), and social media (40.7 %) (multiple answers allowed for each patient) (refer to Table 2). Approximately 8 % used the additional open answer to note that they use the internet for business matters.

Table 2

Patients’ basic habits regarding their personal use of the internet

		Agreement (%)
		Clinic-ePRO	Home-ePRO
		N=113	N=45
Personal internet use	yes	81.4	77.8
Frequency of internet usea	several times per week	66.4	51.1
	several times per month	7.1	15.6
	less than once per month	8.8	6.7
	no internet use	15.0	20.0
	missing	2.7	2,2
Used internet-ready devicec	laptop	53.1	51.1
	personal computer	50.4	35.6
	smart phone	45.1	8.9
	tablet-PC	26.5	20.0
Internet activitiesc	information search	74.3	71.1
	reading and writing e-mailsb	82.2	53.3
	online shopping	45.1	31.1
	online banking/bookingb	72.6	31.1
	entertainmentb	67.1	28.9
	social media	40.7	15.6
Number of pursued internet activities	no internet activities	18.6	17.8
	1 activity	11.5	17.8
	2-3 activities	28.3	28.9
	4 or more activities	41.6	33.3

Open in a separate window

^a N=152, as data from 6 patients is missing

^b N=117, as data from 41 patients is missing

^cMultiple answers allowed for each patient

Home-ePRO patients

Nearly four fifth (77.8 %) of the 45 home-ePRO patients patients stated to use the internet at home, about the half of them (51.1 %) several times per week. The most often used internet-ready device was laptops (51.1 %), followed by PCs (35.6 %) and tablet PCs (20.0 %). Smart phones were the least commonly used device for internet-access (8.9 %). Respectively almost 18 % of patients stated not to pursue any internet activity (i.e. no internet use at all) or at least one activity. The majority of home-ePRO patients were quite familiar with the internet, as 28.9 % chose two to three activities and 33.3 % reported to use the internet for four or more activities. Searching the internet for information (71.1 %) was the by far most common activity on the internet, next to reading and writing e-mails (53.3 %), online shopping and online banking or booking (31.1 %, respectively), entertainment (28.9 %) and social media (15.6 %) (multiple answers allowed for each patient) (refer to Table 2). Just a few home-ePRO patients (4.4 %) used the additional open answer to note that they use the internet for business matters.

Patients’ attitudes towards clinic-ePRO/home-ePRO

Clinic-ePRO patients

Approximately two thirds of clinic-ePRO patients (64.6 %) indicated that they viewed PROs as a useful and adequate method to provide information on their QOL to their attending physicians. Nearly as many patients (61.1 %) were interested in seeing their own results. The vast majority of patients (94.7 %) would further participate in routine PRO assessment at the hospital and 68.1 % would opt for home-ePRO assessments. Of the latter, 43.4 % considered an annual interval to be sufficient. Most clinic-ePRO patients said they preferred electronic data entry for future completion of PRO measures; 23.9 % did not express a preference and seemed to be open-minded about clinic-ePRO (refer to Table 3).

Table 3

Patients’ attitude towards clinic-ePRO/home-ePRO

		Agreement (%)
		Clinic-ePRO	Home-ePRO
		N=113	N=45
Considering PROs as an useful and adequate method to provide information on QOL to physicians		64.6	91.1
Wish to see own PRO results		61.1	77.8
Wish to discuss PRO results with treating physiciana		-	82.2
Willingness to complete QLQ-C30	at the hospital	94.7	84.4
	at homeb	68.1	-
For patients reasonable time frames for PRO completion at home	at least weekly	14.2	53.3
	monthly	18.6	24.4
	yearly	43.4	13.3
	missing	23.8	8.9
Preference of phone interview over home-ePROc	no	97.3	93.2
Preferred mode of PRO assessment	paper-pencil	7.1	17.8
	electronic	67.2	60.0
	no preference	23.9	20.0
	decline of assessment	1.8	2.2

Open in a separate window

^a N=45 patients at the Kufstein County Hospital

^b N=113 patients at Medical University of Innsbruck

^c N=118 patients (73 patients of Medical University of Innsbruck and 45 patients of the Department of Internal Medicine, Kufstein County Hospital)

Home-ePRO patients

The vast majority of home-ePRO patients (91.1 %) indicated that they viewed PROs as a useful and adequate method to provide information on their QOL to their attending physicians. There was apparently great interest in incorporating PRO data into the clinician-patient communication: 82.2 % of home-ePRO patients would appreciate a discussion of their results with their attending physician. Slightly more than half (53.3 %) of home-ePRO patients considered at least a weekly assessment interval for home monitoring reasonable; one fourth (24.4 %) stated a monthly interval to be sufficient. Electronic PRO assessment was preferred by 60 % over paper-pencil assessments. Those without a preference (20 %) might be open-minded towards an electronic assessment mode. Only a minority of home-ePRO patients (6.8 %) would prefer a regular phone call to autonomous data entry (refer to Table 3).

Evaluation of the clinic-ePRO procedure

One hundred thirteen patients completed an evaluation form regarding the clinic-ePRO assessments conducted at Medical University of Innsbruck. Of these, 82.3 % reported no problems in questionnaire completion and 92.8 % experienced no difficulties in device handling. Only one patient had many problems and needed further assistance, which was provided by an accompanying relative. The large majority (91 %) of patients indicated that they were very satisfied with the graphic display of the questionnaire on the tablet screen. Only one suggestion for an improved design of the user interface was made, which was to include adjustable font sizes. Other ideas referred to construction of the response scale. Most patients (86.6 %) were highly satisfied with the level of privacy during PRO completion, and most (58.9 %) agreed that the always-available nature of clinic-ePRO was an advantage. The most frequently perceived disadvantages of clinic-ePRO were being too impersonal (57.5 %) and not allowing for individual situations (30.1 %). Technical issues and complexity were only rarely cited as a drawback (4.1 % combined) (refer to Table 4). One patient added a comment regarding data security concerns. Two patients, who agreed with the previously mentioned disadvantage of clinic-ePRO’s being too impersonal, made notes that paraphrased this answer category.

Table 4

Patients’ rating of feasibility of clinic-ePRO/home-ePRO

		Agreement (%)
		Clinic-ePRO	Home-ePRO
		N=113	N=45
Did you have any difficulties completing the questionnaire?	not at all	82.3	97.8
	a little	14.2	2.2
	quite a bit	3.5	0.0
Did you have any difficulties handling the electronic device?	not at all	92.8	88.9
	a little	3.6	8.9
	quite a bit	2.7	2.2
	very much	0.9	0.0
Did you have any difficulties starting the app or entering your login data?b	not at all	-	72.7
	a little		27.3
How satisfied are you with the presentation of the questionnaire on the screen?	very much	91.0	95.5
	quite a bit	5.4	2.3
	not at all	3.6	2.3
How satisfied are you with the privacy during questionnaire completion?a	very much	86.6	-
	quite a bit	6.2
	a little	2.7
	not at all	4.5
		Agreement (%)
		Clinic-ePRO	Home-ePRO
		N=113	N=45
perceived advantages	no personal contact needed	11.0	6.7
	no drive to the hospital neededb	-	42.2
	availability of questionnaire at any time	58.9	44.4
	low cost	31.5	24.4
	feeling of being well cared for at homeb	-	57.8
perceived disadvantages	handling of the internet is so difficult	0.0	1.4
	the technical requirements are too extensive	1.4	2.2
	does not come up to my individual situation	30.1	8.9
	too complex/confusing	2.7	0.0
	too impersonal	57.5	13.3
Overall, how satisfied are you with electronic quality of life assessment at home?b		Ø (SD, range)	9.1 (1.14, 5–10)

Open in a separate window

^a N=113 patients at the Medical University of Innsbruck

^b N=45 patients at the Kufstein County Hospital

Discussion

This study reports on an evaluation of routine PRO assessments using a specialized software within two contexts: first, as an assessment of patients’ QOL before a follow-up/treatment appointment in a hospital setting (clinic-ePRO), and second, as regular monitoring of the symptom burden of chemotherapy outpatients in the home setting (home-ePRO).

The patients included in our study were relatively high experienced in internet use, in general open-minded towards ePRO assessments, rated them to be useful and would continue regular QOL-monitoring. CHES was well accepted and offered a user-friendly as well as feasible system for different assessment settings. In spite of this overall positive feedback, some previously identified implementation barriers of routine ePRO were replicated, and new ideas for further research areas could be generated.

As disclosed by both our study and the available literature, many important issues for successful implementation of ePRO in the routine clinical setting and at patients’ home can be identified. Efforts are needed not only to improve infrastructure for internet access, but also to encourage patients to use the internet for obtaining medical information and transmitting their health status and QOL information to providers [21]. Furthermore, to echo Rose and Bezjak [22], the person tasked with explaining the concept and implementation of QOL assessments, and instruments like home-ePRO in particular, is crucially important. Older people are particularly prone to have reservations concerning modern computer technology and need to be properly approached. This fact was demonstrated in our study when the most frequently cited reason for refusal of regular QOL-assessment was concern regarding the handling of an internet-enabled device. Simply providing user-friendly devices is not enough; older and/or computer-illiterate patients also need opportunities to familiarize themselves with the devices. Robben et al. [23] suggest another aspect that should be carefully considered, particularly among older patients: Aside from specific stand-alone training programs (such as heart-healthy eating or stress prevention strategies), web-based health applications should not be introduced without general information about the supplemental nature of these electronic tools. Fear of losing personal contact with health care professionals might deter patients from participating in home-ePRO, even if they are generally able to handle the necessary technology. Especially clinic-ePRO patients perceived the electronic assessments to be too impersonal and not coming up to their personal situation. Though we do not know how ePRO data was used by the clinicians of the Medical University of Innsbruck (it was available on their desktop PC), one might hypothesize that this negative perception of clinic-ePRO might relate to an insufficient integration of assessed data into the medical consultation. Further research needs to assess the use of ePRO data of health care professionals within the medical consultation and how this might influence patients’ perception of PROs.

As in other studies, most of our patients appreciated clinic-ePRO and were favorable to its extension from the hospital setting into their homes. Similarly to the results of Broering et al. [24], between 60 (home-ePRO) and 67 % (clinic-ePRO) of our patients stated a preference for an electronic mode of data collection for future PRO assessments. In addition, the vast majority of clinic-ePRO as well as home-ePRO patients (at least 97 per cent) did not perceive the assessment as too complex, difficult or the technical requirements as too extensive. Nonetheless, the phenomenon of a digital divide can be seen, especially for older patients, as patients who refused ePRO assessments at the Kufstein County Hospital or chose phone calls over home-ePRO were approximately 10 years older than home-ePRO participants. Those patients, who like to deal with new technologies and the internet, therefore, have a clear advantage over those who are not familiar with new media. Patients’ socioeconomic status may be of importance as well, and although technical progress continues to bring increasing affordability, future studies should also attend to this issue.

Some limitations of the study should be mentioned, as those restrict the generalizability of the reported results. First, some patients may have been especially motivated to participate in home-ePRO because of the provision of the iPad2, including the preinstalled iOS app. Because completion of the questionnaire was slightly more laborious via a web browser, one might hypothesize that without loaner devices, inclusion rates would have been lower. Second, an age bias emerged as older patients declined participation or preferred regular phone interviews to home-ePRO via an iPad2 or a website. It may be reasonable to assume that those patients also differed in terms of available internet equipment, user experience, and affinity for new media. Therefore, a digital divide may have put those patients at a disadvantage and their higher refusal rates may have whitewashed the overall study results, as only one third of all eligible patients agreed to use home-ePRO. Third, though the number of eligible patients and reasons for refusal were in detail assessed at Kufstein County Hospital, this was not the case at Medical University of Innsbruck. Consequently, we neither know how many patients were approached to participate in clinic-ePRO and its evaluation nor why they did not want to provide PRO-data or to evaluate the used ePRO software. Fourth, it should be noted that the clinic-ePRO group predominantly comprised a large number of relatively healthy and young testicular cancer patients. Those patients represent a quite specific sample of cancer patients, whose attitudes and opinions might differ from a mixed cancer patient sample undergoing active treatment. Overall, the results of our study primarily are based on data provided by patients, who seem to be especially open to ePRO from the outset. Further evaluation of CHES within a more heterogeneous sample of cancer patients should to be done.

Conclusions

In line with other study results, our patients well accepted clinic-ePRO and home-ePRO and experienced it to be feasible. However, we must bear in mind that many patients declined to participate owing to computer illiteracy or worries about their own capability to handle the assessment device. To be more inclusive in the implementation of patient monitoring tools in daily clinical practice, the medical community will need to address several persistent doubts. Physicians as well as patients need further information about the complementary nature of PROs, which cannot replace the conventional medical consultation, but help to detect disturbing symptoms earlier, thereby allowing better structuring and more effective use of time in face-to-face encounters between patients and providers.

Acknowledgments

Footnotes

Contributor Information

L. M. Wintner, Email: [email protected].

J. M. Giesinger, Email: [email protected].

A. Zabernigg, Email: [email protected].

G. Rumpold, Email: [email protected].

M. Sztankay, Email: [email protected].

A. S. Oberguggenberger, Email: [email protected].

E. M. Gamper, Email: [email protected].

B. Holzner, Email: [email protected].

References