Participants

All researchers involved in the PULsE-AI trial were invited to participate in the reflection exercise. Invited researchers included members of the core trial team/Trial Steering Committee (TSC) (including trial logistics personnel, clinical experts, scientific advisors, and industry sponsors) and general practice-based Principal Investigators (PIs) and their research teams. Practices were reimbursed financially for their time to undertake the reflection exercise.

Reflection exercise

Approximately 17 months following completion of the PULsE-AI trial, eligible participants were invited to participate in the qualitative reflection exercise. Members of the TSC were invited to attend a semi-structured focus group session, and PIs and their research teams were invited to attend a semi-structured interview. This mixed approach was taken because by working together throughout the trial the TSC was closely aligned, whereas PIs and their research teams at individual practices had different experiences of the trial. The focus group and interview sessions were led by one of the trial researchers; this approach enabled the interviewer to probe for additional detail where required, but their opinions were not captured as part of the exercise. The focus group and interview questions are included in Supplementary File S1. All participants provided verbal informed consent prior to participating in the interview/focus group sessions for this reflection exercise. All focus group and interview sessions were undertaken virtually via Microsoft Teams® and recorded and transcribed using the platform’s inbuilt software. All transcriptions were checked for accuracy and edited where required. Ethical approval was granted by the Wales Research Ethics Committee 5 and the study was approved by the Health Research Authority (HRA) and Health and Care Research Wales (HCRW) (IRAS project ID: 252,934).

Recruitment

Recommendations to improve response rates

As a result of the reflection exercise, there were a number of recommendations for suggested ways to improve response rates in primary care trials (Fig. 2).

Open in a separate window

Fig. 2

Summary of recommendations to improve participant response rates to trials in primary care

Challenges

General recommendations

The main recommendations suggested by participants in this reflection exercise were based on the following themes:

Better communication between parties, including more face-to-face support for practices from the core trial team, and therefore better development of rapport/relationships:

I feel like I had more engagement with those practices where I went to the …site initiation visit, because you knew who you were talking to, you could picture their face, you could picture their setup, …that, sort of, helped those relationships and conversations. … I wish we’d … spent more time with our practice links, understanding how they updated their systems, recorded things, because there was a lot of to-ing and fro-ing … and we needed to find out all of these, like, missing parts of the jigsaw around practice operationalization, … that was integral …to track the progress of the practices and results.

Clarity around roles and responsibilities, particularly when there are multiple parties involved in the trial:

A clear delegation of responsibilities right from the start might have been one way to facilitate that, the expectations around who’s doing what, where, and when sort of thing.

Better utilisation of online-based tools for the sharing of information such as trial enrolment/completion logs:

If we were doing it again, we’d use a lot more online methodologies, and, you know, we’d have shared tools for completion, rather than this sheet we were emailing each other back and forth. Technology has moved on to enable, we’d hope, more efficient processes if we were to do it again.

Improved guidance and support for new methods/technologies. Technology support was one of the key challenges encountered by the practices, therefore methods to better support researchers are a key recommendation:

The hardest bit I found was the technical side, getting the …device set up, getting them reading [ECGs], … when we had the site initiation visit someone did go through it with us, it, was just perhaps um, you know, you could have a video, a YouTube video, or you know, some more pointers or advice, because people do forget.

Incentivisation for practices (to ensure there is someone with primary responsibility within the practice). At the restart of the trial (July 2020) we implemented a ‘Study Champion’ contract with each practice with a £1,000 payment attached:

Once we implemented the practice champion for the study, that really helped things as well, that was incentivised… because there was a payment that came with that. But it did put somebody, a named person responsible that we could fall back on, and then the practice could lean on in order to answer our questions … so maybe including that a little bit earlier in such a study again, would be good.

But also, incentivisation and/or reimbursement for participants to ensure they aren’t ‘out of pocket’, doing so ensures the research is:

“Conducted equitably, in terms of, rural and social factors, and deprivation, you know, in terms of inequality … that may be difficult to do, but there should be due attention given to that collection of data [ECGs], where it as equitable as possible, in terms of access to primary care.”

Recruitment

On reflection, the majority of researchers felt that the response rates (of 28%) were similar to what they would have expected for a trial in primary care. Similar trials in primary care settings have reported response rates ranging from 8–33%, [17–20] which generally accord with what we observed. However, at the beginning of the trial, initial response rates were relatively low, resulting in somewhat unexpected recruitment challenges. Consequently, we added additional interventions, such as a phone call to follow up on recruitment letters and a ‘pop up’ within the medical records of patients randomised to the intervention arm and identified as high risk of undiagnosed AF to prompt a conversation about the study during a consultation, in an attempt to facilitate recruitment. These interventions appeared to result in a small uplift in participants consenting, and analysis from a recent Cochrane Review on strategies to improve recruitment indicates this uplift can be in the region of 6% [5].

The quality of communication of the value of participating in a study is also an important consideration for maximising response rates. The exact methods will likely depend on the trial population of interest; online-based methodologies such as email campaigns and social media are likely to be more effective for younger participants, whereas older generations are likely to benefit from a mixed approach to recruitment to reflect variations in technology use amongst this cohort. Irrespective of method of recruitment utilised, ensuring approaches are targeted and materials are accessible to the cohort of interest may help improve response rates. Furthermore, ensuring trials are accessible, equitable, and inclusive is also essential to ensure that study populations are more reflective of patient populations and therefore trial outcomes are more widely applicable [21, 22]. The use of recruitment approaches that are more likely to resonate with minority groups are required to encourage greater equality and reduced bias in trial participation.

In the PULsE-AI trial, the smallest practice had the highest response rate to invitation. Good clinician-patient relationships are important to this practice and potentially highlight an important link between relationships and the ability of practice teams to encourage participation in trials in this setting. Overall, no one strategy emerged that, if implemented, will have a impact on response rates, but likely an amalgamation of multiple small changes or interventions that, when combined, will potentially maximise participant responses.

Incentivisation

Incentivisation and reimbursement of time and/or expenses will likely play a greater role in research in the future. Our researchers commented that we rely on goodwill a lot in medical research and they believe that younger generations in the future are unlikely to participate in medical research at the same rate as older generations today, a trend also observed in blood donation [23]. However, there are important ethical factors to consider when offering incentives, particularly that the incentive may reduce an individual’s perception of the risk of participating in the trial, therefore preventing them from providing fully informed consent [24, 25]. Whether incentivisation actually increases recruitment is not well understood; a study reporting on two trials found incentivisation only boosted recruitment in one of the two trials [26]. Ultimately, incentivisation is a subject that could be considered during protocol design, especially in trials where researchers foresee challenges with recruitment. Reimbursement for travel expenses incurred may also support the move to ensure research is conducted equitably and potential participants in more socially deprived and/or rural areas are not dissuaded from participation due to financial or logistical limitations.

The use of home-based ECG monitoring technology

One of the key challenges encountered by researchers in the trial was the technology. The KardiaMobile device used for home-based ECG monitoring requires a compatible smartphone or tablet to record data. Although newer technologies are now available that remove the need for a compatible smart device, alternative options were unavailable at the time. The mean age of participants in the trial identified as at high risk of undiagnosed AF was 78 years, and consequently, many participants did not have access to or lacked the technological confidence to use a smartphone/tablet. There were also additional complexities (beyond standard use of the device) relating to the need for trial data to be uploaded to secure servers for later cardiologist review. The demographics of the trial population combined with these additional complexities placed substantial burden on researchers, who would have appreciated greater support (e.g. video demonstration in addition to written instructions) in managing this component of the trial. As researchers, it is important to remember that practices involved in research are usually doing so in addition to their normal clinical workload, and therefore any support that can be provided to make learning and implementation of new processes simpler may facilitate greater practice engagement with the trial. Furthermore, a pilot study could be undertaken to try and identify any potential issues – such as the issue we encountered with a number of participants not having access to a compatible smartphone/tablet – early and allow time to explore alternative options if suitable.

Trial operations

Given the importance of the time element in our trial, and the need to implement the intervention as quickly as possible to detect undiagnosed AF, if we were to undertake the trial again, we would ensure all practices are set up and ready to go concurrently. Whilst this may be impractical for larger trials with multiple centres, where possible, we recommend ensuring all participating practices are identified and contracted with, well ahead of undertaking site initiation visits. Once all practices are ready to begin, the trial can be initiated, and early weeks are not ‘lost’ to these set-up activities. Whilst it is impossible to plan for all eventualities, when designing trials where time can impact the primary outcome, there may be value in upfront planning where possible in how to manage potential challenges such as slow recruitment.

Communication is also a key component of trial operational success. Where possible, face-to-face contact between the core trial team and practices early in the trial is beneficial for developing relationships. Feedback from one researcher (from the core trial team) was they felt they had a better rapport and a more open channel of communication with those practices in which they attended the site initiation visit. A lot of this required communication between the core trial team and practices related to weekly recruitment updates. At the time, we were using a spreadsheet that was emailed back and forth between researchers and practices. Some practices found this method frustrating, and therefore, if we were to undertake the trial again, we would likely explore the use of a secure web-based platform for the sharing of this information to reduce the burden on the practice research teams.

One event we were unable to predict at the beginning of the PULsE-AI trial was the COVID-19 pandemic and the eventual ‘stay-at-home’ message disseminated by the UK government leading to our trial being paused. During this pause, participants in both intervention and control arms were being diagnosed with AF through routine care, but we were unable to actively recruit or screen any intervention arm participants. On reflection, we could have considered amending the protocol and splitting the intervention arm into two separate cohorts, the first based on all participants who had undergone the screening intervention up to the time at which the ‘stay-at-home’ message was declared, and the other to be a longer-term cohort able to be ‘picked up’ again when research activities were able to resume.

The authors wish to thank the following principal investigators and research nurses for their significant contributions to data collection in the PULsE-AI trial that led to this publication: Dr Andrew Wallis and Terri-Anne Thomson (Alcester Health Centre); Dr Manjit Kainth and Jacqueline Woodward (Primrose Lane Surgery); Dr Paul Ainsworth and Oliva Neely (Sherbourne Medical Centre); Dr Caron Morton (Station Drive Surgery); Dr Claire Jones (Spring Gardens Group Medical Practice); Dr Stefan Lachowicz and Deborah Williams (The Caxton Surgery); and Sian Jones and Jonathan Davies (National Institute for Health Research).

The authors also wish to thank Catherine Jeapes of Health Economics and Outcomes Research (HEOR) Ltd for her contributions to transcription and analysis. HEOR Ltd were paid consultants to Bristol Myers Squibb Pharmaceutical Ltd & Pfizer Inc. in connection with the development of this manuscript.