Abstract

The Decision to Study Severe CAP

We chose severe CAP because it is extremely common, case fatality is high, the strength of evidence guiding treatments is limited, and there is substantial variation in care. Worldwide, CAP remains one of the largest contributors to death and disability-adjusted life-years lost in rich and poor countries alike (4–6). Severe CAP, the subset at risk for acute hypoxemic respiratory failure and shock, is also the most common cause of sepsis, a frequent reason for ICU admission, with a mortality rate of 20–50% (7–9). Finally, viral pneumonia, especially influenza, is the most deadly recurring pandemic infection (4).

The treatment of severe CAP involves multiple therapies, including antimicrobial regimes, host immunomodulation, organ support, and interventions to prevent complications. Several guidelines address severe CAP treatment, but the specific recommendations frequently lack strong evidence. For example, high-quality evidence from RCTs supports only 4 of 44 recommendations in current European guidelines (10–12), 11 of 43 in U.S. guidelines (13), and 7 of 93 Surviving Sepsis Campaign guidelines (14). Furthermore, several statements are contradictory across guidelines. Not surprisingly, guideline compliance is poor and care is variable (15–18), with potentially adverse consequences (18, 19).

Challenges to the Generation of Robust and Useful Evidence for Severe CAP

Two issues hinder generation of high-quality evidence for care of patients with severe CAP. First, for endemic CAP, the effectiveness of interventions may vary by subgroups or use of concomitant treatments. For example, hydrocortisone effectiveness may vary by etiology of CAP (viral or bacterial), presence of shock, and choice of antimicrobial therapy. Traditional RCT designs are not well suited for assessing complex treatment–treatment and treatment–subgroup interactions. Second, RCTs launched in pandemics, such as the 2009 H1N1 influenza or coronavirus disease 2019 (COVID-19) pneumonia outbreaks, even when using “just-in-time” procedures, are often implemented too slowly to generate useful knowledge (3, 20).

A New Approach

Our solution for better evidence generation in severe CAP, the REMAP, combines two designs: a point-of-care RCT and an adaptive platform trial (2, 21, 22). Point-of-care RCTs boost capture of eligible patients via a clinical moment, or point of care, that triggers the trial apparatus (23, 24), ideally in the electronic health record (21). This approach is used for pragmatic comparative effectiveness studies (25, 26). Rather than testing individual interventions in a single homogeneous disease state and terminating when that task is complete, adaptive platform trials focus on a broader set of disease states and test multiple therapies simultaneously and sequentially (22, 27, 28). They are thus an experimental platform, rather than a series of experiments. They are adaptive in that they incorporate rules for changes in entry criteria, study arms, and the proportion randomized to each arm over time. There are several adaptive platform trials outside critical care (29, 30).

Description of the REMAP Design

REMAP combines a point-of-care RCT and an adaptive platform trial to create a design that, like the former, embeds the trigger for patient recruitment in routine clinical care, but, like the latter, then enrolls these patients into a platform capable of addressing complex study questions regarding multiple therapies in multiple subsets of patients (Figure 1) (2). Embedding the trial promotes capture of the greatest number of patients, which is key to generalizability, arguably essential for response to pandemics that “wave” rapidly through different regions (31), and efficient. Embedding also facilitates low operational complexity at the bedside, even though the internal clinical trial machinery may be complex. Thus, with REMAP-CAP, any patient admitted to the ICU with acute respiratory insufficiency due to suspected pneumonia is flagged for enrollment and randomization. Ideally, all eligible patients will be enrolled, generating an automatic custom order sheet relating to all the intervention assignments. Other aspects of the trial, such as ongoing monitoring and data collection, will also be embedded where possible in routine care. The trial design also coordinates with national ICU registries to permit comparison with unenrolled patients and avoid data collection redundancy (Appendix).

Open in a separate window

Figure 1.

Schematic of the REMAP-CAP (Randomized Embedded Multifactorial Adaptive Platform for Community-acquired Pneumonia) design. CAP=community-acquired pneumonia; DSMB=Data Safety Monitoring Board; EHR=electronic health record; ICU=intensive care unit; RCCs = regional coordinating centers; SAC=statistical analysis committee.

The trial is “multifactorial” in that it tests multiple interventions within multiple therapeutic domains and multiple patient strata (Table 1). In REMAP-CAP, the initial interventions are grouped under four domains (an antimicrobial domain consisting of four alternative antibiotic strategies and two host immunomodulation domains [one testing alternative hydrocortisone dosing regimens, one testing use of extended macrolide therapy] and one evaluating antiviral therapy). Domains relating to oxygen therapy and respiratory support strategies will be added, and pandemic COVID-19–specific domains are also now launched (described in the section Adaptation during a Pandemic). Each patient is randomly assigned a specific intervention within each domain; the set of assigned interventions defines the treatment regimen. The strata are patient characteristics identifiable at enrolment for which a differential effect on outcome by intervention is hypothesized. REMAP-CAP commenced with two strata: presence or absence of shock and presence or absence of suspected (or proven) influenza infection.

Table 1.

Summary of REMAP-CAP (Randomized Embedded Multifactorial Adaptive Platform for Community-acquired Pneumonia) features

Feature	Description
Patients
Entry criteria
Inclusion criteria	• Admitted to ICU within 48 h of hospital admission
	• Age ≥18 yr
	• CAP by clinical and radiologic criteria
	• Requiring respiratory (non-invasive or invasive ventilation) or cardiovascular (inotropes/vasopressors) support
Exclusion criteria	• Healthcare-associated pneumonia
	• Imminent death and no commitment to full active treatment
	• Prior enrollment in REMAP-CAP in the last 90 d
Stratum
Definition	A patient characteristic defined at enrollment used for the generation of specific treatment estimates
Starting strata	• Presence of shock or not (defined as hypotension or vasopressor requirement after volume resuscitation)
	• Presence of suspected or proven influenza infection or not
State
Definition	A clinical state that triggers a specific domain
Example	Mechanical ventilation
Operationalization	If a domain is only active for patients who enter a state (either at enrollment or later), the patient is randomized to an intervention within that domain but the intervention is only revealed when the patient enters the state
	Estimates of intervention effects within a state-specific domain are only generated for those who enter the state
Sites and regions
Starting conditions	The study launches at 50 hospitals in Europe, 35 sites in Australia and New Zealand, and 12 sites in Canada
Future additions	Expansion in the United States, Brazil, and Saudi Arabia is under discussion. Long-term planning includes other regions
Interventions
Nomenclature
Intervention	A treatment being tested in REMAP-CAP
Domain	A specific set of competing alternative interventions within a common clinical mode, which, for the purposes of the platform, are mutually exclusive and exhaustive
Regimen	The combination of assigned interventions across domains
Starting conditions
The trial launches with four domains
Antibiotics
	• Ceftriaxone plus macrolide
	• Piperacillin-tazocin plus macrolide
	• Amoxycillin-clavulanate plus macrolide
	• Respiratory quinolone
Immunomodulation with an extended macrolide
	• Standard course (3–5 d)
	• Extended macrolide (14 d)
Immunomodulation with hydrocortisone
	• No corticosteroid
	• Shock-dependent hydrocortisone
	• Hydrocortisone (7-d course)
Antiviral agents active against influenza
	• No antiviral agent
	• Oseltamavir (5 d)
	• Oseltamavir (10-d course)
	Patients can be ineligible for randomization within a domain (e.g., the antiviral domain is only active for those within the influenza stratum). Thus, the trial launches with 240 potential regimens (adding “not eligible” as an option in each domain, no. of regimens=5 antibioticx3 extended macrolidex4 steroidx4 antiviral=240)
Future additions
	Two additional domains (ventilator support and oxygen management) will be added shortly
	The ventilator support domain will be restricted to the state of mechanical ventilation. Interventions to be tested within this state-specific domain will be guideline-recommended ventilation and clinician-preferred ventilation
	The oxygen management will compare two interventions (usual oxygen titration vs. conservative oxygen titration). This domain will be eligible to all patients
	Once these domains launch, each with two options plus “not eligible,” the number of regimens becomes 240×3×3=2,160 regimens
Embedding
Description	To ensure capture of all possible patients, streamline integration with clinical care, and reduce study costs, the study has several features that embed it in clinical practice. Ideally, these embedded strategies are built through integration between REMAP-CAP trial machinery and usual clinical processes. Strategies include:
	• Triggering of patient identification and enrollment from a clinical “point-of-care”
	• Verification of eligibility, documentation of consent, and enrollment activation via software interface
	• Generation of stratum-specific randomly assigned REMAP-CAP regimen as “order set”
	• Intent to embed, where appropriate, within the electronic health record
Endpoints
Primary endpoint	• All-cause mortality at 90 d
Secondary endpoints	• ICU mortality
	• ICU length of stay
	• Ventilator-free days
	• Organ failure free days
	• Proportion of intubated patients receiving tracheostomy
	• Domain-specific end-points
Statistical methods
Overview	The trial is built on a Bayesian inference framework. After an initial run-in period, a prespecified Bayesian inference model is updated each month using the latest trial data to generate updated posterior probabilities of death for each patient regimen-by-stratum group, and hence the probability that any one intervention (or regimen) differs from any other. The model output is used both to update the randomization weights for ongoing random assignments and to trigger thresholds for superiority, equivalence, and inferiority
Multifactorial Bayesian inference model	The model predicts the primary endpoint rate for each patient regimen-by-stratum group, conditional upon patient age, trial site and region, and time era. Terms are included for intervention-by-intervention and intervention-by-stratum interactions, and for patients who are ineligible for either an intervention or a domain. The model is also configured in advance for the incorporation of state-specific domains (e.g., ventilator support)
Response-adaptive randomization	The posterior probabilities from the Bayesian inference model are incorporated into an algorithm that provides updated randomization proportions to each regimen by stratum. This algorithm adjusts for sample size to avoid large, potentially spurious changes. Consequently, interventions that are faring well will be randomly assigned more commonly, and those faring less well will be assigned less commonly
REMAP-CAP statistical conclusions	When an updated probability triggers a threshold, results are communicated to the DSMB and ITSC for public release and decisions regarding ongoing treatment assignment
Superiority	>99% probability that an intervention is superior to alternatives in a domain within one or more strata
Equivalence	>90% probability that odds of death for two interventions differ by <0.2
Inferiority	<1% probability that an intervention is superior in a domain
Operating characteristics	All trial parameters were tested through extensive Monte Carlo simulations of anticipated trial performance under different scenarios (Appendix)

Open in a separate window

Definition of abbreviations: ICU=intensive care unit; CAP=community-acquired pneumonia; DSMB=Data Safety Monitoring Board; ITSC=International Trial Steering Committee; REMAP-CAP=Randomized Embedded Multifactorial Adaptive Platform for Community-acquired Pneumonia.

This table describes REMAP-CAP in interpandemic mode, and excludes the coronavirus disease 2019 adaptations (described in the Adaptation during a Pandemic section of the text).

The trial estimates the effectiveness of one intervention over others within a domain, with the capacity to specify whether effects are affected by the choice of interventions within other domains or by strata. Which interactions are evaluated are prespecified. The trial uses response-adaptive randomization (RAR) (32), with the probability of randomization to any particular regimen adjusted over time to favor better-performing interventions, eventually triggering a stop when a predetermined threshold is attained (Figure 1). Colloquially, RAR allows the trial not to “play the winner,” but to “probably play what is probably the winner.” The RAR rules define separate randomization proportions for each stratum. For example, if one hydrocortisone dosing strategy appears beneficial for patients with shock, but neutral in patients without shock, then the RAR rule increasingly weights the odds for shock patients to receive that strategy, but maintains equal allocation for patients without shock.

Importantly, interventions may not be appropriate for a patient, either because the patient is eligible for a domain, but has a contraindication for a particular intervention within that domain, or because the patient is not in a clinical state that requires treatment within that domain. In the first situation, as long as at least two interventions remain available within the domain, the patient will be randomized. An example of the second situation would be a respiratory support domain restricted to patients requiring mechanical ventilation. If a patient is enrolled in the trial, but not intubated, she will be randomized, but the assignment will not be revealed until she enters the state (requiring mechanical ventilation) that triggers deployment of the intervention. In addition to “patient-level” exclusions, not all domains and interventions may be available at all sites, either because a participating site lacks equipoise or temporarily lacks availability of an intervention. In all these instances, the statistical inference model tracks and accommodates for these varying levels of participation.

Other adaptive trial features include the capacity to introduce new strata, domains, and interventions over time. The rules and operating characteristics of the platform are detailed in the REMAP-CAP core protocol and statistical analysis appendix with separate domain-specific and region-specific appendices to describe interventions and regional participating groups (Appendix; see also www.remapcap.org). The use of separate appendices permits an efficient, modular structure where any update to the design requires only that the relevant appendix or appendices be added or modified (Figure 2A).

Open in a separate window

Figure 2.

Overview of the REMAP-CAP (Randomized Embedded Multifactorial Adaptive Platform for Community-acquired Pneumonia) documentation and oversight. (A) Structure of the REMAP-CAP protocol and appendix documents. (B) Organogram of the REMAP-CAP oversight. ANZ = Australian and New Zealand; ANZ RMC = Australia and New Zealand Regional Management Committee; CRMC = Canadian Regional Management Committee; DSA = domain-specific appendix; DSMB = Data and Safety Monitoring Board; DSWG=domain-specific working group; EU = European; EU RMC = Europe Regional Management Committee; IIG = International Interest Group; IPWG = International Pandemic Working Group; ITSC = International Trial Steering Committee; RMC = Regional Management Committee; RSA = region specific appendix; SAC = Statistical Analysis Committee.

Study Sites, Patients, and Endpoints

Table 1 summarizes key trial features. REMAP-CAP is a global program intended to enroll critically ill patients with CAP worldwide (clinical trials registration no. NCT02735707; universal trial no. U1111-1189-1653). The trial was launched in Europe under the PREPARE (Platform for European Preparedness against (Re-)emerging Epidemics) consortium (https://www.prepare-europe.eu/about-us/workpackages/workpackage-5) with funding from the European Union. REMAP-CAP has also launched in Australia and New Zealand, supported by the Australian and New Zealand Intensive Care Society Clinical Trials Group and in Canada supported by the Canadian Critical Care Clinical Trials Group, with funding from the respective national governments. Together, these programs fund the first 4,000 patients, and are anticipated to recruit in 50 sites in Europe, 35 sites in Australia and New Zealand, and 15 sites in Canada. Other regions of the world will join as funding becomes available. Over 500 patients were enrolled as of March 2020. The trial is overseen by an international trial steering committee. An overview of trial structure is provided in Figure 2B.

To be included, participants must be admitted to the ICU within 48 hours of hospital admission, be ≥18 years of age, have CAP by clinical and radiologic criteria (33), and require respiratory or cardiovascular organ support. Exclusion criteria include healthcare-associated pneumonia, presumption that death is imminent with lack of commitment to full support, and participation in REMAP-CAP in the prior 90 days. There are also domain-specific exclusion criteria described in their appendices (Appendix; www.remapcap.org). The primary objective is to determine the effectiveness of different interventions, alone and in combination, for adult patients with severe CAP in decreasing 90-day mortality. Secondary objectives are to determine the effects on hospital and ICU length of stay, ventilator and organ failure–free days through 28 days, and functional outcomes at Day 180.

Initial Domains and Interventions

Adaptation during a Pandemic

REMAP-CAP adapts to answer time-critical questions relevant to optimal care of patients with pneumonia due to a pandemic infection in several ways. The platform has a “sleeping” stratum for patients with proven or suspected pandemic infection that is triggered at each site. A pandemic-specific model tests the effect of different agents and regimens in the pandemic stratum. This model can use an alternative endpoint and be updated more frequently. The pandemic-specific model can incorporate data from nonpandemic patients with regard to all domains that are relevant in both pandemic and interpandemic periods, with consideration of potential interactions. In addition, other domains, such as novel antiviral therapies, immunoglobulins or convalescent sera, or other immunomodulation approaches, can be deployed.

In February 2020, REMAP-CAP entered pandemic mode in response to the COVID-19 pandemic with several adaptations, essentially as a subplatform, REMAP-COVID. These include a COVID-19 inference model for all confirmed and suspected cases that uses 21 ICU-based respiratory or cardiovascular support-free days (where death is assigned −1 d) as the primary outcome, with RAR as frequently as weekly. A specific REMAP-COVID core protocol was written to streamline onboarding of new sites that only enroll patients with COVID-19. Domains were implemented for COVID-19 antiviral therapy (including hydroxychloroquine and lopinavir-ritonavir) and immune modulation (including interferon [IFN]-β, IL1ra, and IL6ra agents), the corticosteroid domain was modified to include a higher dose, and other domains are under construction. The enrolment criteria were modified to allow entry at some sites of hospitalized patients who do not require ICU care for cardiovascular or respiratory support (defined as “moderate” COVID-19 disease state). The model tracks whether patients have moderate or severe disease at enrolment, includes interactions between domains (e.g., IFN-β and corticosteroids), and allows for nested analyses (e.g., comparing any antiviral therapy vs. none).

Advantages of the REMAP Design

The REMAP design offers four broad advantages: efficient use of data, improved participant safety, reduced downtime between trials, and enhanced knowledge translation (Table 2 and Figure 3). Four features improve efficiency. First, testing multiple interventions simultaneously allows more questions to be evaluated and avoids requiring a separate control group for every two-way comparison. Second, RAR and predetermined thresholds reduce or cease allocation of subjects to inferior arms, increasing power to differentiate between the remaining arms. Third, an overarching multifactorial model that drives RAR and stopping rules integrates information on treatment effects from all patient strata. Fourth, because randomization continues until superiority, equivalence, or inferiority thresholds are met, the platform avoids terminating a domain with indeterminate results.

Table 2.

Randomized embedded multifactorial adaptive platform design advantages

	Efficient Use of Information	Safety of Trial Participants	Avoiding Trial Downtime	Fusing Research with Care	Determining Optimal Disease Management	Learning Healthcare System
Multifactorial		—				—
Response adaptive randomization			—		—
Embedding	—	—	—		—
Frequent adaptive analyses			—	—
Analysis by stratum/subgroup			—	—		—
Evaluation of interaction	—		—	—		—
Substitution of new interventions		—		—		—

Open in a separate window

Open in a separate window

Figure 3.

Trial simulations comparing REMAP (Randomized Embedded Multifactorial Adaptive Platform) to traditional randomized clinical trial designs. The operating characteristics of alternative study designs are evaluated by running a Monte Carlo program, which randomly draws trial samples from simulated populations with predetermined characteristics (alternative “truths” about the true yet unknown effect of an intervention or regimen in a population). Each simulated trial accrues patients one at a time until a sample size of 2,000. The simulated trials are repeated 10,000-fold and the summary of all trials under each simulated scenario provides estimates of average trial performance. In all instances, the simulations are of trials testing eight regimens, consisting of three domains with two interventions in each domain (2³=8 regimens). Results are presented for a comparison of a standard trial design, with equal allocation to each arm, versus a REMAP design, using response-adaptive randomization (RAR) to preferentially assign patients over time to better-performing arms. Sample size (primary y-axis) is 250 per arm for the standard design (represented by a solid horizontal line); gray bars indicate the REMAP design. Probability of superiority (a proxy for power, secondary y-axis) is represented as an open red circle for the standard design and a solid red circle for the REMAP design. The predetermined characteristics of the underlying simulated population are represented in the upper portion of each panel. (A) Result summary under a simulated truth where regimen 8 is superior, regimen 5 is second best, and all others are inferior, but equivalent. (B) Result summary where regimens 5 and 8 are equally good, but regimens 1, 3, 4, and 7 are harmful with respect to regimens 2 and 6. In both scenarios, power is similar or superior with the REMAP design, yet, because RAR minimizes exposure to arms performing less well, results are generated with fewer deaths.

The REMAP design enhances safety because the adaptive rules promote greater allocation to better-performing interventions and, by corollary, less exposure to poorly performing interventions, over time. As the trial learns, the benefits of reduced uncertainty are translated rapidly into improved odds of exposure to the optimal strategy for participants. Thus, although individuals may still be assigned to interventions that perform poorly, if the trial is testing therapies that affect outcome, but for which the conventional wisdom is equipoise (and exposure outside the trial is balanced), then the patient is, on average, safer in the trial than out of it.

There is considerable downtime between traditional one-at-a-time trials, which is costly and burdensome for clinical trial units and contributes to delay in the acquisition of medical knowledge, or even failure to accrue knowledge in situations like pandemics (3). Because REMAP is a single perpetual platform trial, this downtime is largely eliminated. Instead, new interventions or domains of interest are simply added to the on-going platform through protocol appendix amendments. When fully embedded in an entire healthcare system, a REMAP becomes a platform for continuous quality improvement (and instant knowledge translation), where all patients are flagged at admission, and assigned therapies proportional to the level of certainty that these therapies are optimal.

Discussion

Although we outlined numerous potential advantages of the REMAP trial design, there are considerable barriers. First, the ability to embed the trial requires a new paradigm for engagement between clinicians and researchers in many ICU settings. Such close partnership exists in other fields, such as within oncology trial networks. Similarly, the large acute myocardial infarction trials in the 1980s and 1990s relied on extremely high capture rates. In critical care, the fluid resuscitation trials by the Australian and New Zealand Intensive Care Society Clinical Trials Group also achieved extremely high capture rates (70, 71), in part by generating a culture that any patient requiring resuscitation prompted the clinical team to enroll the patient. These efforts share a common commitment to education, engagement, and attention to practical details at participating sites.

One concern will be the use of Bayesian inference and flexible sample sizes. For example, Bassler and colleagues (72) argued that early stopping overinflates estimation of treatment effects. However, trials that stop early for superiority are trials that, on average, would overestimate treatment effect even if they run to term (just as trials that do not trigger early stopping underestimate the true effect) (73). Assuming that appropriate rules are in place, early stopping does not, in and of itself, significantly overestimate treatment effect (or inflate the chance of type 1 error). The best estimate of treatment effect is the summary of all trial results. If REMAP-CAP generates an early large superiority signal for an intervention, and no other trial data exist, it would be appropriate to consider the true effect size as somewhat smaller (74).

As with all Bayesian adaptive designs, traditional estimation methods for type 1 and type 2 errors are not possible. Rather, these error rates are explored through simulation of trial operating characteristics under different scenarios and assumptions. The U.S. Food and Drug Administration and others provide guidance, but there is little question that considerable expertise is required (75, 76). This expertise is currently limited, and concentrated particularly within a few companies. To broaden expertise, all the government grants for REMAP-CAP stipulate efforts to expand competency among local academic trials groups. As such, REMAP-CAP has several regional initiatives and runs an international statistics and reporting interest group (>40 statisticians and trialists from 16 universities; Figure 2). The statistical group for REMAP-CAP provides the design and simulation software for free to academic groups, and serves as a free National Institutes of Health–supported consultation service for prospective researchers.

There will be issues regarding the reporting of REMAP trials, and all adaptive platform trials (22). For example, REMAP conclusions are generated from a model that incorporates all the data from the entire trial. It is unclear whether the report should include information on all patients enrolled thus far, including those whose data are still contributing to ongoing questions, or to some portion of the patients most directly relevant to the portion of the trial that has stopped. Because most RCTs are frequentist, trial reports that use Bayesian statistics will be unfamiliar to many readers, impeding understanding and dissemination. However, Bayesian trials and analyses are becoming considerably more common, which should reduce this problem (30, 77–79).

In summary, we present a novel class of study design with an example tailored specifically to determine optimal therapies for severe interpandemic and pandemic pneumonia. The design generates information that is broad (reflecting real-world practice) and narrow (generating precision estimates for patients with particular clinical features). The platform can incorporate new study arms, making it ideal for pandemic situations. The design will nonetheless face challenges. However, with funding to launch REMAP-CAP on three continents, we expect that many lessons will be learned, aiding broader, more efficient use of REMAPs in critical care and elsewhere.

Supplementary Material

Footnotes

References