Abstract

Key Summary Points

Introduction

Chronic Hand Eczema (CHE) is a frequent chronic inflammatory disease affecting the hands and wrists [1]. It is defined as the presence of hand eczema (HE) that persists for more than 3 months or returns at least twice in 12 months [2]. Most cases of CHE have a multifactorial origin, including exposure to exogenous agents (i.e., irritants and allergens) and predisposing endogenous factors, with contact dermatitis appearing as the main diagnosis [3–5]. CHE is often associated with recurrent flares and poor prognosis [1], and the majority of patients with CHE are reported to have moderate to severe disease [6]. Clinically, CHE is characterized by signs of hyperkeratosis, inflammation (edema, papules, erythema), vesicles, crusting, scaling, and fissures, causing symptoms of itch and pain [2, 7, 8]. These signs and symptoms are reported to severely impact patients’ health-related quality of life (HRQoL), causing difficulties with performing daily life activities (e.g., household chores, dressing), physical functioning (gripping/holding objects), and the ability to work, which in turn may impact psychological well-being [1, 9–11].

As HRQoL is a multidimensional construct, patient-reported outcome (PRO) measures used within clinical trials must accurately assess the different components associated with HRQoL that are clinically relevant and important to patients. This is also referred to as “content validity” and is the most important measurement property of any PRO measure. Furthermore, it is important to show evidence of construct validity, reliability, and the ability to detect change for scores derived from the measure in the target population [12–15]. Within CHE clinical trials, two commonly used HRQoL measures are the Dermatology Life Quality Index (DLQI) [16] and the Quality of Life in Hand Eczema Questionnaire (QOLHEQ) [17, 18]. As a dermatology-specific measure of the impact of skin disease on patients’ quality of life, the DLQI is unlikely to comprehensively assess in detail the aspects of HRQoL that are specific to the experience of patients with CHE. Indeed, several concepts identified as relevant to the experience of patients with CHE in previous research [10] are notably absent from the DLQI, including key aspects of physical functioning (e.g., touching or holding objects), activities of daily living (e.g., housework that involves getting the hands wet), and self-care (e.g., washing or bathing). This could limit the DLQI’s precision and sensitivity to change in a CHE population [19]. Further, the QOLHEQ is a condition-specific measure, developed to assess impairment of HRQoL in patients with HE. While the QOLHEQ demonstrates conceptual coverage of most HRQoL concepts identified as relevant to the experience of patients with CHE in previous research [10], it has limitations that may make it less likely to be accepted by regulators in support of label claims. This includes limited evidence of content validity (i.e., limited direct input from patients with CHE in concept elicitation interviews during instrument development), the use of clinical terminology (e.g., fissuring), and the grouping of items assessing physical functioning and social/familial relationships under a single domain. It was therefore decided to develop a new PRO measure, the Hand Eczema Impact Scale (HEIS), specifically designed to assess the multidimensional aspects of HRQoL associated with CHE.

The objective of this research was to develop and evaluate content and psychometric validity of the HEIS to confirm its suitability for use in CHE clinical trials and clinical practice.

Methods

Psychometric Validation

The HEIS that emerged from the content validity interviews was included as an exploratory endpoint in a phase 2b double-blind, multicenter, prospective, randomized, five-arm, vehicle controlled, parallel-group trial evaluating the efficacy and safety of delgocitinib cream in adult participants with mild to severe CHE (NCT03683719). Data from the phase 2b trial (n=258) were used to support item reduction, development of scoring, and evaluation of psychometric properties, including anchor- and distribution-based analyses to support interpretation of change. The study design and eligibility criteria for the phase 2b trial have been previously described [23]. Participants were recruited from clinical sites in Denmark, Germany, and the US. Prior to initiation of the phase 2b trial, the HEIS was translated to Danish and German and linguistically validated by native speakers in line with best-practice guidelines [24, 25]. All participants completed the HEIS at the trial site on an electronic handheld device at screening, baseline, and during weeks 1, 2, 4, 6, 8, 10, 12, 14, and 16 prior to investigator assessment of treatment efficacy.

Trial Assessments

Patient- and clinician-reported outcome measures were administered alongside the HEIS during the phase 2b trial to support construct validity analyses, define participants with stable CHE for test–retest reliability, and define participants who experienced change (Table 1).

Table 1

Overview of other relevant clinical outcome assessments included in the phase 2b clinical trial

Assessment	Description
Patient Global Assessment of Disease Severity (PaGA)	• The PaGA is a single-item, patient-reported global assessment of HE severity with a 5-level scale (0, clear; 1, almost clear; 2, mild; 3, moderate; and 4, severe). The PaGA was administered during site visits at all timepoints
Investigator Global Assessment of Chronic Hand Eczema (IGA-CHE) [36]	• The IGA-CHE allows investigators to assess overall disease severity at one given timepoint and consists of a 5-level severity scale (i.e., 0, clear; 1, almost clear; 2, mild; 3, moderate; 4, severe). Each severity level on the scale is characterized in terms of the clinical characteristics of erythema, scaling, hyperkeratosis, vesiculation, edema, and fissures. The IGA-CHE was administered during site visits at all timepoints
Patient Global Impression of Change (PGI-C)	• The PGI-C is a single-item assessment that asked patients to choose the response option (i.e., much better, a little better, no change, a little worse, much worse) that best describes the overall change in their HE signs/symptoms since starting the trial medication. The PGI-C was assessed at weeks 4, 8, and 16
Dermatology Quality of Life Index (DLQI) [16]	• The DLQI is a validated dermatology-specific questionnaire widely used as a trial endpoint in dermatological conditions. It consists of ten items addressing the patient’s perception of the impact of their skin disease on six different aspects of quality of life over the last week, including symptoms, feelings, leisure, work or school, personal relationships, and treatment. The total score ranges from 0 to 30, where a high score is indicative of poor quality of life. A change in DLQI score of at least four points in adults is considered clinically meaningful [37]. Individual subscale scores can also be calculated as the sum of all items on a given subscale. The DLQI was assessed at screening, baseline, and weeks 1, 4, 8, 12, and 16
Quality of Life in Hand Eczema Questionnaire (QOLHEQ) [18, 38]	• The QOLHEQ is a validated questionnaire used to assess disease-specific HRQoL in patients with HE. It consists of 30 items and assesses disease-related impairment over the last 7 days within four domains: (i) symptoms, (ii) emotions, (iii) limitations in functioning, and (iv) treatment/prevention. Each item is answered on a 5-point scale (never, rarely, sometimes, often, all the time). Scores range between 0 and 117, with high scores indicating worse health. The QOLHEQ was assessed at screening, baseline, and weeks 2, 6, 12, and 16

Open in a separate window

Statistical Methods

Table ​Table22 details the main psychometric analyses used in this study. All analyses were conducted in the psychometric analysis population (composed of all 258 participants randomized and exposed to treatment) and using data from week 4, unless otherwise specified. Week 4 was selected as this timepoint was expected to provide a greater distribution of scores than other timepoints. All statistical analyses were detailed a priori in a psychometric analysis plan and conducted in accordance with European Medicines Agency (EMA) and the US Food and Drug Administration (US FDA) standards and US FDA guidance for PROs and other clinical outcome assessments (COAs) [12, 13, 25, 26]. All analyses used SAS version 9.4 [27], apart from confirmatory factor analysis (CFA), which was performed using Mplus version 8 [28]. The consensus-based standards for the selection of health measurement instruments (COSMIN) reporting guidelines for studies on measurement properties of PROs were also considered for this paper [29].

Table 2

Summary of psychometric analyses performed in the phase 2b clinical trial data

Analysis	Description
Stage 1: Item properties
Quality of completion	Quality of completion was evaluated to identify any items with unexpectedly high levels of missing data. Missing data were described at baseline and weeks 1, 2, 4, 6, 8, 10, 12, 14, and 16
Item response distributions	Item distributions (frequency and percentage of each endorsed response) were assessed at baseline and weeks 8 and 16. Items with a large proportion (>20.0%) of respondents selecting the best (ceiling effect) and worst (floor effect) possible options were flagged for further consideration
Stage 2: Dimensionality and scoring
Inter-item correlations	Correlations between items (inter-item) assessed the homogeneity of items, and items correlating>0.90 were considered potentially redundant. Items within a hypothesized, multi-item domain were anticipated to correlate more highly with each other than with items in other domains
Confirmatory factor analysis (CFA)	CFA was used to confirm the extent of the hypothesized conceptual framework comprising a total score and five domain scores: proximal daily activity limitations (PDAL; item 2, use of soaps/cleaning products; item 5, do housework; item 6, grip/hold objects; item 7, wash), embarrassment with appearance of the hands (Emb; item 3, felt embarrassed and item 4, dislike appearance of the hands), frustration with Chronic Hand Eczema (item 8), sleep (item 1), and work (item 9) was supported. Two models were hypothesized: a single factor influencing responses to all nine HEIS items (model A) and a bifactor structure with one factor influencing responses to all nine items but with four items loading on a domain reflecting PDAL (model B). For model B, correlated errors between items in the hypothesized Emb domain were modeled. Model fit was assessed using root mean square error of approximation [RMSEA;<0.10 with 90% confidence interval (CI) indicates acceptable fit], [39] comparative fit index (CFI;≥0.95 indicates good fit), [40] Tucker–Lewis index (TLI;≥0.90 indicates good fit), and standardized root mean squared residuals (SRMR;≤0.08 indicates good fit) [40]. Items were expected to load on their hypothesized domain with a standard factor loading of>0.40 [41]
Stage 3: Reliability and validity of scores
Reliability
Internal consistency reliability	Internal consistency of the HEIS score, HEIS PDAL score, and HEIS Emb score was evaluated using Cronbach’s alpha coefficient (≥0.70 was considered indicative of good internal consistency) [42]. The Spearman–Brown formula was used to adjust for the fact that the HEIS Emb score only includes two items and the PDAL score only includes three items [43, 44]. The impact of item removal on internal consistency reliability was examined by calculating Cronbach’s alpha, with each item removed from the HEIS score and HEIS PDAL in turn. If the removal of an item causes the alpha value to notably increase, then that item may not be fitting well within its domain. Corrected item total correlations were also calculated by computing the Pearson’s correlation coefficient of each item with the sum of the remaining items within its corresponding score. Items with a correlation<0.40 were considered evidence that the item does not fit well with the other items [45]
Test–retest reliability	Test–retest reliability evaluated the consistency of scores between weeks 12 and 16 and weeks 14 and 16 in patients with “stable” CHE, defined as those with no change on the PaGA, IGA-CHE, and relevant domains of the DLQI and QOLHEQ. For the HEIS score, HEIS PDAL score, and HEIS Emb score, intra-class correlation coefficients (ICCs) were evaluated using the following prespecified cutoff criteria:<0.50 indicating poor reliability, 0.50–0.75 indicating moderate reliability, 0.75–0.90 indicating good reliability, and>0.90 indicating excellent reliability [46]. Pearson’s correlation coefficients were also calculated. For the single-item domain scores, Cohen’s kappa with quadratic weighting was used to assess reliability. This involved evaluating the level of disagreement between each patient’s time 1 and time 2 score and using this to define reliability in terms of deviation from perfect agreement. The amount of disagreement was weighted so that a 2-point difference over time was accounted for by the model as a more severe discrepancy than a 1-point difference over time
Construct validity
Convergent validity	Convergent validity was evaluated by examining correlations of the HEIS score and all domain scores with the relevant domains of the DLQI and QOLHEQ. Scores assessing similar or related concepts were expected to have moderate-to-strong correlations (≥0.50), whereas scores assessing unrelated concepts were expected to show small (<0.30) or negligible correlations [47]
Known-groups analysis	Construct validity was also assessed using the known-groups method to evaluate differences in HEIS score, HEIS PDAL score, and HEIS Emb score among groups of participants expected to differ in severity. Groups were defined by responses to the PaGA, IGA-CHE, and relevant domains of the DLQI and QOLHEQ. The magnitude of difference was considered using between-group effect size estimates (small change=0.20, moderate change=0.50, large change=0.80) [47]. F-test calculated by one-way analysis of variance (ANOVA) was used to evaluate whether differences were statistically significant (p<0.05)
Ability to detect change	Ability to detect change was assessed for the HEIS score, HEIS PDAL score, HEIS Emb score, HEIS item 6: felt frustrated, and HEIS item 8: hard to work. Within-group mean change scores were compared from baseline to week 16 for individuals who experienced different changes on the PaGA, IGA-CHE, and DLQI item 2 (categories: improved,≥1 grade improvement; stable,no change; worsened,≤1 grade worsening), PGI-C (categories: improved=much better or a little better, stable=no change, worsened=much worse or a little worse), DLQI total score (categories: improved,≥4-point improvement; stable,<4-point change; worsened,≥4-point worsening), [37] QOLHEQ total score (categories:≥13-point improvement, stable=<13-point change, worsened=≥13-point worsening), [18] and DLQI limitations in functioning domain, DLQI work and school domain, and QOLHEQ emotions domain (categories: improved,≥0.5 SD improvement; stable,<0.5 SD change; worsening=≥0.5 SD worsening), using one-way ANOVA F-test (between group) and within-group effect sizes [47, 48]
Interpretation of scores
Anchor-based methods	Anchor-based meaningful change analyses were performed for the HEIS score, HEIS PDAL score, and HEIS Emb score using change from baseline to week 16 data. Anchor-based change scores had to correlate>0.30 with the domain change score to be considered relevant for interpretation [49]. Thresholds were calculated by evaluating within-group and between-group mean change scores, estimated using mixed models for repeated measures (MMRM). The within-group responder definition estimates were calculated using least square (LS) mean change scores for the minimally improved and moderately improved groups as follows: • HEIS score: an improvement of 0.5–1.0 SD on the QOLHEQ total score and of 5–8 points on the DLQI total score for the minimally improved group. There was no moderately improved group for the HEIS score • HEIS PDAL and Emb scores: an improvement of 0.5–1.0 SD on the QOLHEQ limitations in functioning score (PDAL only) and emotions score (Emb only), a one-level improvement on the PaGA, a one-level improvement on the IGA-CHE, and a score of a little better on the PGI-C for the minimally improved group. The anchors used to define the moderately improved group were two-level improvements on the PaGA and IGA-CHE The between-group minimal important difference (MID) score estimate for each anchor was defined as the difference in LS mean change score between the minimally improved and stable groups, supported by 95% CIs. Receiver operating characteristic (ROC) curves were calculated for the selected scores between baseline and week 16 to find the change score that optimally discriminated between participants who worsened or were stable from those who had any level of improvement as defined by the anchors. Empirical cumulative distribution function (eCDF) and probability density function (PDF) plots were used to allow various proposed within-individual responder definitions generated to be evaluated simultaneously. To guide the triangulation of estimates, the different MID (or responder definition) estimates with 95% CIs (as available) were visualized on forest plots to identify the most appropriate range of values
Distribution-based methods	Distributional properties of the HEIS score, HEIS PDAL score, and HEIS Emb score were used to provide an indication of the amount of change beyond measurement error that may be considered meaningful. The MID was estimated by calculating 0.5 of the SD at baseline [44, 50] and the standard error of measurement (SEM). The SEM was calculated as the SD at baseline multiplied by the square root of one minus the reliability of the score at baseline [SD × (1–r)1/2]. Internal consistency and Cronbach’s alpha at baseline were used for the reliability for scores with three or more items. This was then repeated using the lowest ICC arising from the test–retest analyses. A value of 1 SEM was used as the estimate of the responder threshold

Open in a separate window

Results

Figure 1 presents an overview of the HEIS development and validation process, which led to the final 9-item instrument.

Open in a separate window

Fig. 1

HEIS development and validation overview

Qualitative Interviews

Interview Sample Characteristics

The interview sample characteristics are presented in Table S3. Briefly, a total of 20 adults with CHE from the US were interviewed (n=10 in each interview round). The mean age of the sample was 46.6 years (range 18–69) and most were female (n=14; 70.0%) and white/Caucasian (n=13; 65.0%).

Cognitive Debriefing of HEIS

The items developed to assess HRQoL concepts identified as important during concept elicitation [10] were generally found to be relevant and well understood (Table 3). Relevance of items was comparable across CHE subtype (Fig. S1). Most participants correctly understood and accurately used the recall period throughout the HEIS (≥75.0%). Similarly, the response options (not at all, a little, moderately, a lot, and extremely) were well understood by participants (≥88.9% for all items). All participants who were asked found the electronic handheld device touchscreen to be responsive (n=11/11), navigating between the items to be easy (n=9/9), and the font size appropriate and easy to read (n=12/12). Most participants asked (n=7/8) also found the device light and easy to hold.

Table 3

Overview of HEIS items understanding and relevance

HEIS item	Reported during concept elicitation	Cognitive debriefing understanding	Cognitive debriefing relevance	Supportive quote
Sleep	16/20	20/20	16/20	“…problems going to sleep…waking up in the middle of the night because of any of those symptoms…” (Male aged 41 with severe allergic CHE)
Embarrassment	16/20	20/20	19/20	“…embarrassed means you’re ashamed of your hands. I don’t like showing my hands to nobody…” (Female aged 53 with moderate atopic CHE)
Self-consciousness	16/20	20/20	19/20	“You kind of hide yourself or hide it, keep them under you so nobody can see them…” (Male aged 61 with mild irritant CHE)
Frustrationa	16/20	9/10	9/10	“Looking back, um, I think when I’m, when I’m itchy I feel frustrated” (Male aged 38 with moderate atopic CHE)
Feeling annoyeda	16/20	9/10	9/10	“…the itching. It’s very annoying, frustrating” (Female aged 40 with severe atopic CHE)
Use soaps/cleaning products	15/20	18/20	19/20	“The soap dries it out, dries the skin out or cleaning products is going to do the same thing” (Male aged 61 with mild irritant CHE)
Dislike of appearance of hands	14/20	19/20	20/20	“…do I think my hands are ugly?” (Female aged 60 with severe atopic CHE)
Housework	13/20	19/20	15/20	“…washing, uh, dishes, when I take a shower, when I touch soap. You know, getting anything that has to do with, um, getting my hands wet” (Female aged 33 with moderate atopic CHE)
Holding objects	11/20	19/20	9/20	“…when I’m holding a cell phone that’s what I think of, holding a cell phone. Grip is when I’m grabbing something…” (Male aged 38 with moderate atopic CHE)
Touching objects	12/20	19/20	8/20	“It asks me basically has…my eczema declined me from touching things…” (Male aged 32 with moderate irritant CHE)
Worka	12/20	6/10	6/10	“It’s asking me in the last past seven days has it made it hard for me to work, yes” (Male aged 32 with moderate irritant CHE)
Getting dressed	12/20b	19/20	9/20	“Like if it affected me trying to dress myself I guess” (Female aged 18 with mild atopic CHE)
Washing		19/20	9/20	“I was thinking of like bathing basically, bath or shower” (Female aged 47 with mild allergic CHE)
Wearing glovesa	N/Ac	9/10	7/10	“I cannot drive without gloves. I cannot go stand outside without gloves” (Female aged 65 with severe atopic CHE)

Open in a separate window

HEIS Hand Eczema Impact Scale

^aItems only debriefed in round two interviews; ^bpatients were asked more generally about difficulties with self-care during the concept elicitation interviews and therefore specific counts were not captured for dressing and washing; ^cwearing gloves was not explored independently during the concept elicitation interviews but was mentioned spontaneously as a coping mechanism or trigger by both patients and expert dermatologists

Findings from the first round of interviews suggested that several impacts were similarly related or conceptually equivalent. To explore item redundancy and inform item reduction, seven participants (n=7/10) in round two were asked to group impacts they perceived to overlap conceptually. The most frequently reported conceptual overlaps were between “using soaps/cleaning products” and “housework” (i.e., getting hands wet) (n=6/7); “feeling self-conscious” and “dislike in appearance of hands” (n=5/7); “embarrassment” and “feeling frustrated” (n=5/7); and “feeling frustrated” and “feeling annoyed” (n=5/7).

Round one findings suggested several items (i.e., “do housework,” “hold/grip objects,” “touch objects,” “get dressed,” and “wash”) would likely have substantial floor effects due to participants developing adaptive coping strategies. Key strategies described by participants included avoidance of triggers (e.g., certain soaps) and wearing gloves (e.g., when completing housework or washing themselves). To mitigate for this, the original wording for these items was modified from: “How much your hand eczema limited your ability to…” to “How much your hand eczema made it harder to…”, to account for participants who may not have been entirely limited in their ability to complete these activities but may have found them more challenging. Additionally, the examples included for items assessing ability to “hold/grip objects” and “get dressed” were updated to better reflect the examples used by participants. Four items were also added to assess additional concepts frequently reported during CE (i.e., “feeling annoyed,” “feeling frustrated,” “impact on work,” and “wearing gloves”) but not included in the original draft version. This resulted in a 14-item HEIS for subsequent testing in round two interviews.

Additional modifications were made following round two. Two items were removed due to conceptual overlap with other items; namely, “felt self-conscious” was removed due to overlap with “felt embarrassed,” and “feel annoyed” was removed due to overlap with “feel frustrated.” Across both interview rounds, relevance of the items assessing participants’ ability to “touch objects” and “get dressed” were consistently reported to be low (40.0% and 45.0%, respectively) and were therefore removed from the HEIS. The item assessing “wearing gloves” was also removed as participants were required to wear gloves during the trial, which could impact their responses to this item. Three items (i.e., do housework, hold/grip objects, and wash) were also reworded from “made it harder” to “made it hard” to mitigate against participants’ comparing their experience with a previous state.

This resulted in the 9-item HEIS taken forward for psychometric evaluation in the phase 2b trial. It was hypothesized that the nine items could be grouped into five domains: proximal daily activity limitations (PDAL; 4 items), embarrassment with the appearance of hands (Emb; 2 items), frustration with CHE (1 item), sleep (1 item), and work (1 item) (Fig. 2).

Open in a separate window

Fig. 2

HEIS conceptual framework taken forward for phase 2b trial

Psychometric Validation of the HEIS in the Phase 2b Trial

Trial Sample Characteristics

Demographic and clinical characteristics of the trial sample are presented in Table 4. Participants were mostly female (61.2%) and classified as Fitzpatrick skin types II or III (56.2% and 32.2%, respectively).

Table 4

Demographic and clinical characteristics of phase 2b trial sample

Description	Psychometric analysis population (N=258)
Age
Mean (SD)	46.0 (14.5)
Median	48.0
Minimum, maximum	18.0, 79.0
Sex, n (%)
Female	158 (61.2)
Male	100 (38.8)
Ethnicity, n (%)
Hispanic or Latino	9 (3.5)
Non-Hispanic or Latino	249 (96.5)
Race, n (%)
White	254 (98.4)
Asian	3 (1.2)
Other (unspecified)	1 (0.4)
Region, n (%)
Europe	237 (91.9)
North America	21 (8.1)
Fitzpatrick skin type, n (%)
Type I	17 (6.6)
Type II	145 (56.2)
Type III	83 (32.2)
Type IV	13 (5.0)
Main diagnosis (CHE subtype), n (%)
Allergic contact dermatitis	15 (5.8)
Atopic hand eczema	97 (37.6)
Hyperkeratotic eczema	46 (17.8)
Irritant contact dermatitis	80 (31.0)
Acute recurrent vesicular hand eczema	20 (7.8)
Smoking history, n (%)
Smoker	79 (30.6)
Ex-smoker	70 (27.1)
Nonsmoker	109 (42.2)
IGA-CHE severity, n (%)
Mild	61 (23.6)
Moderate	145 (56.2)
Severe	52 (20.2)

Open in a separate window

CHE Chronic Hand Eczema, IGA-CHE Investigator Global Assessment of Chronic Hand Eczema, SD standard deviation

Stage 1: Item Properties

Quality of Completion

Quality of completion was high across all timepoints (>90.0%), with no more than 8.5% (n=20) of the psychometric analysis population missing scores in any given week.

Item-Response Distributions

For all items, participants used the full range of possible responses, with most response options endorsed by more than 20.0% of the sample for most items (see Figs. S2–S4). Ceiling effects (>20% scoring the highest possible score) were detected at baseline for “dislike appearance of the hands” (27.0%) and “hard to do housework” (21.9%). However, it was expected that at baseline, participants would respond at the highest end of the response scale. Post-baseline (week 8 and 16) floor effects (>20% scoring the lowest possible score) became dominant (between 34.0 and 63.3% reporting no impact on each item at week 16), reflecting improvements associated with treatment.

Stage 2: Dimensionality and Scoring

Inter-Item Correlations

Inter-item correlations were calculated to provide an initial exploration of dimensionality (Table S4). All inter-item correlations exceeded 0.50, suggesting they are at least moderately related. Six item pairs correlated highly (>0.80), but most of these shared a hypothesized theoretical domain. However, “feel frustrated” correlated highly (0.85) with both items of the Emb domain (“felt embarrassed” and “dislike appearance”), suggesting these concepts may be more closely related than expected.

Dimensionality of the HEIS

The appropriateness of grouping the HEIS items into a total score (comprising all nine items) and five distinct domains representing the key impacts of CHE (i.e., PDAL, Emb, frustration with CHE, sleep, and work) were tested using CFA. For the unidimensional model (model A), all factor loadings were strong (≥0.80), providing support for a total score. The bifactor model (model B) provided some support for the hypothesized HEIS PDAL domain; however, the item “hard to hold/grip objects” had a factor loading<0.40, suggesting that this item may not be as closely related to the other items as expected (see Table S5 for factor loadings for the HEIS). Fit indices were supportive of good model fit, except for RMSEA values for both models, which were higher than prespecified thresholds (Table 5). Modification indices suggested that adding the item “feel frustrated” to the HEIS Emb domain would have the greatest impact on model fit.

Table 5

Global fit indices and model information for the HEIS CFA at week 4

Model (n=242)	Chi-squared	df	p	RMSEA (CI)	CFI	TLI	SRMR
Model A—Unidimensional	354.72	27	<0.001	0.22 (0.20–0.25)	0.95	0.93	0.08
Model B—Bifactor	182.24	22	<0.001	0.17 (0.15–0.20)	0.98	0.96	0.05

Open in a separate window

Thresholds for acceptable fit: RMSEA=<0.10 with 90% CI; CFI=≥0.95; TLI=≥0.90; SRMR=≤0.80

CFI comparative fit index, df degrees of freedom, HEIS Hand Eczema Impact Scale, RMSEA root mean square error of approximation, TLI Tucker–Lewis Index, SRMR standardized root mean square residual

Item Reduction

Although moving the item “feel frustrated” to the HEIS Emb domain showed a large model fit improvement (RMSEA=0.04, CFI=1.00, TLI=1.00, SRMR=0.01), this was not supported by the previous qualitative work. Interview participants most commonly reported feeling frustrated with CHE in general (n=9/16, 53.0%) and due to the incessant nature of scratching (n=7/16, 44.0%), while expert dermatologists suggested frustration was due to symptom recurrence, burden of treatment, pain, sleep disturbances, inability to concentrate, and flaking. These results suggest frustration due to CHE is distinct and unrelated to embarrassment. Therefore, it was decided to retain this item as part of the total score but not include it in the HEIS Emb domain score.

Further, while the item “hard to hold/grip objects” loaded below the prespecified threshold (>0.40) on the HEIS PDAL domain, its response distributions and inter-item correlations with other items were acceptable. Qualitative findings further indicated that this item assesses a concept important to patients not assessed by any other item in the measure. Therefore, while item deletion was considered, it was decided to retain “hard to hold/grip objects” as part of the HEIS total score, but not include it in the HEIS PDAL domain score.

Performance of all other items supported the current scoring algorithm; thus, no items were removed from the measure and no further changes were made to the hypothesized scoring structure. This resulted in the 9-item HEIS, consisting of two multi-item domain scores, four single-item domain scores, and a total score (Fig. 3). The HEIS PDAL domain is scored by taking an average of the items “hard to use soaps/cleaning products,” “hard to do housework,” and “hard to wash.” The HEIS Emb domain is scored by taking an average of the items “felt embarrassed” and “dislike appearance of hands”. Other items can be scored independently. The total HEIS score is given by taking an average of all nine items.

Open in a separate window

Fig. 3

Conceptual framework showing the HEIS structure following item reduction

Stage 3: Reliability and Validity of Scores

Internal Consistency Reliability

Internal consistency reliability was examined to assess the homogeneity of items belonging to the HEIS score (average across all nine items), HEIS PDAL score (average across three items), and HEIS Emb score (average across two items). Cronbach’s alpha was very high for all scores (range 0.89–0.93), indicating good internal consistency (Table S6). Alpha coefficients remained above the 0.70 threshold for both the HEIS PDAL score and the HEIS Emb score (0.76 and 0.82, respectively) when using the Spearman–Brown formula to adjust for the fact that the HEIS Emb score includes only two items, and the HEIS PDAL score includes only three items. For the HEIS score, calculation of the alpha coefficient with each item deleted in turn did not improve the results, providing support for retaining all items.

Test–Retest Reliability

Test–retest reliability results for participants defined as stable were good to excellent (ICC range 0.78–0.93) for the HEIS score, HEIS PDAL score, and HEIS Emb score between weeks 12 and 16 and between weeks 14 and 16. Pearson’s correlation coefficients (range 0.78–0.93) were similar to ICCs, providing further evidence of strong test–retest reliability (Table 6). For the single item scores (Table S7), Cohen’s kappa values were>0.75 for most anchor scores and timepoints, indicating good test–retest reliability.

Table 6

Test–retest reliability analysis of the HEIS scores between weeks 12–16 and weeks 14–16

Anchor	Timepoint	N	ICC (95% CI)	Pearson’s correlation coefficient
HEIS score
PaGA	Weeks 12–16	116	0.89 (0.85, 0.92)	0.89
	Weeks 14–16	129	0.93 (0.90, 0.95)	0.93
IGA-CHE	Weeks 12–16	111	0.81 (0.74, 0.87)	0.81
	Weeks 14–16	132	0.87 (0.83, 0.91)	0.87
DLQI (total score)	Weeks 12–16	160	0.87 (0.82, 0.90)	0.87
QOLHEQ (total score)	Weeks 12–16	159	0.90 (0.87, 0.93)	0.90
HEIS PDAL score
PaGA	Weeks 12–16	116	0.82 (0.75, 0.87)	0.82
	Weeks 14–16	129	0.89 (0.84, 0.92)	0.89
IGA-CHE	Weeks 12–16	111	0.78 (0.70, 0.84)	0.78
	Weeks 14–16	132	0.83 (0.77, 0.88)	0.83
DLQI (daily activities domain)	Weeks 12–16	118	0.80 (0.73, 0.86)	0.80
QOLHEQ (limitations in functioning domain)	Weeks 12–16	163	0.87 (0.83, 0.90)	0.87
HEIS Emb score
PaGA	Weeks 12–16	116	0.88 (0.83, 0.91)	0.88
	Weeks 14–16	129	0.89 (0.85, 0.92)	0.89
IGA-CHE	Weeks 12–16	111	0.81 (0.73, 0.86)	0.81
	Weeks 14–16	132	0.86 (0.81, 0.90)	0.86
QOLHEQ (emotions domain)	Weeks 12–16	149	0.81 (0.75, 0.86)	0.81

Open in a separate window

DLQI Dermatology Life Quality Index, ICC intra-class correlation coefficient, IGA-CHE Investigator Global Assessment of Chronic Hand Eczema, PaGA Patient Global Assessment of Disease Severity, QOLHEQ Quality of Life in Hand Eczema Questionnaire

Convergent Validity

All convergent correlations between the HEIS score and domain scores (including both the multi-item and single-item domains) and relevant scores from the DLQI at week 4 and QOLHEQ at week 6 (Table 7) were strong (range 0.66–0.87), providing evidence of convergent validity.

Table 7

Convergent validity correlations at week 4 (DLQI) and week 6 (QOLHEQ)

HEIS score	Convergent measure	Correlation	Coefficient
HEIS score	QOLHEQ (total score)	Pearson	0.87 (n=226)
		Spearman	0.86 (n=226)
HEIS PDAL score	QOLHEQ (limitations in functioning domain)	Pearson	0.82 (n=226)
		Spearman	0.81 (n=226)
	DLQI (daily activities domain)	Pearson	0.73 (n=240)
		Spearman	0.69 (n=240)
HEIS Emb score	QOLHEQ (emotions domain)	Pearson	0.82 (n=226)
		Spearman	0.79 (n=226)
	DLQI (symptoms and feelings domain)	Pearson	0.83 (n=240)
		Spearman	0.81 (n=240)
HEIS item 8: Feel frustrated	QOLHEQ (emotions domain)	Pearson	0.82 (n=226)
		Spearman	0.78 (n=226)
HEIS item 9: Hard to work	DLQI (work and school domain)	Pearson	0.66 (n=240)
		Spearman	0.67 (n=240)

Open in a separate window

DLQI Dermatology Life Quality Index, Emb embarrassment with the appearance of hands, PDAL proximal daily activity limitations, HEIS Hand Eczema Impact Scale, QOLHEQ Quality of Life in Hand Eczema Questionnaire

Known-Groups Validity

The HEIS score, HEIS PDAL score, and HEIS Emb score were compared among groups that differed in severity as measured by the PaGA, IGA-CHE, and relevant DLQI and QOLHEQ scores. For the HEIS score (Table 8), there was a pattern of significantly higher mean scores (indicating worse HRQoL) for participants who also scored higher (worse) on the PaGA, IGA-CHE, QOLHEQ (total score and item 6: itch), and DLQI (total score and item 1: itch, soreness, pain, stinging) (p<0.001). Effect sizes indicated differences between adjacent groups were moderate to large (ES range 0.72–2.16). Similar patterns of results were found for the HEIS PDAL scores (Table S8) and HEIS Emb scores (Table S9), providing evidence of known groups validity.

Table 8

Known-groups validity analysis of the HEIS total score at week 6 or week 8

Grouping variable	n	Mean HEIS score (SD)	Between groups effect size	Linear trends p-value	Pairwise tests p-value
Week 6: QOLHEQ total score
<Median score	103	0.47 (0.49)		N/A
≥Median score	111	1.64 (0.93)	−1.57		<0.0001
Week 6: QOLHEQ (item 6: Itch)
<Median score	68	0.56 (0.72)		N/A
≥Median score	146	1.32 (0.95)	−0.86		<0.0001
Week 8: PaGA
0–1 (clear or almost clear)	44	0.27 (0.28)		<0.0001
2 (mild)	100	0.78 (0.53)	−1.09		<0.0001
3–4 (moderate or severe)	74	1.97 (0.97)	−2.16		<0.0001
Week 8: IGA-CHE
0–1 (clear or almost clear)	49	0.47 (0.55)		<0.0001
2 (mild)	99	0.97 (0.76)	−0.72		<0.0001
3–4 (moderate or severe)	70	1.65 (1.09)	−1.30		<0.0001
Week 8: DLQI total score
<Median score	96	0.36 (0.33)		N/A
≥Median score	121	1.65 (0.89)	−1.84		<0.0001
Week 8: DLQI (item 1: Itchy, sore, painful, stinging)
<Median score	25	0.24 (0.25)		N/A
≥Median score	192	1.19 (0.95)	−1.06		<0.0001

Open in a separate window

DLQI Dermatology Life Quality Index, HEIS Hand Eczema Impact Scale, IGA-CHE Investigator Global Assessment of Chronic Hand Eczema, PaGA Patient Global Assessment of Disease Severity, SD standard deviation, QOLHEQ Quality of Life in Hand Eczema Questionnaire

Ability to Detect Change

Changes in HEIS scores, HEIS PDAL scores, and HEIS Emb scores were compared among participants defined as improved, stable, and worsened on the basis of their PaGA, IGA-CHE, and relevant DLQI and QOLHEQ scores between baseline and week 16. These results provide evidence that the HEIS score and multi-domain scores can detect change over time, regardless of the rating used to define change. As presented in Table 9, for all anchors there was strong evidence that the HEIS score was responsive to improvements over time, with consistently large within-group effect sizes (ES≥1.36) and statistically significant improvements in the improved groups compared with small-to-moderate effect sizes in the stable group (ES range 0.05–0.42). The number of participants who worsened across all grouping variables was extremely low (n≤13) and therefore should be interpreted with caution. Differences between change groups were statistically significant, and effect sizes were large between groups (≥1.11). Results were similar and equally strong for the HEIS PDAL scores (Table S10) and HEIS Emb scores (Table S11).

Table 9

Ability of the HEIS score to detect change between baseline and week 16

Grouping variable	n	Mean change score (SD)	Median change score (min–max)	Within groups effect size	Between groups effect size	Between groups p-value
PaGA
≥1 point improvement	134	−1.49 (0.87)	−1.56 (−3.4 to 0.2)	−1.68
Change score=0	57	−0.22 (0.69)	−0.33 (−1.4 to 1.9)	−0.21	−1.34
≥1 point worsening	13	0.45 (0.79)	0.33 (−0.8 to 2.1)	0.53	−0.65	<0.001
IGA-CHE
≥1 point improvement	146	−1.33 (0.97)	−1.33 (−3.4 to 2.1)	−1.36
Change score=0	52	−0.28 (0.87)	−0.33 (−2.1 to 1.9)	−0.31	−1.11
≥1 point worsening	6	0.30 (0.60)	0.39 (−0.6 to 1.2)	0.30	−0.64	<0.001
PGI-C
A little better+Much better	100	−1.40 (0.98)	−1.44 (−3.2 to 2.1)	−1.52
No change	20	0.05 (0.76)	0.11 (−1.6 to 1.7)	0.05	−1.54
A little worse+Much worse	6	0.04 (0.72)	−0.28 (−0.7 to 1.0)	0.03	0.01	<0.001
DLQI (total score)
≥4 point improvement	111	−1.67 (0.83)	−1.67 (−3.4 to 0.2)	−2.15
<4 point change	81	−0.40 (0.59)	−0.44 (−1.8 to 1.3)	−0.42	−1.49
≥4 point worsening	12	0.89 (0.70)	0.72 (0.2–2.1)	1.13	−1.38	<0.001
QOLHEQ (total score)
≥13 point improvement	130	−1.49 (0.89)	−1.50 (−3.4 to 0.4)	−1.65
<13 point change	66	−0.27 (0.73)	−0.22 (−2.1 to 1.9)	−0.26	−1.28
≥13 point worsening	8	0.49 (1.21)	0.28 (−1.8 to 2.1)	0.49	−0.73	<0.001

Open in a separate window

DLQI Dermatology Life Quality Index, HEIS Hand Eczema Impact Scale, IGA-CHE Investigator Global Assessment of Chronic Hand Eczema, PaGA Patient Global Assessment of Disease Severity, PGI-C Patient Global Impression of Change, QOLHEQ Quality of Life in Hand Eczema Questionnaire, SD standard deviation

Interpretation of Scores

Correlations between change in HEIS score, HEIS PDAL score, and HEIS Emb score and change in theoretically linked measures revealed that all suggested anchors had a moderate-to-large relationship (r>0.40) and were therefore suitable for use in the meaningful change analysis.

The means and confidence intervals (CIs) arising from a mixed effects model for repeated measures (MMRM) assessing group level HEIS change over time were assessed. Anchor-based within-group mean change estimates provided a range of values that could be considered plausible as the meaningful change threshold for a within-individual responder definition: for the HEIS total score these estimates ranged from 0.90 to 1.35 on the basis of the range of minimally improved anchor categories at week 16. As there was no moderately improved group for this anchor, it could be argued that emphasis should be placed on the upper end of this range. For the HEIS PDAL score, estimates ranged from 1.01 to 1.34 using the upper CI of the minimally improved group and lower CI of the moderately improved group. Finally, for the HEIS Emb score the estimates ranged from 1.13 to 1.66 using the same method as for the HEIS PDAL score. ROC curves provided support for these ranges of meaningful change thresholds for each score (see Fig. 4 as an example for the HEIS score; for examples of the HEIS PDAL score and HEIS Emb score, respectively, see Figs. S5, S6, S7, and S8 in the Online Supplementary Material). Empirical cumulative distribution function (eCDF) and probability density function (PDF) curves also supported the ranges of possible meaningful change thresholds suggested by the anchor-based analyses (see Figs. S9, S10, and S11 in Online Supplementary Material for example eCDF curves). Between-group minimal important differences (MIDs) were also estimated with plausible thresholds of around 0.60 for the HEIS score and 1.0 for the HEIS PDAL score and HEIS Emb score.

Open in a separate window

Fig. 4

ROC analysis of change from baseline HEIS score predicting QOLHEQ baseline to week 16 responder status in the psychometric analysis population

The estimates derived from these anchor-based methods were triangulated to form recommended thresholds for within-individual responder definitions. Estimates were summarized on forest plots to identify convergence around a small range of values (see Figs. S12, S13, and S14 in the Online Supplementary Material). These results suggest that a change of 1.3 points may be an appropriate threshold for defining a within-individual responder definition threshold for the HEIS score and HEIS PDAL score, and a responder definition of 1.5 may be an appropriate threshold for defining meaningful change for the HEIS Emb score.

Discussion

This study reports on the development of the HEIS, a CHE-specific measure of impacts on functioning and HRQoL associated with CHE, developed and validated according to best practice methods for COA development and regulatory guidance [12–15]. The intention was to develop a measure that is fit-for-purpose to provide endpoint measurements in clinical trials and clinical practice to support the development of therapies and clinical management practices to improve HRQOL in patients with CHE. The HEIS comprises nine items reflecting the HRQoL concepts identified as most important and relevant to patients in previous qualitative research [10] and the published literature [30–32]. As reported in this study, all HEIS items were found to be highly relevant to most patients in both the qualitative and psychometric work and well understood by patients in the qualitative interviews.

There were low levels of missing data throughout the phase 2b trial, which could suggest the HEIS is not overly burdensome to complete. The evaluation of item response distributions suggests that the response scales for each item are capable of capturing variability in impact scores, as well as changes in these scores over time. Inter-item correlations and CFA generally supported the hypothesized domain structure of the 9-item HEIS, and the very high internal consistency reliability results provide further support that the HEIS score is unidimensional and all items assess HRQoL as a single underlying trait. However, the item assessing “hard to hold/grip objects” did not load on the HEIS PDAL domain as expected. Qualitative evidence suggested that this item assesses an important concept not captured by any other items in the measure; therefore, the bi-factor model was taken forward with “hard to hold/group objects” removed from the HEIS PDAL domain score but retained on the overall general factor (or total score).

The final 9-item HEIS demonstrated strong construct validity, reliability (both internal consistency and test–retest reliability), and the ability to detect change over time in the specific context of adult patients with mild to severe CHE. The triangulation of various anchor-based methods suggested that within-individual responder definitions of 1.3 for improvements in HEIS score and HEIS PDAL score and of 1.5 for improvements in HEIS Emb score may be appropriate thresholds for defining minimal meaningful change in patients with mild to severe CHE.

Strengths of this research include the capture of insights from 40 patients with CHE in separate concept elicitation (n=20) [10] and cognitive debriefing (n=20) interviews during instrument development and validation. Moreover, the subsequent psychometric evaluation of the HEIS was performed in accordance with regulatory guidance for assessing the measurement properties of COAs and for obtaining representative patient input during clinical development [12, 13, 25, 26].

One limitation of this study is that the HEIS was developed and tested only with patients in the US. To establish broader relevance across countries, there would be value in assessing content validity in a non-US population. Further, most study participants, particularly in the development phase of the research, were female (n=14/20, 70.0%); however, there was still adequate representation of male participants. As HE is more prevalent in women than in men [33], it could be argued that the sample is reflective of the general CHE population. Another limitation is that for the psychometric validation, participants were predominantly white/Caucasian, which introduces racial bias. Future confirmation of psychometric validity in a more racially diverse population would be of value. All psychometric evaluation to date has also been performed in clinical trial samples. If the HEIS is to be used in general clinical practice, further evaluation in a “real-world” sample is recommended to confirm the generalizability of the measurement properties.

Conclusion

The HEIS is a CHE-specific PRO measure developed and validated in line with regulatory guidance to capture the subjective experience of CHE on key domains of HRQoL over the past 7 days. Strong evidence of content validity of the HEIS was established in cognitive debriefing interviews with patients with CHE. Psychometric analyses showed strong evidence of construct validity, reliability, and the ability to detect change, and that an improvement of≥1.3 points in both HEIS score and HEIS PDAL score, and improvements of≥1.5 points in HEIS Emb score, represent clinically meaningful, important changes. The HEIS was included as a secondary endpoint in the pivotal CHE phase 3 trials (NCT04871711, NCT04872101) to measure patient-reported CHE impacts [34].

Supplementary Information

Below is the link to the electronic supplementary material.

Acknowledgements

Author Contributions

Rob Arbuckle, Laura Grant and Lotte Seiding Larsen were involved throughout the design, data collection and analysis of the qualitative cognitive interviews. Rob Arbuckle, Laura Grant, Amy Jones, Piper Fromy and Lotte Seiding Larsen contributed to the planning of the psychometric analyses for the data from the ph2b clinical trial. All authors (Rob Arbuckle, Laura Grant, Lotte Seiding Larsen, Amy Jones, Piper Fromy, Elke Weisshaar, Yasemin Topal Yuksel, Tove Agner, Cherry Lou Balita-Crisostomo, Nanna Nyholm Mathiasen, Henrik Thoning and Christian Apfelbacher) contributed to interpretation of study results described in the manuscript and provided input into the manuscript and reviewed and approved the final manuscript.

Funding

Adelphi Values was commissioned by LEO Pharma A/S to conduct this research and the sponsor contributed to the study design, data collection, and preparation of the manuscript for publication. The sponsor is funding the journal’s rapid service fee.

Data Availability

The datasets generated and/or analyzed from the qualitative interviews are not publicly available in order to protect participant confidentiality. The datasets generated and/or analyzed during psychometric validation are available from the corresponding author on reasonable request.

Declarations

References