Performance of Common Genetic Variants in Breast-Cancer Risk Models

METHODS

RESULTS

The associations between each predictor and breast-cancer risk were consistent with published data, and they were similar in the different models (Table 2 in the Supplementary Appendix). Notably, the effect of a family history of breast cancer did not materially change after adjustment for SNPs. The risk of breast cancer among women who carried 13 or more of a maximum of 20 riskconferring variant alleles (which occur in about 4% of the population) was nearly three times the risk among women who carry 6 or fewer variants (12%).

Table 3 in the Supplementary Appendix shows a comparison of how the models discriminated between women with breast cancer and those without the disease. The AUC for the demographic model was 53.4%. The addition of the SNP from the fibroblast growth factor receptor 2 gene (FGFR2) increased the AUC to 55.7%. The addition of four more SNPs (from Tox high-mobility group box family member 3 [TOX3], mitogenactivated protein kinase kinase kinase 1 [MAP3K1], chromosome 1p, and a RAD51-like 1 [RAD51L1]) increased the AUC further, to 57.9%. The AUC for the 7 SNPs described by Pharoah et al.¹⁵ was 58.6%, and the addition of the 3 others to incorporate all 10 SNPs listed in Table 3 in the Supplementary Appendix increased the AUC only to 59.7%. The discrimination in the simpler genetic variant-count model was somewhat less (58.9%).

Among the individual nongenetic factors, a history of breast biopsy yielded the greatest AUC (56.2%). The nongenetic model, with the use of all four components of the Gail model, showed an AUC of 58.0%. The inclusive model, with components of the Gail model and 10 SNPs, yielded an AUC of 61.8%, an increase of about 3.8 percentage points over the nongenetic model and 2.1 percentage points over the genetic individualvariant model. In short, the AUC for the genetic models was only slightly higher than the AUC for the nongenetic model, and the addition of the components from the Gail model increased the AUC by 2 more percentage points; thus, discrimination by means of the inclusive model is better than discrimination by means of either the genetic or nongenetic models. In parallel, integrated discrimination improvement, or the difference in the estimated annual risk of breast cancer between case subjects and control subjects, ranged from 0.015 to 0.022% in the inclusive model (Table 3 in the Supplementary Appendix). Our empirical analysis indicated a modest increment in discrimination by means of the inclusive model in relative and absolute terms, despite very low P values.

We also empirically examined the performance of high and low absolute risk (i.e., the performance of the models for women who might be singled out for more aggressive or less aggressive intervention than women at average risk). The genetic models yielded better discrimination than the nongenetic models among women at the lowest estimated risk (Fig. 1B), but not among women at the highest estimated risk (Fig. 1C). For women with a risk estimated from the inclusive model that was above the median in control subjects, the AUC was 58.8%, but it was only 56.6% for women with an estimated risk that was above the 80th percentile.

Table 4 in the Supplementary Appendix shows how the addition of genetic data affected classification of the case subjects with breast cancer. The percentages of all case subjects according to quintiles of absolute risk are shown for the inclusive and nongenetic models. The estimated annual risk was above 0.575% for substantially more case subjects with the inclusive model than with the nongenetic model (27.7% vs. 18.9%). Slightly fewer than half the case subjects (47.2%) were in the same category in both models. A total of 32.5% of the case subjects were in a higher-risk category with the inclusive model than with the nongenetic model, and 20.4% were in a lower category. The corresponding percentages for the control subjects were 26% and 28%. We found no evidence that a complex model involving interactions among Gail model components and SNPs performed materially better than the models described here (P>0.10).

DISCUSSION

In our study involving 5590 case subjects with breast cancer and 5998 control subjects, the addition of information on 10 genetic variants to a standard clinical breast-cancer risk model predicted the risk of breast cancer only slightly better than the clinical model alone. In the inclusive model, the ROC curve was above 60% of the possible AUC. That is, about 60% of the time, a randomly selected patient with breast cancer had a higher estimated risk than the risk for a randomly selected woman in whom breast cancer did not develop during the follow-up period. By contrast, a single dichotomous risk factor detected in 60% of case subjects and 40% of control subjects (odds ratio, 2.25; AUC, 60%) would discriminate about as well as our model by the AUC criterion. We saw similarly modest improvement using measures based on the change in estimated risk. Although our data suggest that the SNP-only model predicted risk slightly better than the Gail model, the nongenetic clinical variables are available at essentially no cost, whereas the costs of obtaining genetic information are likely to be substantial.

We have evaluated risk-prediction models as we would any other clinical test. Our results are presented in terms of absolute risks, which are easily translated into positive and negative predictive values in considering who will benefit from a clinical test, including a risk model.^24-26 Our findings were based on a large number of case subjects and control subjects, drawn from four prospective cohorts and one population-based case–control study. Our inclusive model incorporated information on components of the Gail model and genotypes of newly established SNPs; this information is not often included together in the same study. Our analysis also had technical advantages over other risk-modeling efforts: we fitted the effects of age and cohort so as not to give the models credit for fit based on demographic factors; we were able to assess the effect of adding SNPs on estimated risk and on changes in risk category for case subjects and for a hypothetical population.

There were some unavoidable weaknesses of this study and its data sources, particularly in evaluating risk for clinical use. For example, we pooled data from four U.S. cohort studies and a case–control study from Poland; these studies had different designs and enrollment characteristics. We included only women of European ancestry in the empirical analysis, and we did not consider subtypes of breast cancer. As a group, the study participants are not representative of any specific population. Screening practice may vary within and among studies. Because of self-selection for participation in the studies, we expect that the average risks in the U.S. studies will be different from each other and from the U.S. average. Similarly, in a clinical context, we cannot expect the risk for an individual woman to be the same as the average risk for a population. Our estimates of the performance of the risk models may be slightly higher than can be expected in typical clinical settings because we report results with minor overfitting. Given the complexity of the model with 4 Gail model components and 10 SNPs, we chose to fit each factor without constraints but did not attempt to evaluate hundreds of interactions among the factors or with age and cohort. Our simple interaction models gave no suggestion of improved performance from interactions between genetic and Gail model components; more sophisticated modeling may improve the performance of these factors in predicting breast-cancer risk. Finally, comparisons of models are slightly unfavorable to the Gail model because we could not include the history of atypical hyperplasia and information on mammographic density. Although they are not routinely incorporated into the Gail model, these factors may improve the performance of future versions of the model.¹⁴

Our analysis indicates that the genetic variants we studied provide modest improvements in discrimination and prediction models, whether measured as the AUC, as a discrimination index, or as a change in position in broad bands of risk, such as might be used in clinical settings. We see little evidence of benefit from including genetic variants at the extremes of high and low risk, categories in which further stratification might be most valuable. This empirical demonstration of the potential benefit of adding SNP data to breast-cancer risk models based on individual data is generally consistent with theoretical predictions^15,27 that use published estimates of effect.

Because statistics such as the AUC and integrated discrimination improvement do not provide a readily intuitive sense of the clinical usefulness of these models, we focused much of our discussion on the degree of incremental improvement associated with adding the genetic variants. As in diabetes²⁸ and cardiovascular disease,²⁹ the addition of the common SNPs added little to the predictive value of the clinical models. On the basis of theoretical models, Gail³⁰ has shown that increases in the AUC similar to those observed here are not sufficiently large to improve meaningfully the identification of women who might benefit from tamoxifen prophylaxis or assignment of screening mammography.

Although the Gail model and SNP-inclusive models may help to identify groups of women who have an increased risk of breast cancer for trials of interventions, none of the models in our set of data accurately predicted the development of breast cancer. Our results indicate that the recent identification of common genetic variants does not herald the arrival of personalized prevention of breast cancer in most women. Even with the addition of these common variants, breast-cancer risk models are not yet able to identify women at reduced or elevated risk in a clinically useful way.

Supplementary Material