Abstract

INTRODUCTION

Despite decades of clinical research with multi-modality therapy, the overall prognosis of ovarian cancer is still largely determined by whether a patient is diagnosed at an early stage when the cancer has not spread beyond the ovary (1). At present there is no effective screening strategy for ovarian cancer that will detect disease at an early stage. Screening modalities evaluated to date have included transvaginal sonography (TVS) and the use of the serum tumor marker CA 125. Three major studies have evaluated TVS for early detection and have yielded a positive predictive value of less than 10%, at the margin of the specificity required for effective screening (2-4). CA 125, a high molecular weight mucin (MUC 16) (5) was first detected with a radioimmunoassay in patients with advanced ovarian cancer (6). CA 125 levels can be elevated 10-21 months prior to conventional diagnosis (7-9) and are increased in 58% of patients with stage I ovarian cancer (10-12). The specificity of individual values of CA 125 is not adequate for effective screening, particularly in a premenopausal population where endometriosis, adenomyosis and retrograde menstruation can increase CA 125 levels. Using a two-stage strategy, however, a combination of CA 125 followed by sonography does attain specificity in excess of 99.6% (13). Using this strategy, Jacobs, et al, randomized 21,962 postmenopausal women more than 45 years of age in the United Kingdom to a control group or to a screened group. CA 125 was measured annually for three years. An elevated CA125 level >30 units/mL prompted TAU and surgery was undertaken if the TAU demonstrated a pelvic mass. Among 10,985 women screened, 29 operations were performed to detect 6 cancers, providing a positive predictive value of 21. Median survival in the screened group (72.9 months) was significantly greater (p=0.0112) than that in the control group (41.8 months).

Use of CA 125 as an initial stage in a 2-step screening strategy is limited by the fact that CA 125 is expressed by only 80% of epithelial ovarian cancers. The sensitivity of such an approach is further reduced due to the inability of sonography to detect a mass that has yet to become appreciable. Multiple serum markers might be utilized to improve sensitivity. Over the years a large number of markers for ovarian cancer have been reported to complement CA 125 (14). Sensitivity has been improved by 5 to 15%, but specificity has inevitably been reduced. In a study of 89 sera from patients with stage I ovarian cancer, use of 3 markers in combination (CA 125, OVX1 and M-CSF) detected 84% of cancers, whereas CA 125 alone detected 69%. Specificity, however, declined from 99% to 84% with the combination (11, 12).

Novel mathematical analyses might improve sensitivity using multiple markers without sacrificing specificity. A number of different models have been suggested to combine multiple tumor markers. Statistical models such as multivariate logistic regression and nonlinear classification methods such as classification and regression tree (CART) have also been tested in the literature (15). Specificity of CA 125 could be further improved by following the trend of CA 125 with an algorithm that estimated the risk of ovarian cancer (16, 17).

Artificial neural network (ANN) analysis as a statistical modeling tool has demonstrated the ability to assimilate information from multiple sources and detect subtle and complex patterns. There have been a large number of reports on the use of ANN for clinical diagnosis (18), staging and prognosis (19), and management of cancers (20). In a previous study, an ANN-based composite diagnostic index derived using a panel of four serum markers, CA 125II, CA 72-4, CA 15-3, and lipid-associated sialic acid (LASA), was evaluated for it ability to discriminate malignant from benign pelvic masses (21). Using data from 255 patients with benign or malignant pelvic masses, the ANN index increased test specificity by approximately 20% over that of using CA125II alone, while maintaining a comparable level of sensitivity. Among an additional group of 196 healthy donors, the index had a specificity of 100%. A total of 18 patients with stage I or II epithelial ovarian cancer were included in the test set of this study. For these patients, CA 125II at the cutoff value of 35 U/ml had a sensitivity of 78% and a specificity of 68% for all benign cases. The ANN index, on the other hand, at a cutoff that yielded the same specificity as CA125II for all benign cases, provided a sensitivity of 88.9% for the stage I and II epithelial ovarian cancers.

Due to the small number of early stage ovarian cancer cases (n=18) in the previous study (21), statistical significance was not attained in comparing the performance of the ANN index against that of CA125 alone. In this paper, we studied sera from a larger number of early stage epithelial ovarian cancer patients to extend our evaluation of artificial neural network analysis for combining multiple serum tumor markers to form a single-valued index for detecting early stage ovarian cancer with both improved sensitivity and specificity compared to CA125II alone.

SUBJECTS AND METHODS

RESULTS

The allocations of specimens to the three data sets are listed in Table 1. In the following, results on the in-training cross-validation set were not reported due to the small number of epithelial ovarian cancer cases and the lack of any early stage cases.

In Tables 2, descriptive statistics of the four tumor markers and the ANN results of the training set (Table 2A) and the independent test set (Table 2B) are listed. Statistically significant differences (p < 0.000001) were noted between epithelial ovarian cancer and apparently healthy women in the training set for all tumor markers and the ANN. In the test set, the relative elevations of the four markers among cancer patients, in either means or medians, were smaller than those observed in the training set, reflecting possible differences among the clinical sites. However, the differences for the four markers and the ANN remained statistically significant (p = 0.000379 ~ < 0.000001).

Figure 1 compares side by side the histograms of CA 125II and the ANN-derived composite index for women in the test set. In Figure 1A, when CA 125II was >55 U/ml, the number of early stage epithelial ovarian cancer patients rose abruptly. However, a large number of the cancer patients (54%) were distributed over the low CA125 range of 5-35 U/ml and overlapped with the healthy women. When the ANN was plotted in Figure 1B, overlap was also observed, but the majority of patients with early stage ovarian cancer (71%) had an ANN value > 0.5, a range with very few overlapped healthy women.

Open in a separate window

Open in a separate window

Figure 1

Comparison of histograms of A) CA 125II and B) the ANN-derived composite index among healthy donors and early stage epithelial ovarian cancer patients in the independent test data.

The added value of the ANN is visually demonstrated in the scatter-plot in Figure 2 where the ANN results of the independent test samples are plotted against their CA125II results. In the figure, a fair number of early stage ovarian cancer patients with a relatively low serum CA125II level had an elevated ANN result. For example, at a cutoff of 0.5 and a corresponding false positive rate of 2%, the ANN could detect five (including four invasive) early stage ovarian cancer patients who had a CA125II < 35 U/ml. Furthermore, the ANN would be able to detect 13 (including 12 invasive) early stage ovarian cancer patients who would otherwise be missed by CA125II alone if the cutoff of CA125II was to be set to match the 2% false positive rate.

Open in a separate window

Figure 2

Scatter plot of the independent test data compares CA 125II against the ANN-derived composite index. Darkened lines indicate cutoff values of ANN at 0.5, CA125II at 35 U/ml, and CA125II with 2% false positive rate. Notice that a fair number of patients with early stage epithelial ovarian cancer yet a relatively low CA125II level had an elevated ANN index value. Note that some of the samples overlapped in the plot.

In ROC curve analysis using only data from healthy donors and early stage epithelial ovarian cancer patients, the ANN-derived composite index had a higher overall diagnostic power. On the training data set, the estimated areas-under-curve (AUCs) for CA 125II and the ANN were 0.9356 and 0.9756, respectively (p = 0.0429, figure not shown). On the external independent test data set, the AUCs for CA125II and ANN were 0.9089 and 0.9449, respectively (p = 0.0333, Figure 3). ROC curves for the other markers (CA 72.4, M-CSF and CA 15-3) were also plotted in Figure 3. Since these markers individually do not perform as well as CA125II, relative improvements in AUC by the ANN index over them were more significant (p = 0.0007 ~ <0.0001).

Open in a separate window

Figure 3

ROC curves and AUCs of individual tumor markers and the ANN-derived composite index based in the independent test data.

In the test data set, CA 125II exhibited a sensitivity of 65% for early stage ovarian cancer and 67% for invasive disease using a cut-off at 35 U/ml (Table 3), but there were 6% false positives. When the false positives were held constant at 2% for CA 125II and for the ANN, CA 125II detected only 46% of all early stage disease and 43% of invasive disease, whereas the ANN detected 71% of all early stage and 71% of invasive ovarian cancers. In Table 3, the separately estimated 95% confidence intervals of the sensitivities of CA125 and the ANN overlapped slightly. However, the differences in sensitivities between CA125 and the ANN were statistically significant in test for difference in paired proportions (p = 0.047 for all early stage epithelial cancers and p = 0.040 for invasive cases only).

DISCUSSION

Our analysis suggests that the combined use of multiple serum tumor markers through ANN improves both sensitivity and specificity for detection of Stage I epithelial ovarian cancer. In ROC curve analysis, ANN proved superior to the use of the best individual marker CA125II. At a fixed specificity of 98%, the differences in sensitivities between the ANN and CA125II were statistically significant for detecting all early stage cases and for detecting the invasive early stage cases only. The total number of early stage cases for independent test in the current study is still relatively small. Additional studies with larger early stage case populations may be needed to further strengthen the statistical power of the evidence. However, the differences, if established, would be clinically very significant. As shown by ROC curve comparison (figure 3), the present report supports the conclusion that ANN analysis of multiple markers can potentially permit improved sensitivity without sacrificing specificity when compared to the use of the individual markers alone.

Due to the relatively low prevalence yet high mortality of ovarian cancer, an effective screening strategy for ovarian cancer in general needs to attain minimally a positive predictive value of 10%, i.e. 10 operative procedures for each case of ovarian cancer detected. With a prevalence of 1 in 2,500 among postmenopausal women in the United States, this equates to requiring a test to have a sensitivity of 75% and a specificity above 99.7%. Even with the improved sensitivity and specificity, the current ANN algorithm using the four serum tumor markers by itself will not meet this requirement. ANN analysis of multiple tumor markers does, however, appear promising as an initial step in a two-stage strategy where an abnormal blood test triggers a second test such as transvaginal sonography (TVS). In early clinical studies, a combination of CA 125 and transabdominal ultrasound produced a specificity of 99.7% (17, 24). In general, in a two-stage test strategy, improvement in specificity (hence positive predictive value) is often at the expense of a lowered sensitivity (hence worse negative predictive value). A first-stage test with a much improved sensitivity while maintaining the same level of specificity will certainly make such a trade-off in the overall screening strategy a more viable choice. It needs to be noted, however, that the effectiveness of transvaginal sonography in differentiating malignant from benign masses could affected by access to quality ultrasound examination (e.g., equipment and experience of the operators) and patient conditions (e.g., menopausal status).

The objective of the current paper is to evaluate the performance of the ANN-derived index in detecting early stage epithelial ovarian cancer. The inclusion of late stage epithelial ovarian cancer in the training data set to increase the otherwise relatively small number of cancer cases was based on the assumption that at the molecular level, or at least as far as the four tumor markers were concerned, early and late stage ovarian cancers represent only the quantitative progression of the same disease. Consequently, the expression patterns among the multiple tumor markers captured by the ANN from the mixture of early and late stage patients may be applicable to the test data of healthy donors and early stage patients. A drawback of the current study is that specificity of the ANN was not assessed on independent test samples of patients with benign diseases. In a previous report, an ANN that was closely related to the current ANN (the two shared a significant portion of training samples and had a common design) had a specificity for benign cases in the 90% range (21).

The distributions of CA 125II among healthy donors and among early stage ovarian cancer patients are statistically significantly different (p < 0.000001). However, as indicated by the histogram plot in Figure 1A, CA 125II exhibits a distribution among healthy women with a low, yet long, tail that spreads across almost the entire distribution range of CA 125II in patients with ovarian cancer. As a result, the sensitivity of CA 125II deteriorates significantly when the cutoff is set for a high specificity. In Figure 1B, the overlap between the distributions of the ANN composite index in the two diagnostic groups is much smaller.

The longitudinal algorithm developed by Skates (16) assesses one’s risk of having ovarian cancer by detecting the change-point in sequential tests of the single assay CA 125II (i.e., the point at which CA 125II changing from normal biological variations to disease induced elevation). It was reported based on preliminary data that its overall positive predictive value exceeded the minimum requirement for it to be a viable tool for ovarian cancer screening. However, ovarian cancer is a heterogeneous disease. It could still be difficult for a single marker, even measured over time, to achieve a level of robust performance to allow for cost-effective screening of a general asymptomatic population. Result from the current study indicates the potential of combining multiple markers to improve both sensitivity and specificity. It is conceivable that by incorporating existing or newly discovered tumor markers in multivariate predictive models, a similarly devised longitudinal algorithm could improve significantly the sensitivity and specificity in detecting true early signs of ovarian cancer.

Acknowledgments

Footnotes

References