Europe PMC

This website requires cookies, and the limited processing of your personal data in order to function. By using the site you are agreeing to this as outlined in our privacy notice and cookie policy.

A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models.

Author information

Affiliations

  • 1. Department of Development & Regeneration, KU Leuven, Herestraat 49 box 805, Leuven, 3000 Belgium.
      Authors
    • Christodoulou E 1
    (1 author)
  • 2. Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Botnar Research Centre, University of Oxford, Windmill Road, Oxford, OX3 7LD UK.
      Authors
    • Ma J 2
    (1 author)
  • 3. Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Botnar Research Centre, University of Oxford, Windmill Road, Oxford, OX3 7LD UK; Oxford University Hospitals NHS Foundation Trust, Oxford, UK.
      Authors
    • Collins GS 3
    (1 author)
  • 4. Department of Biomedical Data Sciences, Leiden University Medical Centre, Albinusdreef 2, Leiden, 2333 ZA The Netherlands.
      Authors
    • Steyerberg EW 4
    (1 author)
  • 5. Department of Development & Regeneration, KU Leuven, Herestraat 49 box 805, Leuven, 3000 Belgium; Department of Public Health & Primary Care, KU Leuven, Kapucijnenvoer 33J box 7001, Leuven, 3000 Belgium; Nuffield Department of Primary Care Health Sciences, University of Oxford, Woodstock Road, Oxford, OX2 6GG UK.
      Authors
    • Verbakel JY 5
    (1 author)
    • Show all (6)

ORCIDs linked to this article

Journal of Clinical Epidemiology, 11 Feb 2019, 110:12-22
https://doi.org/10.1016/j.jclinepi.2019.02.004 PMID: 30763612 

Review

A comment on this article appears in "Statistics versus machine learning: definitions are interesting (but understanding, methodology, and reporting are more important)." J Clin Epidemiol. 2019 Dec;116:137-138. A comment on this article appears in "Statistical thinking, machine learning." J Clin Epidemiol. 2019 Dec;116:136-137.

Abstract 

Objectives

The objective of this study was to compare performance of logistic regression (LR) with machine learning (ML) for clinical prediction modeling in the literature.

Study design and setting

We conducted a Medline literature search (1/2016 to 8/2017) and extracted comparisons between LR and ML models for binary outcomes.

Results

We included 71 of 927 studies. The median sample size was 1,250 (range 72-3,994,872), with 19 predictors considered (range 5-563) and eight events per predictor (range 0.3-6,697). The most common ML methods were classification trees, random forests, artificial neural networks, and support vector machines. In 48 (68%) studies, we observed potential bias in the validation procedures. Sixty-four (90%) studies used the area under the receiver operating characteristic curve (AUC) to assess discrimination. Calibration was not addressed in 56 (79%) studies. We identified 282 comparisons between an LR and ML model (AUC range, 0.52-0.99). For 145 comparisons at low risk of bias, the difference in logit(AUC) between LR and ML was 0.00 (95% confidence interval, -0.18 to 0.18). For 137 comparisons at high risk of bias, logit(AUC) was 0.34 (0.20-0.47) higher for ML.

Conclusion

We found no evidence of superior performance of ML over LR. Improvements in methodology and reporting are needed for studies that compare modeling algorithms.

Full text links 

Read article at publisher's site: https://doi.org/10.1016/j.jclinepi.2019.02.004

Read article for free, from open access legal sources, via Unpaywall: https://ora.ox.ac.uk/objects/uuid:4167c905-5171-4fc7-b157-ff36d56a73db/files/m0c822b1760523fd587b9a6769e616049Unpaywall

Citations & impact 

Impact metrics

601
Citations
Jump to Citations

Citations of article over time

Alternative metrics

Altmetric item for https://www.altmetric.com/details/55313891
Altmetric
Discover the attention surrounding your research
https://www.altmetric.com/details/55313891

Smart citations by scite.ai
Smart citations by scite.ai include citation statements extracted from the full text of the citing article. The number of the statements may be higher than the number of citations provided by EuropePMC if one paper cites another multiple times or lower if scite has not yet processed some of the citing articles.
Explore citation contexts and check if this article has been supported or disputed.
https://scite.ai/reports/10.1016/j.jclinepi.2019.02.004

Supporting
Mentioning
Contrasting
88
836
4

Article citations

Go to all (601) article citations

Similar Articles 

To arrive at the top five similar articles we use a word-weighted algorithm to compare words from the Title and Abstract of each citation.

Funding 

Funders who supported this work.

Cancer Research UK (3)

FWO (1)

KU Leuven (1)

NIHR Biomedical Research Centre

    National Institute for Health Research (NIHR) (1)