Europe PMC

This website requires cookies, and the limited processing of your personal data in order to function. By using the site you are agreeing to this as outlined in our privacy notice and cookie policy.

Abstract 


Background

Numerous meta-analyses have investigated the effect of exercise in different populations and for single cardiovascular risk factors, but none have specifically focused on the metabolic syndrome (MetS) patients and the concomitant effect of exercise on all associated cardiovascular risk factors.

Objective

The aim of this article was to perform a systematic review with a meta-analysis of randomized and clinical controlled trials (RCTs, CTs) investigating the effect of exercise on cardiovascular risk factors in patients with the MetS.

Methods

RCTs and CTs ≥4 weeks investigating the effect of exercise in healthy adults with the MetS and published in a peer-reviewed journal up to November 2011 were included. Primary outcome measures were changes in waist circumference (WC), systolic and diastolic blood pressure, high-density lipoprotein cholesterol (HDL-C), triglycerides and fasting plasma glucose. Peak oxygen uptake ([Formula: see text]) was a secondary outcome. Random and fixed-effect models were used for analyses and data are reported as means and 95% confidence intervals (CIs).

Results

Seven trials were included, involving nine study groups and 206 participants (128 in exercise group and 78 in control group). Significant mean reductions in WC -3.4 (95% CI -4.9, -1.8) cm, blood pressure -7.1 (95% CI -9.03, -5.2)/-5.2 (95% CI -6.2, -4.1) mmHg and a significant mean increase in HDL-C +0.06 (95% CI +0.03, +0.09) mmol/L were observed after dynamic endurance training. Mean plasma glucose levels -0.31 (95% CI -0.64, 0.01; p = 0.06) mmol/L and triglycerides -0.05 (95% CI -0.20, 0.09; p = 0.47) mmol/L remained statistically unaltered. In addition, a significant mean improvement in [Formula: see text] of +5.9 (95% CI +3.03, +8.7) mL/kg/min or 19.3% was found.

Conclusions

Our results suggest that dynamic endurance training has a favourable effect on most of the cardiovascular risk factors associated with the MetS. However, in the search for training programmes that optimally improve total cardiovascular risk, further research is warranted, including studies on the effects of resistance training and combined resistance and endurance training.

Free full text 


Sports Med. 2013; 43(2): 121–133.
Published online 2012 Dec 19. https://doi.org/10.1007/s40279-012-0003-z
PMCID: PMC3693431
PMID: 23329606

The Effect of Exercise on the Cardiovascular Risk Factors Constituting the Metabolic Syndrome

A Meta-Analysis of Controlled Trials

Abstract

Background

Numerous meta-analyses have investigated the effect of exercise in different populations and for single cardiovascular risk factors, but none have specifically focused on the metabolic syndrome (MetS) patients and the concomitant effect of exercise on all associated cardiovascular risk factors.

Objective

The aim of this article was to perform a systematic review with a meta-analysis of randomized and clinical controlled trials (RCTs, CTs) investigating the effect of exercise on cardiovascular risk factors in patients with the MetS.

Methods

RCTs and CTs ≥4 weeks investigating the effect of exercise in healthy adults with the MetS and published in a peer-reviewed journal up to November 2011 were included. Primary outcome measures were changes in waist circumference (WC), systolic and diastolic blood pressure, high-density lipoprotein cholesterol (HDL-C), triglycerides and fasting plasma glucose. Peak oxygen uptake (equation M1) was a secondary outcome. Random and fixed-effect models were used for analyses and data are reported as means and 95% confidence intervals (CIs).

Results

Seven trials were included, involving nine study groups and 206 participants (128 in exercise group and 78 in control group). Significant mean reductions in WC −3.4 (95% CI −4.9, −1.8) cm, blood pressure −7.1 (95% CI −9.03, −5.2)/−5.2 (95% CI −6.2, −4.1) mmHg and a significant mean increase in HDL-C +0.06 (95% CI +0.03, +0.09) mmol/L were observed after dynamic endurance training. Mean plasma glucose levels −0.31 (95% CI −0.64, 0.01; p = 0.06) mmol/L and triglycerides −0.05 (95% CI −0.20, 0.09; p = 0.47) mmol/L remained statistically unaltered. In addition, a significant mean improvement in equation M2 of +5.9 (95% CI +3.03, +8.7) mL/kg/min or 19.3% was found.

Conclusions

Our results suggest that dynamic endurance training has a favourable effect on most of the cardiovascular risk factors associated with the MetS. However, in the search for training programmes that optimally improve total cardiovascular risk, further research is warranted, including studies on the effects of resistance training and combined resistance and endurance training.

Introduction

The ‘metabolic syndrome’ (MetS) or ‘syndrome X’ is a constellation of cardiovascular risk factors including abdominal obesity, impaired glucose control, hypertension and dyslipidaemia [1]. The MetS components separately increase the risk for cardiovascular disease (CVD) and all-cause mortality, but the full syndrome is associated with an increase in risk that is greater than the risk of each separate risk factor [2]. Therapeutic lifestyle changes, including exercise, are recommended as firstline strategy in the treatment of cardiovascular risk factors [3]. Previous meta-analyses demonstrated that dynamic endurance exercise decreases blood pressure (BP) in hypertensive individuals [4, 5], causes a significant reduction in blood lipids and lipoproteins in patients with hyperlipidaemia [6, 7], reduces glycated haemoglobin (HbA1c; percentage) in patients with type 2 diabetes mellitus [7] and is associated with improved body composition in obese individuals [7]. However, to the best of the authors’ knowledge, meta-analyses have not been performed to determine the concomitant effect of exercise on all risk factors of the MetS in patients with the MetS.

Therefore, the aim of this study was to perform a systematic review with a meta-analysis of RCTs or CTs investigating the effect of exercise on cardiovascular risk factors in patients with the MetS.

Research Design and Methods

Literature Search

We conducted a systematic literature search with the MEDLINE computerized database from its inception to November 2011 using the following medical subject headings (MeSH terms): ‘Metabolic Syndrome X’ AND (‘Exercise’ OR ‘Motor Activities’). A second literature search was done in MEDLINE using the following title terms: ‘Metabolic Syndrome’ OR ‘Syndrome X’ AND ‘Exercise’. In addition, the reference lists from published original and review articles were searched manually to identify other possible eligible studies.

Inclusion Criteria

We included studies in the meta-analysis if the following requirements were met: (i) (RCTs) or CTs investigating the effect of exercise (dynamic endurance exercise—dynamic resistance exercise—or a combination); (ii) the duration of intervention was at least 4 weeks; (iii) intervention in adults (≥18 years) with the MetS (WHO 1998 [8], National Cholesterol Education Program Adult Treatment Panel III [NCEP/ATP III] 2001 [9] or International Diabetes Federation [IDF] 2005 [10]) but with no other CVD; (iv) full publication in a peer-reviewed journal up to November 2011.

Measured Outcomes

The primary outcomes were changes in cardiovascular risk factors associated with the MetS: i.e. waist circumference (WC), fasting plasma glucose, systolic and diastolic blood pressure (SBP, DBP), high-density lipoprotein cholesterol (HDL-C) and triglycerides. Secondary outcomes included peak oxygen uptake (equation M3), body weight, body mass index (BMI), fat mass, total cholesterol, low-density lipoprotein cholesterol (LDL-C) and plasma insulin.

Data Extraction

Two unblinded reviewers (N.P. and S.R.T.E.) independently conducted data extraction. A specifically developed data extraction sheet was used to extract data on study source, study design, characteristics of the participants, the exercise characteristics and the different outcomes in each study. Disagreements were solved by consensus or by a third reviewer (V.C.).

Study Quality

Study quality was assessed using an adapted Physiotherapy Evidence Database (PEDro) scale [11], which is an 11-item questionnaire designed to collect data on eligibility criteria, random allocation, concealed allocation, similarity of baseline values, blinding of therapists and/or assessors, key outcomes, intent-to-treat analysis, between group differences, and point and variability measures. We regarded the quality criteria ‘blinding of participants’ and ‘blinding of therapists’ as not applicable in exercise intervention studies, and omitted both criteria. All questions were binary (yes [1] or no [0]). The minimum score was 0 and the maximum was 9, with a higher number reflecting a better study quality. The PEDro-scale has been reported to be valid and reliable [12, 13].

Statistical Analysis

Statistical analyses were performed using SAS® version 9.3 (SAS Institute, Cary, NC, USA) and Review Manager Software (RevMan 5.1; Cochrane Collaboration, Oxford, UK). Descriptive data are reported as mean ± standard deviation (SD) or median and range. The mean baseline values were calculated by combining mean values from the intervention and control groups, weighted by the total number of participants in each study group. Since we worked with continuous data, we took the approach of assuming that randomization would adjust for baseline differences and used endpoint data only in the meta-analytical statistics. Each effect was weighted by the inverse of its variance. The results were combined using fixed and random effect models and presented with 95% confidence intervals (CI). Heterogeneity between trial results was tested with a standard Chi-squared (χ²) test. The I² parameter was used to quantify any inconsistency (I² = [(Q − df)] × 100%, where Q is the χ² statistic and df are the degrees of freedom). A value for I² > 50% has been considered to be substantial heterogeneity. To examine the influence of each study on the overall results, analyses were also performed with each study deleted from the model. In addition, we performed a simple meta-regression analysis to investigate whether changes in the primary outcomes were related to changes in WC. Two-sided tests for overall effects were considered significant at p ≤ 0.05. Finally, funnel plots were used to assess the potential of small publication bias.

Results

Literature Search

From 822 potentially relevant studies retrieved from our search, 9 trials met our inclusion criteria [1422]. A flow diagram of literature search and selection is presented in Fig. 1. Two studies were a duplicate of another study, i.e. using the same population and intervention, and so we included the most complete publication [1719, 22]. Hence, seven trials were included in the final analysis. Some of the trials applied different training regimens [17, 18] so that a total of nine dynamic endurance training interventions, one dynamic resistance training group and two combined dynamic endurance/resistance training groups were available for qualitative analysis. We used the meta-analytic techniques only on the nine dynamic endurance training groups.

An external file that holds a picture, illustration, etc.
Object name is 40279_2012_3_Fig1_HTML.jpg

Prisma 2009 flow diagram

Characteristics of the Participants and Study Designs

Characteristics of the 12 included intervention groups are summarized in Table 1. All studies were published between 2003 and 2010. Five of them were RCTs [15, 17, 18, 20, 21] and two of them were CTs [14, 16]. Diagnosis of the MetS was based on WHO [8] criteria [14, 18], ATP III [9] criteria [16] or IDF [10] criteria [17, 20, 21]. One trial [15] used a mixture of the three definitions i.e. stage 1 or 2 hypertension (SBP 140–159 or ≥160 mmHg; DBP 90–99 or ≥100 mmHg); plasma insulin concentration greater than 80 μU/mL (>556 pmol/L) 2 h after the ingestion of 75 g of glucose; dyslipidaemia, defined as high triglyceride (≥150 mg/dL; ≥1.7 mmol/L) and/or low HDL-C levels (men ≤40 mg/dL; ≤1.04 mmol/L, women ≤50 mg/dL; ≤1.3 mmol/L).

Table 1

Descriptive characteristics of the study groups at baseline and characteristics of the intervention

Study (year)Study designMetS criteriaMedicationGroupPre/post (n)Sex (M/F)Mean age (years)Duration (weeks)InterventionOutcomes
Dynamic endurance training
 Dumortier et al. [14] (2003)CTWHO 1998NoEx28/287/21528F: 3×/weekMetS: WC, plasma glucose, HDL-C, TG
Con11/114/753I: maximal lipid oxidation (defined by exercise calorimetry)Other: equation M4, WHR, BW, BMI, FM, FFM (PBF), plasma insulin, TC, LDL-C
T: 40 min
T: aerobic continuous training (cycling)
 Watkins et al. [15] (2003)RCTOthera NoEx21/14BothNA26F: 3–4×/weekMetS: fasting glucose, SBP, DBP, HDL-C, TG
Con11/11BothNAI: 70–85% HRres Other: equation M5, BW, BMI, PBF, fasting insulin, TC, LDL-C
T: 55 min (10 min warm-up, 35 min at training intensity, 10 min cool-down)
T: aerobic continuous training (cycling, walking, jogging)
 Gomes et al. [16] (2008)CTATP III 2001Yes, except lipid medicationEx18/18NA4612F: 3×/weekMetS: WC, glucose, SBP, DBP, HDL-C, TG
Con10/10NA46I: HR on anaerobic threshold, 10% below RCPOther: BMI, TC, LDL-C, HR
T: 45 min
T: aerobic continuous training (cycling)
 Irving et al. [17] (2008)RCTIDF 2005NoEx 112/90/95116F: 5×/weekMetS: WC, fasting glucose, SBP, DBP, HDL-C, TG
Ex 215/110/115116I: 2 sessions below lactate threshold (RPE 10–12) and 3 sessions between lactate threshold and equation M6 (RPE 15–17)Other: equation M7, BW, BMI, FM, FFM (PBF)
Con9/90/951T: time depending on energy expenditure
T: high-intensity aerobic continuous training (walking, running)
F: 5×/weekMetS: WC, fasting glucose, SBP, DBP, HDL-C, TG
I: below lactate threshold (RPE 10–12)Other: equation M8, BW, BMI, FM, FFM (PBF)
T: time depending on energy expenditure
T: low-intensity aerobic continuous training (walking, running)
 Tjønna et al. [18] (2008)RCTWHO 1998YesEx 112/114/75516F: 3×/weekMetS: WC, WHR, fasting glucose, insulin, SBP, DBP, HDL-C, TG
Ex 210/84/45216I: 10 min warm-up 70% HRpeak, 4 × 4 min 90% HRpeak and 3 × 3 min 70% HRpeak, 5 min min cool-downOther equation M9, BW, BMI
Con10/95/450T: 40 min
T: aerobic interval training (walking, running uphill)
F: 3×/weekMetS: WC, fasting glucose, SBP, DBP, HDL-C, TG
I: 70% HRpeak Other: equation M10, WHR, BW, BMI, insulin
T: 47 min
T: aerobic continuous training (walking, running uphill)
Balducci et al. [20] (2010)RCTIDF 2005YesEx 120/1812/86452F: 2×/weekMetS: WC, SBP, DBP, HDL-C, TG
Con20/2011/961I: 70-80% equation M11
T: 60 minOther: equation M12, BW, BMI, PBF, TC, LDL-C, HbA1c, HOMA-index
T: aerobic continuous training (walking, jogging, cycling)
Stensvold et al. [21] (2010)RCTIDF 2005YesEx 111/11Both5012F: 3×/weekMetS: WC, glucose, SBP, DBP, HDL-C, TG
Con11/11Both47I: 10 min warm-up 70% HRpeak, 4 × 4 min 90–95% HRpeak and 4 × 3 min 70% HRpeak, 5 min cool-downOther: equation M13, BW, BMI, FM, FFM (PBF), TC, HbA1c
T: 43 min
T: aerobic interval training (walking, running)
Dynamic resistance training
 Stensvold et al. [21] (2010)RCTIDF 2005YesEx 211/10Both5112F: 3×/week (2 × programme 1, 1 × programme 2)MetS: WC, glucose, SBP, DBP, HDL-C, TG
Con11/11Both47I: warm-up 2 sets, 15–20 repetitions, 40–50% 1RM, week 1: 60% 1RM; from week 2 on: 3 sets, 8–12 repetitions, 80% 1RMOther: equation M14, BW, BMI, FM, FFM (PBF), TC, HbA1c
T: programme 1: 40 min; programme 2: 50 min
T: dynamic resistance training, programme 1: 3 exercises; programme 2: 5 exercises
Combined intervention: dynamic endurance + dynamic resistance training
 Balducci et al. [20] (2010)RCTIDF 2005YesEx 222/2014/86152F: 2×/weekMetS: WC, SBP, DBP, HDL-C, TG
Con20/2011/961I: aerobic endurance training 70–80% equation M15 dynamic resistance training 80% 1RMOther: equation M16, BW, BMI, PBF, TC, LDL-C, HbA1C, HOMA-index
T: aerobic endurance training 40 min dynamic resistance training 20 min
T: aerobic continuous training (walking, jogging, cycling) dynamic resistance training (4 exercises)
Stensvold et al. [21] (2010)RCTIDF 2005YesEx 310/9Both5312F: 3×/week (2 × aerobic endurance training, 1 × dynamic resistance training)MetS: WC, glucose, SBP, DBP, HDL-C, TG
Con11/11Both47I: aerobic endurance training 10 min warm-up 70% HRpeak, 4 × 4 min 90–95% HRpeak and 4 × 3 min 70% HRpeak, 5 min cool-down dynamic resistance training warm-up 2 sets, 15–20 repetitions, 40–50% 1RM, week 1: 60% 1RM; from week 2 on: 3 sets, 8–12 repetitions, 80% 1RMOther: equation M17, BW, BMI, FM, FFM (PBF), TC, HbA1c
T: aerobic endurance training 43 min dynamic resistance training 40 min
T: aerobic interval training (walking, running) dynamic resistance training (3 exercises)

1RM one repetition maximum, ATP III Adult Treatment Panel III, BMI body mass index, BW body weight, Con control group, CT controlled trial, DBP diastolic blood pressure, Ex exercise group (numbered as 1, 2 and 3 when more intervention groups within one study), F female, F: I: T: T: frequency: intensity: time: type:, FFM fat free mass, FM fat mass, HbA 1c glycated haemoglobin, HDL-C high-density lipoprotein cholesterol; HR heart rate, HR peak peak HR; HR res HR reserve; IDF International Diabetes Federation, LDL-C low-density lipoprotein cholesterol, M males, MetS metabolic syndrome (cardiovascular outcomes constituting the MetS), n number of participants; NA not available, Other additional outcomes of the included trials, PBF percent body fat, Post after the intervention, Pre before the intervention, RCP respiratory compensation point, RCT randomized controlled trial, RPE rate of perceived exertion, SBP systolic blood pressure, TC total cholesterol, TG triglycerides, equation M18 peak oxygen uptake, w women, WC waist circumference, WHR waist hip circumference

aInclusion criteria of Watkins et al.: stage 1 or 2 hypertension (SBP 140–159 or ≥160 mmHg; DBP 90–99 or ≥100 mmHg); plasma insulin concentration greater than 80 μU/mL (>556 pmol/L) 2 h after the ingestion of 75 g of glucose; dyslipidaemia, defined as high TG (≥150 mg/dL; ≥1.7 mmol/L) and/or low HDL-C levels (men ≤ 40 mg/dL; ≤1.04 mmol/L, women ≤ 50 mg/dL; ≤1.3 mmol/L)

272 participants were randomized or allocated to an exercise group (dynamic endurance training, n = 147; dynamic resistance training, n = 11; combination, n = 32) or a control group (n = 82). Average drop out was 10% (range 0–33%), resulting in a total of 245 participants that could be included in the final analysis: 128 of them completed a dynamic endurance training intervention, 10 completed a dynamic resistance training intervention, 30 completed a combined intervention and 78 were control subjects.

Median age was 52 years (range 46–64). Six study trials included both men and women, and one trial included only women [17] (±38% and ±41% males in the exercise and control group, respectively) [15]. Four trials defined ‘not being sedentary’ as an exclusion criterion [1417], while this was not specifically reported in the other three trials [18, 20, 21]. Two trials did not report on diabetes status, four study trials excluded individuals with diabetes [1417] whereas Balducci et al. [20] included only MetS patients with diabetes. The use of medication was reported in three trials [18, 20, 21]; antihypertensive agents, lipid-lowering agents and oral hypoglycaemic agents or insulin use were common pharmacotherapy treatments. Three other trials defined the use of medications as an exclusion criterion [14, 15, 17]. Gomes et al. [16] defined drugs known to affect lipoprotein metabolism as an exclusion criterion.

The 12 interventions lasted between 8 and 52 weeks (median 16). The frequency of training varied between two and five training sessions weekly (median 3), with an average of 47.5 min per session (range 40–60). All trials used walking, jogging or cycling as the modes of exercise. Two dynamic endurance training groups performed interval training [18, 21], while the other seven endurance training groups reported using a continuous training programme. Intensities ranged from moderate (e.g. ‘maximal lipid oxidation’) to high (e.g. intervals at 90–95% of heart rate peak [HRpeak]). The exercise programmes were supervised in all studies except for Dumortier et al. [14] where only the first training sessions were supervised and the patients continued the training at home afterwards.

In Table 2, the results of the adapted PEDro-scale are represented. The median PEDro score was 6, with a range from 5 [15, 17] to 8 [20].

Table 2

PEDro scores for the included intervention trials

Study (year)Eligibility criteriaRandomly allocatedAllocated concealedSimilar baseline valuesBlinding of assessorsKey outcomes in 85% of included subjectsIntent to treatBetween-group statistical comparisonPoint + variability measureTotal PEDro score
Dumortier et al. [14] (2003)1001011116
Watkins et al. [15] (2003)1101000115
Gomes et al. [16] (2008)1001011116
Irving et al. [17] (2008)1101000115
Tjønna et al. [18] (2008)1101010116
Balducci et al. [20] (2010)1111110118
Stensvold et al. [21] (2010)1111010117

PEDro Physiotherapy Evidence Database

Primary Outcomes: Effect of Exercise on Cardiovascular Risk Factors Associated with the Metabolic Syndrome

Dynamic endurance training resulted in a mean reduction in WC of 3.36 cm (Fig. 2; baseline 104.62; 95% CI −4.87, −1.84; p < 0.0001; I² = 35%), a mean BP decrease of 7.11 (Fig. 3; baseline 137.97; 95% CI −9.03, −5.18; p < 0.0001; I² = 0%) mmHg/5.15 mmHg (Fig. 4; baseline 87.60; 95% CI −6.16, −4.13; p < 0.0001; I² = 28%) and a mean increase in HDL-C of 0.06 (Fig. 5; baseline 1.04; 95% CI +0.03, +0.09; p < 0.0001; I² = 38%) mmol/L. By contrast, levels of triglycerides (Fig. 6; −0.05; baseline 2.05; 95% CI −0.20, +0.09; p = 0.47; I² = 0%) mmol/L and plasma glucose (Fig. 7; −0.31; baseline 5.98; 95% CI −0.64, +0.01; p = 0.06; I² = 0%) mmol/L remained statistically unaltered.

An external file that holds a picture, illustration, etc.
Object name is 40279_2012_3_Fig2_HTML.jpg

Average net changes and corresponding 95% confidence intervals for waist circumference. Tjønna et al. [18] A and B represents moderate continuous training programme and aerobic interval training programme, respectively; and Irving et al. [17] A and B represents high-intensity training and low-intensity training, respectively. CI confidence intervals, df degrees of freedom, IV intervention, χ 2 Chi-squared

An external file that holds a picture, illustration, etc.
Object name is 40279_2012_3_Fig3_HTML.jpg

Average net changes and corresponding 95% confidence intervals for systolic blood pressure. Tjønna et al. [18] A and B represents moderate continuous training programme and aerobic interval training programme, respectively; and Irving et al. [17] A and B represents high-intensity training and low-intensity training, respectively. CI confidence intervals, df degrees of freedom, IV intervention, χ 2 Chi-squared

An external file that holds a picture, illustration, etc.
Object name is 40279_2012_3_Fig4_HTML.jpg

Average net changes and corresponding 95% confidence intervals for diastolic blood pressure. Tjønna et al. [18] A and B represents moderate continuous training programme and aerobic interval training programme, respectively; and Irving et al. [17] A and B represents high-intensity training and low-intensity training, respectively. CI confidence intervals, df degrees of freedom, IV intervention, χ 2 Chi-squared

An external file that holds a picture, illustration, etc.
Object name is 40279_2012_3_Fig5_HTML.jpg

Average net changes and corresponding 95% confidence intervals for HDL-C. Tjønna et al. [18] A and B represents moderate continuous training programme and aerobic interval training programme, respectively; and Irving et al. [17] A and B represents high-intensity training and low-intensity training, respectively. CI confidence intervals, df degrees of freedom, IV intervention, χ 2 Chi-squared

An external file that holds a picture, illustration, etc.
Object name is 40279_2012_3_Fig6_HTML.jpg

Average net changes and corresponding 95% confidence intervals for triglycerides. Tjønna et al. [18] A and B represents moderate continuous training programme and aerobic interval training programme, respectively; and Irving et al. [17] A and B represents high-intensity training and low-intensity training, respectively. CI confidence intervals, df degrees of freedom, IV intervention, χ 2 Chi-squared

An external file that holds a picture, illustration, etc.
Object name is 40279_2012_3_Fig7_HTML.jpg

Average net changes and corresponding 95% confidence intervals for plasma glucose. Tjønna et al. [18] A and B represents moderate continuous training programme and aerobic interval training programme, respectively; and Irving et al. [17] A and B represents high-intensity training and low-intensity training, respectively. CI confidence intervals, df degrees of freedom, IV intervention, χ 2 Chi-squared

To assess the effect of individual dynamic endurance studies on the summary estimate we performed a sensitivity analysis, in which the pooled estimates for the different outcomes were recalculated omitting one or two studies at a time. When we excluded the two aerobic interval training (AIT) studies, results were similar. However, when we omitted the only trial in which all patients had type 2 diabetes [17], overall changes in DBP, HDL-C and WC were no longer statistically significant (p = 0.07; p = 0.44; p = 0.40, respectively).

The results of the meta-regression showed that decreases in DBP were significantly associated with reductions in WC (r = 0.91; p = 0.001). A non-significant trend was found for the association of WC with SBP (r = 0.67; p = 0.10) and for WC with HDL-C (r = −0.68; p = 0.06). triglyceride and plasma glucose were not related to changes in WC (p > 0.20 for all).

Only one study investigated the effect of dynamic resistance training on components of the MetS [21]. They reported a significant reduction of WC (from mean ± SD 111.5 ± 10.8 to 110.1 ± 11.0 cm; p < 0.05) whereas HDL-C, triglycerides, plasma glucose, SBP and DBP remained statistically unaltered (p > 0.05 for all).

Two trials, involving 33 participants investigated the effect of combined dynamic endurance and dynamic resistance training in patients with the MetS [20, 21]. Both trials observed a significant reduction of WC. In addition, Balducci et al. [20] reported positive findings for HbA1c (from mean ± SD 7.29 ± 1.35 to 6.34 ± 0.96%; p < 0.0001) and HDL-C (from mean ± SD 44.1 ± 2.0 to 47.6 ± 1.85 mg/dL; p < 0.001) whereas this remained statistically unaltered in the study of Stensvold et al. [21]. Finally, none of the combined exercise studies found a statistically significant effect on triglycerides, SBP or DBP.

Secondary Outcomes: Effect of Dynamic Endurance Exercise on Other Cardiovascular Risk Factors

As shown in Table 3, dynamic endurance training also positively affected other blood lipids (LDL-C, total cholesterol), some anthropometric characteristics (body weight, BMI) and resulted in a significant increase of equation M19 with 19.3% compared with controls.

Table 3

Effect of dynamic endurance training on secondary outcomes

OutcomeMean baseline valueMean effect95% CISignificanceInterventions (n)Heterogeneity I² (%)Model used
Blood lipids
 LDL-C (mmol/L) 3.34−0.38−0.75, −0.01p = 0.05460RE
 TC (mmol/L) 5.55−0.53−0.64, −0.42p < 0.001523FE
Anthropometric characteristics
 Weight (kg)92.06−3.51−5.40, −1.61p < 0.0018 0FE
 BMI (kg/m²)32.18−1.65−2.81, −0.48p < 0.019 0FE
 FM (kg)38.73−1.92−6.41, +2.57NS4 0FE
Glucose/insulin parameters
 Plasma insulin (mmol/L)18.68−0.98−4.51 to +2.56NS4 0FE
Other secondary outcomes
 equation M20 (mL/kg/min)26.61+5.86+3.03, +8.69p < 0.001858RE

BMI body mass index, CI confidence interval, FE fixed effect model, FM fat mass, LDL-C low-density lipoprotein cholesterol, NS non-significant, RE random effect model, TC total cholesterol, equation M21 peak oxygen uptake

Publication Bias

Funnel plots did not show any significant publication bias for all of the primary and secondary outcomes, meaning that there was no asymmetric relationship between treatment effects and study size (data not shown).

Discussion

The main findings of this meta-analysis suggest that in patients with the MetS, dynamic endurance training is associated with favourable effects on most cardiovascular risk factors related to the MetS, that is WC, HDL-C, SBP and DBP. Our results further demonstrate that in these patients dynamic endurance training also favourably affects other important cardiovascular risk factors including LDL-C, total cholesterol, BMI and equation M22.

It is generally accepted that we need better and more affordable prevention and treatment strategies to improve wide-scale cardiovascular health outcome and to prevent the epidemic of MetS from reaching global proportions and straining public health and the economy [18]. Exercise is a key component in the treatment of patients with the MetS and in the prevention of CVD morbidity and mortality. The results of this study are in line with previous meta-analyses and extensive reviews that focused on the effect of exercise on single risk factors in populations with different cardiovascular risk factors [47].

First of all, we observed a significant improvement in measures of body composition including reduction in abdominal obesity, assessed by means of WC, body weight and BMI and a non-significant reduction in fat mass. Improvements in body composition characteristics are suggested to be associated with beneficial changes in lipids and lipoproteins through mechanisms related to insulin resistance [23, 24]. Indeed, the results of our meta-regression show a trend (p = 0.06) between changes in WC and HDL-C. In line with the meta-analysis of Kelley et al. [25], we observed significant increases in HDL-C and decreases in LDL-C and total cholesterol, without an effect on triglycerides. Wilson et al. [26] found that an HDL-C level between 41 and 46 mg/dL (1.06–1.19 mmol/L) for men resulted in a relative risk (RR) of all-cause death of 1.27 and HDL-C levels between 23 and 44 mg/dL (0.59–1.14 mmol/L) in an RR of 1.47 for women, compared with groups with values of more than 55 and 70 mg/dL, respectively. In our meta-analysis, HDL-C levels rose from 1.05 mmol/L to 1.13 mmol/L in a sex-mixed population, still showing non-optimal HDL-C plasma levels. However, it is stated in a meta-analysis of Kelley et al. [25] that a decrease of 1% in HDL-C would be associated with an increase in the risk for coronary heart disease of 2–3%. Assuming that the reverse is true, the approximate increase of 8% observed in our meta-analysis should decrease the coronary heart disease risk by 16–24%.

In addition, patients with the MetS do have type 2 diabetes or at least have an increased risk for developing it. Whereas plasma insulin levels remained statistically unaltered, we observed a trend towards lower values of mean plasma glucose following dynamic endurance exercise (p = 0.06). Earlier, Snowling and Hopkins [27] reported in patients with type 2 diabetes significant reductions in fasting glucose, HbA1c and insulin sensitivity after dynamic endurance training, with some evidence of small additional benefits resulting from combining endurance and resistance training. Moreover, it was suggested that the effects were somewhat larger for those with a more severe disease status, in particular for HbA1c. The lack of a significant effect on glucose-insulin dynamics in the current meta-analysis might be partly explained by the fact that the majority of participants were pre-diabetic but did not have diabetes. Indeed, the study of Balducci et al. [20] in patients with type 2 diabetes and the MetS did result in significant decreases in HbA1c after 12 months of exercise training, which confirms our suggestion. In addition, measuring plasma glucose and plasma insulin after an oral glucose tolerance test is often a better marker of changes in glucose tolerance after an exercise or dietary intervention. In the study of Watkins et al. [15], significant decreases in 2 h of plasma insulin were found in the dynamic endurance exercise group but not in the control group after 26 weeks, while there were no changes in fasting glucose and fasting insulin.

Following dynamic endurance training, a mean reduction in BP of 7/5 mmHg was achieved. These mean differences are statistically significant but more importantly, the magnitude of these differences is likely to be clinically relevant. Results from large, prospective intervention studies suggested that small reductions in resting SBP and DBP of 3 mmHg can reduce coronary heart disease risk by 5%, stroke by 8% and all-cause mortality by 4% [28].

Although the observed effect on each of the individual risk factors might seem small, when we combine all the effect sizes, that is a BP reduction to a mean of 131/82 mmHg, a decrease in WC to an average of 101 cm and for plasma glucose and triglycerides to a mean of 5.67 and 2.0 mmol/L, respectively and an increase of HDL-C to 1.1 mmol/L, a large number of patients would probably no longer be classified as having the MetS. Earlier, Katzmarzyk et al. [29] showed that 30.5% of patients (mean age 44.7 years) with the MetS at baseline, were no longer classified as having the MetS after a 20-week supervised dynamic endurance training programme. This reduction was mostly due to an improvement in triglycerides (43%), BP (38%) and WC (28%) whereas beneficial effects on HDL-C and plasma glucose were seen in fewer patients (16 and 9%, respectively). Similarly, Anderssen et al. [30] reported that 23.5% of 34 patients no longer had the MetS after an exercise intervention of 1 year. The only study included in this meta-analysis reporting on the number of patients with the MetS showed similar decreases with a 37.5% (from 8/8 to 5/8) and a 45% decrease (from 11/11 to 6/11) following 16 weeks of continuous and interval training, respectively [18]. Although it is difficult to quantify exactly the overall risk reduction associated with all observed changes, these results are compatible with an overall improvement of cardiovascular risk. Moreover, the results of this meta-analysis study are supported by studies that have examined the cross-sectional relationship between physical activity and the MetS. For example, in 1298 healthy police employees of the Utrecht Police Lifestyle Intervention Fitness and Training (UPLIFT) study, average physical activity intensity, average time spent at physical activity, physical activity volume and physical fitness, were each associated with reduced odds of the MetS [31]. Taken together, the results of these studies, and those from the present meta-analysis, reinforce the notion that physical activity is an important treatment option for the MetS, as stated in Katzmarzyk et al. [29].

Finally, low aerobic capacity has been shown to be a stronger predictor of CVD and mortality compared with other established risk factors [32]. The five studies that reported on the effect of dynamic endurance training on equation M23 showed a significant mean increase of 6 mL/kg/min or 19.3%. An increase of one metabolic equivalent task (=3.5 mL O2/kg/min) in exercise capacity referred to an RR for all-cause death of 0.84 [33]. Earlier, physical activity and physical fitness have been shown to be inversely associated with the clustering of metabolic abnormalities [31]. With regard to physical activity, it seems that intensity and more specifically higher intensity is the main characteristic of physical activity determining its effect on the combination of CVD risk factors [31]. However, as physical fitness exerted greater effects [31], this suggests that in daily practice physical activity should preferably increase physical fitness to maximize its effect on the prevention of the MetS. This might most likely be achieved by exercise at higher intensity [31]. Given the lack of adequate reporting of training characteristics, we were unable to investigate the effect of intensity in this meta-analysis. A considerable number of additional studies will be required to adequately discuss this aspect.

With regard to the effect of other training characteristics on cardiovascular risk factors in patients with the MetS, more research is warranted to allow for appropriate exercise prescriptions. That is, only one intervention group investigated the effect of dynamic resistance training on the clustering of cardiovascular risk factors in individuals with the MetS and found that only WC was decreased significantly [21]. In a study of Banz et al. [34], with obese subjects, the effect of dynamic resistance training also resulted in significant changes in WC, but in no other significant beneficial cardiovascular effects. Smutok et al. [35] found significant changes in glucose and insulin responses after an oral glucose tolerance test was administered after a dynamic resistance training programme in individuals with two cardiovascular risk factors, but blood lipids and lipoproteins or BP outcomes didn’t improve significantly. It seems that dynamic resistance training has potential beneficial effects on the cardiovascular risk profile, but that dynamic endurance training is more beneficial [34, 35]. In a large meta-analysis of Cornelissen et al. [36], the effect of resistance training on BP and other cardiovascular risk factors was reviewed. They found positive effects on SBP, DBP and equation M24, but no changes were seen in HDL-C, triglycerides and plasma glucose.

The effect of combined exercise on the clustering of cardiovascular risk factors in patients with MetS was discussed in two studies [20, 21]. Balducci et al. [20] concluded that combined endurance and resistance training had larger beneficial effects on upper body strength and body weight than dynamic endurance training only. No significant differences were found for the other outcome measures [20]. Stensvold et al. [21] found slightly larger benefits for the dynamic endurance training on SBP and DBP than that of combined exercise. The training responses were similar for WC, HDL-C, triglycerides and plasma [21]. However, more research is needed to investigate the potential benefits of combined training programmes.

The results of this meta-analysis should be interpreted within the context of its limitations: First, only one study fulfilling the inclusion criteria included a dynamic resistance intervention [21], and only two trials used a combined intervention of dynamic resistance training and endurance training [20, 21]. Further research about the effect of dynamic resistance training only or in combination with dynamic endurance exercise training is warranted in patients with the MetS. Second, a sensitivity analysis demonstrated that omitting the only trial in which all patients had type 2 diabetes [20], the effect of exercise on DBP, HDL-C and WC were no longer significant (p = 0.07; p = 0.44; p = 0.40, respectively). The severity of the MetS risk factors or a pathological state (type 2 diabetes) can probably influence the effects of exercise training, with more severe CV risk factors resulting in potentially larger improvements. Third, due to the small number of study groups, a subgroup analysis for sex or medication intake was not possible. More research is needed on the effect of variables that could potentially affect the outcomes of an exercise programme, especially the effect of sex and medication. Further, the mean age of participants was 52 years with a range between 46 and 64 years; therefore it may not be possible to generalize these results for younger or older individuals with the MetS. Finally, it has been suggested that one exercise dose does not fit for all components of the MetS and that recommendations should vary regarding the different risk factors [37]. Therefore, it is necessary to strike a balance for the most optimal exercise programme to improve the cluster of cardiovascular risk factors in patients with the MetS.

Conclusion

It can be concluded that exercise interventions have beneficial effects on most of the cardiovascular risk factors in patients with the MetS, but further research is needed to expand the knowledge and to strengthen the results found in this meta-analysis.

Acknowledgements

No potential conflicts of interest directly relevant to this content of this article were reported. Véronique A. Cornelissen is supported as a Postdoctoral Fellow by Research Foundation Flanders (F.W.O.). Prof. Luc Vanhees is the holder of the faculty chair ‘Lifestyle and Health’ at the University of Applied Sciences, Utrecht, the Netherlands; Nele Pattyn is a doctoral researcher at the KU Leuven, Department of Rehabilitation Sciences; Saeed R. Toghi Eshghi was a Master’s student at KU Leuven, at the time of the preparation of this article.

Footnotes

N. Pattyn and V. A. Cornelissen are joint first authors.

References

1. Grundy SM, Cleeman JI, Daniels SR, et al. Diagnosis and management of the metabolic syndrome: an American Heart Association/National Heart, Lung, and Blood Institute Scientific Statement. Circulation. 2005;112(17):2735–2752. 10.1161/CIRCULATIONAHA.105.169404. [Abstract] [CrossRef] [Google Scholar]
2. Gami AS, Witt BJ, Howard DE, et al. Metabolic syndrome and risk of incident cardiovascular events and death: a systematic review and meta-analysis of longitudinal studies. J Am Coll Cardiol. 2007;49(4):403–414. 10.1016/j.jacc.2006.09.032. [Abstract] [CrossRef] [Google Scholar]
3. Grundy SM. Metabolic syndrome pandemic. Arterioscler Thromb Vasc Biol. 2008;28(4):629–636. 10.1161/ATVBAHA.107.151092. [Abstract] [CrossRef] [Google Scholar]
4. Whelton SP, Chin A, Xin X, et al. Effect of aerobic exercise on blood pressure: a meta-analysis of randomized, controlled trials. Ann Intern Med. 2002;136(7):493–503. 10.7326/0003-4819-136-7-200204020-00006. [Abstract] [CrossRef] [Google Scholar]
5. Cornelissen VA, Fagard RH. Effects of dynamic aerobic endurance training on blood pressure, blood pressure-regulating mechanisms and cardiovascular risk factors. Hypertension. 2005;46(4):667–675. 10.1161/01.HYP.0000184225.05629.51. [Abstract] [CrossRef] [Google Scholar]
6. Kelley GA, Kelley KS. Aerobic exercise and lipids and lipoproteins in men: a meta-analysis of randomized controlled trials. J Mens Health Gend. 2006;3(1):61–70. [Europe PMC free article] [Abstract] [Google Scholar]
7. Thomas D, Elliot EJ, Naughton GA. Exercise for type 2 diabetes mellitus. Cochrane Database Syst Rev. 2006;3:CD002968. [Abstract]
8. Alberti KG, Zimmet PZ. Definition, diagnosis and classification of diabetes mellitus and its complications: part 1. Diagnosis and classification of diabetes mellitus provisional report of a WHO consultation. Diabet Med. 1998;15(7):539–553. 10.1002/(SICI)1096-9136(199807)15:7<539::AID-DIA668>3.0.CO;2-S. [Abstract] [CrossRef] [Google Scholar]
9. National Cholesterol Education Program (NCEP) Expert Panel on detection evaluation, and treatment of high blood cholesterol in adults (Adult Treatment Panel III). Third report of the National Cholesterol Education Program (NCEP) Expert Panel on detection, evaluation, and treatment of high blood cholesterol in adults (Adult Treatment Panel III) Final report. Circulation. 2002;106(25):3143–3421. [Abstract] [Google Scholar]
10. Alberti KG, Zimmet P, Shaw J. IDF Epidemiology Task Force Consensus Group. The metabolic syndrome: a new worldwide definition. Lancet. 2005;366(9491):1059–1062. 10.1016/S0140-6736(05)67402-8. [Abstract] [CrossRef] [Google Scholar]
11. Physiotherapy Evidence Database. PEDro scale (online). http://www.pedro.org.au/english/downloads/pedro-scale/. Accessed 9 Dec 2011.
12. De Morton NA. The PEDro scale is a valid measure of the methodological quality of clinical trials: a demographic study. Aust J Physiother. 2009;55(2):129–133. 10.1016/S0004-9514(09)70043-1. [Abstract] [CrossRef] [Google Scholar]
13. Maher CG, Sherrington C, Herbert RD, et al. Reliability of the PEDro scale for rating quality of randomized controlled trials. Phys Ther. 2003;83(8):713–721. [Abstract] [Google Scholar]
14. Dumortier M, Brandou F, Perez-Martin A, et al. Low intensity endurance exercise targeted for lipid oxidation improves body composition and insulin sensitivity in patients with the metabolic syndrome. Diabetes Metab. 2003;29(5):509–518. 10.1016/S1262-3636(07)70065-4. [Abstract] [CrossRef] [Google Scholar]
15. Watkins LL, Sherwood A, Feinglos M, et al. Effects of exercise and weight loss on cardiac risk factors associated with syndrome X. Arch Intern Med. 2003;163(16):1889–1895. 10.1001/archinte.163.16.1889. [Abstract] [CrossRef] [Google Scholar]
16. Gomes VA, Casella-Filho A, Chagas ACP, et al. Enhanced concentrations of relevant markers of nitric oxide formation after exercise training in patients with metabolic syndrome. Nitric Oxide. 2008;19(4):345–350. 10.1016/j.niox.2008.08.005. [Abstract] [CrossRef] [Google Scholar]
17. Irving BA, Davis CK, Brock DW, et al. Effect of exercise training intensity on abdominal visceral fat and body composition. Med Sci Sports Exerc. 2008;40(11):1863–1872. 10.1249/MSS.0b013e3181801d40. [Europe PMC free article] [Abstract] [CrossRef] [Google Scholar]
18. Tjønna AE, Lee SJ, Rognmo Ø, et al. Aerobic interval training versus continuous moderate exercise as a treatment for the metabolic syndrome: a pilot study. Circulation. 2008;118(4):346–354. 10.1161/CIRCULATIONAHA.108.772822. [Europe PMC free article] [Abstract] [CrossRef] [Google Scholar]
19. Irving BA, Weltman JY, Patrie JT, et al. Effect of exercise training intensity on nocturnal growth hormone secretion in obese adults with the metabolic syndrome. J Clin Endocrinol Metab. 2009;94(6):1979–1986. 10.1210/jc.2008-2256. [Europe PMC free article] [Abstract] [CrossRef] [Google Scholar]
20. Balducci S, Zanuso S, Nicolucci A, et al. Anti-inflammatory effect of exercise training in subjects with type 2 diabetes and the metabolic syndrome is dependent on exercise modalities and independent of weight loss. Nutr Metab Cardiovasc Dis. 2010;20(8):608–617. 10.1016/j.numecd.2009.04.015. [Abstract] [CrossRef] [Google Scholar]
21. Stensvold D, Tjønna AE, Skaug EA, et al. Strength training versus aerobic interval training to modify risk factors of metabolic syndrome. J Appl Physiol. 2010;108(4):804–810. 10.1152/japplphysiol.00996.2009. [Abstract] [CrossRef] [Google Scholar]
22. Tjønna AE, Rognmo Ø, Bye A, et al. Time course of endothelial adaptation after acute and chronic exercise in patients with metabolic syndrome. J Strength Cond Res. 2011;25(9):2552–2558. 10.1519/JSC.0b013e3181fb4809. [Abstract] [CrossRef] [Google Scholar]
23. Dattilo AM, Kris-Etherton PM. Effects of weight reduction on blood lipids and lipoproteins: a meta-analysis. Am J Clin Nutr. 1992;56(2):320–328. [Abstract] [Google Scholar]
24. Rashid S, Genest J. Effect of obesity on high-density lipoprotein metabolism. Obesity (Silver Spring) 2007;15(12):2875–2888. 10.1038/oby.2007.342. [Abstract] [CrossRef] [Google Scholar]
25. Kelley GA, Kelley KS, Tran ZV. Exercise, lipids, and lipoproteins in older adults: a meta-analysis. Prev Cardiol. 2005;8(4):206–214. 10.1111/j.0197-3118.2005.03769.x. [Europe PMC free article] [Abstract] [CrossRef] [Google Scholar]
26. Wilson PW, Abbott RD, Castelli WP. High density lipoprotein cholesterol and mortality: the Framingham Heart Study. Arteriosclerosis. 1988;8(6):737–741. 10.1161/01.ATV.8.6.737. [Abstract] [CrossRef] [Google Scholar]
27. Snowling NJ, Hopkins WG. Effects of different modes of exercise training on glucose control and risk factors for complications in type 2 diabetic patients. Diabetes Care. 2006;29(11):2518–2527. 10.2337/dc06-1317. [Abstract] [CrossRef] [Google Scholar]
28. Lewington S, Clarke R, Qizilbash N, et al. Prospective Studies Collaboration. Age-specific relevance of usual blood pressure to vascular mortality: a meta-analysis of individual data for one million adults in 61 prospective studies. Lancet. 2002;360(9349):1903–1913. 10.1016/S0140-6736(02)11911-8. [Abstract] [CrossRef] [Google Scholar]
29. Katzmarzyk PT, Leon AS, Wilmore JH, et al. Targeting the metabolic syndrome with exercise: evidence from the HERITAGE Family Study. Med Sci Sports Exerc. 2003;35(10):1703–1709. 10.1249/01.MSS.0000089337.73244.9B. [Abstract] [CrossRef] [Google Scholar]
30. Anderssen SA, Carroll S, Urdal P, et al. Combined diet and exercise intervention reverses the metabolic syndrome in middle-aged males: results from the Oslo Diet and Exercise Study. Scand J Med Sci Sports. 2007;17(6):687–695. 10.1111/j.1600-0838.2006.00631.x. [Abstract] [CrossRef] [Google Scholar]
31. Sassen B, Cornelissen VA, Kiers H, et al. Physical fitness matters more than physical activity in controlling cardiovascular disease risk factors. Eur J Cardiovasc Prev Rehabil. 2009;16(6):677–683. 10.1097/HJR.0b013e3283312e94. [Abstract] [CrossRef] [Google Scholar]
32. Myers J, Prakash M, Froelicher V, et al. Exercise capacity and mortality among men referred for exercise testing. N Engl J Med. 2002;346(11):793–801. 10.1056/NEJMoa011858. [Abstract] [CrossRef] [Google Scholar]
33. Laukkanen JA, Rauramaa R, Salonen JT, et al. The predictive value of cardiorespiratory fitness combined with coronary risk evaluation and the risk of cardiovascular and all-cause death. J Intern Med. 2007;262(2):263–272. 10.1111/j.1365-2796.2007.01807.x. [Abstract] [CrossRef] [Google Scholar]
34. Banz WJ, Maher MA, Thompson WG, et al. Effects of resistance versus aerobic training on coronary artery disease risk factors. Exp Biol Med. 2003;228(4):434–440. [Abstract] [Google Scholar]
35. Smutok MA, Reece C, Kokkinos PF, et al. Aerobic versus strength training for risk factor intervention in middle-aged men at high risk for coronary heart disease. Metabolism. 1993;42(2):177–184. 10.1016/0026-0495(93)90032-J. [Abstract] [CrossRef] [Google Scholar]
36. Cornelissen VA, Fagard RH, Coeckelberghs E, et al. Impact of resistance training on blood pressure and other cardiovascular risk factors: a meta-analysis of randomized, controlled trials. Hypertension. 2011;58(5):950–958. 10.1161/HYPERTENSIONAHA.111.177071. [Abstract] [CrossRef] [Google Scholar]
37. Vanhees L, Geladas N, Hansen D, et al., on behalf of the writing group of the EACPR. Importance of characteristics and modalities of physical activity and exercise in the management of cardiovascular health in individuals with cardiovascular risk factors: recommendations from the EACPR (part II). Eur J Cardiovasc Prev Rehabil (epub 30 Nov 2011). [Abstract]

Citations & impact 


Impact metrics

Jump to Citations

Citations of article over time

Alternative metrics

Altmetric item for https://www.altmetric.com/details/1189680
Altmetric
Discover the attention surrounding your research
https://www.altmetric.com/details/1189680

Article citations


Go to all (138) article citations