A case against default effect sizes in sport and exercise science

Meta-analysis of variation in sport and exercise science: Examples of application within resistance training research

Meta-analysis has become commonplace within sport and exercise science for synthesising and summarising empirical studies. However, most research in the field focuses upon mean effects, particularly the effects of interventions to improve outcomes such as fitness or performance. It is thought that individual responses to interventions vary considerably. Hence, interest has increased in exploring precision or personalised exercise approaches. Not only is the mean often affected by interventions, but variation may also be impacted. Exploration of variation in studies such as randomised controlled trials (RCTs) can yield insight into interindividual heterogeneity in response to interventions and help determine generalisability of effects. Yet, larger samples sizes than those used for typical mean effects are required when probing variation. Thus, in a field with small samples such as sport and exercise science, exploration of variation through a meta-analytic framework is appealing. Despite the value of embracing and exploring variation alongside mean effects in sport and exercise science, it is rarely applied to research synthesis through meta-analysis. We introduce and evaluate different effect size calculations along with models for meta-analysis of variation using relatable examples from resistance training RCTs.

The Effects of Strength and Conditioning Interventions on Sprinting Performance in Team Sport Athletes: A Systematic Review and Meta-Analysis

Murphy, A, Burgess, K, Hall, AJ, Aspe, RR, and Swinton, PA. The effects of strength and conditioning interventions on sprinting performance in team sport athletes: a systematic review and meta-analysis. J Strength Cond Res 37(8): 1692–1702, 2023—Linear sprinting is a key determinant of athletic performance within team sports. The aims of the review were to quantify and compare the effectiveness of popular strength and conditioning (S&C) training modes to improve sprint performance in team sport athletes, with additional focus on potential moderators and the relationships between improvements in physical factors (e.g., strength, power, and jump performance) and improvements in sprint performance. Inclusion was restricted to resistance, plyometric, sprint, and combined training interventions comprising team sport athletes. Multilevel, Bayesian’s meta-analysis and meta-regression models conducted with standardized mean difference effect sizes were used to investigate training modes and potential moderators. Weighted regression models conducted on shrunken estimates from initial Bayesian’s meta-analyses were used to quantify relationships between improvements in physical factors and sprint performance. Certainty of evidence was assessed using the grading of recommendations assessment development and evaluation (GRADE) approach. Similar improvements in sprint performance were obtained across training modes, with some evidence of the largest effects with resistance training ( = 0.55 [95% credible interval [CrI]: 0.36–0.78; very low certainty]). A strong moderating effect of training intensity was identified across all training modes with evidence of greater improvements in sprint performance with high-intensity training ( = 0.17 [95% CrI: 0.01–0.33; very low certainty]). Strong positive relationships were identified between improvements in all physical factors and sprint performance ( = 0.56 [95% CrI: 0.36–0.77; low certainty], = 0.80 [95% CrI: 0.50–1.0; low certainty], = 0.78 [95% CrI: 0.57–0.97; low certainty]). The findings indicate that focus on developing speed in team sport athletes should be placed on S&C training with high intensities, including the use of resisted sprint training.

The Effects of Sprint Interval Training on Physical Performance: A Systematic Review and Meta-Analysis

ABSTRACT

Hall, AJ, Aspe, RR, Craig, TP, Kavaliauskas, M, Babraj, J, and Swinton, PA. The effects of sprint interval training on physical performance: a systematic review and meta-analysis. J Strength Cond Res 37(2): 457-481, 2023-The present study aimed to synthesize findings from published research and through meta-analysis quantify the effect of sprint interval training (SIT) and potential moderators on physical performance outcomes (categorized as aerobic, anaerobic, mixed aerobic-anaerobic, or muscular force) with healthy adults, in addition to assessing the methodological quality of included studies and the existence of small study effects. Fifty-five studies were included (50% moderate methodological quality, 42% low methodological quality), with 58% comprising an intervention duration of ≤4 weeks and an array of different training protocols. Bayesian's meta-analysis of standardized mean differences (SMD) identified a medium effect of improved physical performance with SIT (ES 0.5 = 0.52; 95% credible intervals [CrI]: 0.42-0.62). Moderator analyses identified overlap between outcome types with the largest effects estimated for anaerobic outcomes (ES 0.5 = 0.61; 95% CrI: 0.48-0.75). Moderator effects were identified for intervention duration, sprint length, and number of sprints performed per session, with larger effects obtained for greater values of each moderator. A substantive number of very large effect sizes (41 SMDs > 2) were identified with additional evidence of extensive small study effects. This meta-analysis demonstrates that short-term SIT interventions are effective for developing moderate improvements in physical performance outcomes. However, extensive small study effects, likely influenced by researchers analyzing many outcomes, suggest potential overestimation of reported effects. Future research should analyze fewer a priori selected outcomes and investigate models to progress SIT interventions for longer-term performance improvements.

With Great Power Comes Great Responsibility: Common Errors in Meta-Analyses and Meta-Regressions in Strength & Conditioning Research

BACKGROUND AND OBJECTIVE

Meta-analysis and meta-regression are often highly cited and may influence practice. Unfortunately, statistical errors in meta-analyses are widespread and can lead to flawed conclusions. The purpose of this article was to review common statistical errors in meta-analyses and to document their frequency in highly cited meta-analyses from strength and conditioning research.

METHODS

We identified five errors in one highly cited meta-regression from strength and conditioning research: implausible outliers; overestimated effect sizes that arise from confusing standard deviation with standard error; failure to account for correlated observations; failure to account for within-study variance; and a focus on within-group rather than between-group results. We then quantified the frequency of these errors in 20 of the most highly cited meta-analyses in the field of strength and conditioning research from the past 20 years.

RESULTS

We found that 85% of the 20 most highly cited meta-analyses in strength and conditioning research contained statistical errors. Almost half (45%) contained at least one effect size that was mistakenly calculated using standard error rather than standard deviation. In several cases, this resulted in obviously wrong effect sizes, for example, effect sizes of 11 or 14 standard deviations. Additionally, 45% failed to account for correlated observations despite including numerous effect sizes from the same study and often from the same group within the same study.

CONCLUSIONS

Statistical errors in meta-analysis and meta-regression are common in strength and conditioning research. We highlight five errors that authors, editors, and readers should check for when preparing or critically reviewing meta-analyses.

Durability is improved by both low and high intensity endurance training

Introduction: This is one of the first intervention studies to examine how low- (LIT) and high-intensity endurance training (HIT) affect durability, defined as 'time of onset and magnitude of deterioration in physiological-profiling characteristics over time during prolonged exercise'. Methods: Sedentary and recreationally active men (n = 16) and women (n = 19) completed either LIT (average weekly training time 6.8 ± 0.7 h) or HIT (1.6 ± 0.2 h) cycling for 10 weeks. Durability was analyzed before and after the training period from three factors during 3-h cycling at 48% of pretraining maximal oxygen uptake (VO_2max): 1) by the magnitude and 2) onset of drifts (i.e. gradual change in energy expenditure, heart rate, rate of perceived exertion, ventilation, left ventricular ejection time, and stroke volume), 3) by the 'physiological strain', defined to be the absolute responses of heart rate and its variability, lactate, and rate of perceived exertion. Results: When all three factors were averaged the durability was improved similarly (time x group p = 0.42) in both groups (LIT: p = 0.03, g = 0.49; HIT: p = 0.01, g = 0.62). In the LIT group, magnitude of average of drifts and their onset did not reach statistically significance level of p < 0.05 (magnitude: 7.7 ± 6.8% vs. 6.3 ± 6.0%, p = 0.09, g = 0.27; onset: 106 ± 57 min vs. 131 ± 59 min, p = 0.08, g = 0.58), while averaged physiological strain improved (p = 0.01, g = 0.60). In HIT, both magnitude and onset decreased (magnitude: 8.8 ± 7.9% vs. 5.4 ± 6.7%, p = 0.03, g = 0.49; onset: 108 ± 54 min vs. 137 ± 57 min, p = 0.03, g = 0.61), and physiological strain improved (p = 0.005, g = 0.78). VO_2max increased only after HIT (time x group p < 0.001, g = 1.51). Conclusion: Durability improved similarly by both LIT and HIT based on reduced physiological drifts, their postponed onsets, and changes in physiological strain. Despite durability enhanced among untrained people, a 10-week intervention did not alter drifts and their onsets in a large amount, even though it attenuated physiological strain.

Long-term outcomes of cognitive behavioural therapy for social anxiety disorder: A meta-analysis of randomised controlled trials

Cognitive Behavioural Therapy (CBT) is effective in treating Social Anxiety Disorder (SAD). However, less is known about whether gains achieved in disorder-specific and secondary outcomes (e.g., depression, general anxiety, quality of life, and self-esteem) are maintained 12 months or longer. A systematic literature search yielded 25 relevant studies that administered CBT to participants with SAD. Multivariate meta-analyses of post-treatment assessments, found that CBT was superior to control conditions in reducing social anxiety (g =.74), depression (g =.52), general anxiety (g =.69) and improving quality of life (g =.39). The within-groups effect sizes revealed that 12 months or more after CBT treatment, symptoms continued to improve for social anxiety (g_av =.23) and quality of life (g_av =.17), and gains were maintained for depressive (g_av =.06) and general anxiety symptoms (g_av =.03). However, meta-analyses of long-term outcomes lack comparison groups. Moderation was non-significant for the treatment model, format, number of sessions, treatment duration, or inclusion of booster sessions. Future research may investigate what drives improvement after treatment cessation and how CBT affects other co-occurring symptomatology. Taken together, CBT produces many benefits for SAD and individuals with co-occurring symptoms can benefit from extant SAD-focused treatments.

Interpreting magnitude of change in strength and conditioning: Effect size selection, threshold values and Bayesian updating

The magnitude of change following strength and conditioning (S&C) training can be evaluated comparing effect sizes to thresholds. This study conducted a series of meta-analyses and compiled results to identify thresholds specific to S&C, and create prior distributions for Bayesian updating. Pre- and post-training data from S&C interventions were translated into standardised mean difference (SMD_pre) and percentage improvement (%Improve) effect sizes. Bayesian hierarchical meta-analysis models were conducted to compare effect sizes, develop prior distributions, and estimate 0.25-, 0.5-, and 0.75-quantiles to determine small, medium, and large thresholds, respectively. Data from 643 studies comprising 6574 effect sizes were included in the analyses. Large differences in distributions for both SMD_pre and %Improve were identified across outcome domains (strength, power, jump and sprint performance), with analyses of the tails of the distributions indicating potential large overestimations of SMD_pre values. Future evaluations of S&C training will be improved using Bayesian approaches featuring the information and priors developed in this study. To facilitate an uptake of Bayesian methods within S&C, an easily accessible tool employing intuitive Bayesian updating was created. It is recommended that the tool and specific thresholds be used instead of isolated effect size calculations and Cohen's generic values when evaluating S&C training.

Myths and Methodologies: The use of equivalence and non-inferiority tests for interventional studies in exercise physiology and sport science

NEW FINDINGS

What is the topic of this review? The traditional null-hypothesis test allows making informed decisions about whether an experimental intervention is superior to a control while controlling the Type I and Type II error rates. Equivalence and non-inferiority tests allow determining whether two interventions are similar in efficacy or whether one is not unacceptably worse than the other. What advances does it highlight? Equivalence and non-inferiority designs may better fit the research questions of exercise physiologists and sport scientists when investigating new interventions that have better cost-effectiveness, are safer and easier to implement, or are less demanding than the standard.

ABSTRACT

Exercise physiology and sport science have traditionally made use of the null hypothesis of no difference to make decisions about experimental interventions. This article aims to review current statistical approaches typically used by exercise physiologists and sport scientists for the design and analysis of experimental interventions and to highlight the importance of including equivalence and non-inferiority studies, which address different research questions than deciding whether an effect is present. Firstly, we briefly describe the most common approaches, along with their rationale, to investigate the effects of different interventions. We then discuss the main steps involved in the design and analysis of equivalence and non-inferiority studies, commonly performed in other research fields, with worked examples from exercise physiology and sport science scenarios. Finally, we provide recommendations to exercise physiologists and sport scientists who would like to apply the different approaches in future research. We hope this work will promote the correct use of equivalence and non-inferiority designs in exercise physiology and sport science whenever the research context, conditions, applications, researchers' interests, or reasonable beliefs, justify these approaches. This article is protected by copyright. All rights reserved.

Earlier Boston Marathon Start Time Mitigates Environmental Heat Stress

PURPOSE

This study aimed to compare the wet bulb globe temperature (WBGT) index and other environmental parameters between early and late Boston Marathon race start times from 1995 to 2016.

METHODS

Environmental data from 1995 to 2016 (excluding 1996) were used to compare two identical time frames using the 0900-1300 h start versus the 1100-1500 h start. This included the WBGT, dry bulb (Tdb), black globe (Tbg), wet bulb (Twb), solar radiation, relative humidity, and air water vapor pressure. To make comparisons between start times, the difference in the area under the curve (AUC) for each environmental variable was compared within each year with a Wilcoxon signed rank test with a Holm-Bonferroni correction.

RESULTS

AUC exposures for WBGT (P = 0.027), Twb (P = 0.031), Tdb (P = 0.027), Tbg (P = 0.055), and solar radiation (P = 0.004) were reduced with an earlier start, whereas those for relative humidity and air water vapor pressure were not. Overall, an earlier race start time by 2 h (0900 vs 1100 h) reduced the odds of experiencing a higher flag category 1.42 times (β = 0.1744, P = 0.032).

CONCLUSIONS

The 2007 decision to make the Boston Marathon start time earlier by 2 h has reduced by ~1.4 times the odds that runners will be exposed to environmental conditions associated with exertional heat illness.

Rest-pause and drop-set training elicit similar strength and hypertrophy adaptations compared to traditional sets in resistance-trained males

The present paper aimed to compare the effect of drop-set (DS) and rest-pause (RP) systems versus traditional resistance training (TRT) with equalized total training volume on maximum dynamic strength (1RM) and thigh muscle thickness (MT).Twenty-eight resistance-trained males were randomly assigned to either RP (n = 10), DS (n = 9) or TRT (n = 9) protocols performed twice a week for 8 weeks. 1RM and MT of the proximal, middle and distal portions of the lateral thigh were assessed at baseline and post intervention.A significant time x group interaction was observed for 1RM (P = 0.025) in the barbell back squat after 8-weeks. Post hoc comparisons revealed that RP promoted higher 1RM than TRT (P = 0.001); no statistical differences in strength were observed between the other conditions. A significant main effect of time was revealed for MT at the proximal (P = 0.0001) and middle (P = 0.0001) aspects of the lateral thigh for all training groups; however, the distal portion did not show a time effect (P = 0.190). There were no between-group interactions for MT. Our findings suggest that RP promotes slightly superior strength-related improvements compared to TRT, but hypertrophic adaptations are similar between conditions. Novelty bullets • Rest-pause elicited a slightly superior benefit for strength adaptations compared to traditional resistance training. • Resistance training systems do not promote superior hypertrophic adaptations when total training volume is equalized. • Muscle thickness in distal portion of thigh are similar to baseline. Although modest, effect sizes tended to favor rest-pause.

Sprint Interval Exercise Performance in Vegans

OBJECTIVE

The impact of a vegan diet on sprint interval exercise performance is unknown. Thus, the purpose of the present study was to compare performance during a sprint interval exercise between omnivores and vegans.

METHODS

Nine healthy omnivores (4 men and 5 women) and nine healthy vegans (4 men and 5 women), with similar levels of daily physical activity, performed four bouts (5-min rest between bouts) of a 30-s all-out sprint exercise on a cycle ergometer. Peak power, mean power, fatigue index, and time to reach maximal power output in each bout were recorded.

RESULTS

There was a higher peak power in bouts 1 and 2 compared with bouts 3 and 4 (p < 0.02), and a higher mean power in bout 1 compared with bouts 2, 3 and 4 (p < 0.02). However, for all bouts, there were no significant difference between omnivores and vegans in peak power (7.60 ± 1.55 vs. 8.16 ± 1.27, 7.52 ± 1.6 vs 7.61 ± 0.73, 7.00 ± 1.44 vs. 7.00 ± 1.05 and 6.95 ± 1.42 vs. 6.49 ± 0.90 W.kg^-1, all p > 0.05) and in mean power (5.35 ± 0.93 vs. 5.69 ± 0.84, 5.10 ± 0.88 vs. 5.21 ± 0.49, 4.79 ± 0.81 vs. 4.79 ± 0.45 and 4.81 ± 0.81 vs. 4.69 ± 0.47 W.kg^-1, all p > 0.05). Fatigue index and time to reach maximal power output were not affected by diet or bouts (all p > 0.05).

CONCLUSIONS

These findings indicate that a vegan diet does not compromise sprint interval exercise performance.