Statistics, truth, and error reduction in sport and exercise sciences

The dose-response relationship between interval-training and VO2max in well-trained endurance runners: A systematic review

Success in endurance running is primarily determined by maximal aerobic power (VO_2max), fractional utilization, and running economy (RE). Within the literature, two training modalities have been identified to improve VO_2max; continuous training (CT) and interval-training (IT). The efficacy of IT to improve VO_2max in well-trained runners remains equivocal, as does whether a dose-response relationship exists between the IT training load performed and changes in VO_2max. A keyword search was performed in five electronic databases. Seven studies met the inclusion criteria for this systematic review. The training impulse (TRIMP) was calculated to analyse relationships between training load and changes in VO_2max, by calculating the time accumulated in certain intensity domains throughout a training intervention. Non-significant (P>0.05) improvements in VO_2max were reported in six studies, with only one study reporting a significant (P<0.05) improvement following the IT interventions. A relationship between the training session impulse of the interval-training performed (IT S_TRIMP) and VO_2max improvements were observed. The efficacy of IT to improve VO_2max in well-trained runners remains equivocal, nevertheless, the novel method of training-load analysis demonstrates a relationship between the IT S_TRIMP and VO_2max improvements. This provides practical application for the periodization of IT within the training regime of well-trained distance runners.

Systematic review of the use of "magnitude-based inference" in sports science and medicine

Magnitude-based inference (MBI) is a controversial statistical method that has been used in hundreds of papers in sports science despite criticism from statisticians. To better understand how this method has been applied in practice, we systematically reviewed 232 papers that used MBI. We extracted data on study design, sample size, and choice of MBI settings and parameters. Median sample size was 10 per group (interquartile range, IQR: 8-15) for multi-group studies and 14 (IQR: 10-24) for single-group studies; few studies reported a priori sample size calculations (15%). Authors predominantly applied MBI's default settings and chose "mechanistic/non-clinical" rather than "clinical" MBI even when testing clinical interventions (only 16 studies out of 232 used clinical MBI). Using these data, we can estimate the Type I error rates for the typical MBI study. Authors frequently made dichotomous claims about effects based on the MBI criterion of a "likely" effect and sometimes based on the MBI criterion of a "possible" effect. When the sample size is n = 8 to 15 per group, these inferences have Type I error rates of 12%-22% and 22%-45%, respectively. High Type I error rates were compounded by multiple testing: Authors reported results from a median of 30 tests related to outcomes; and few studies specified a primary outcome (14%). We conclude that MBI has promoted small studies, promulgated a "black box" approach to statistics, and led to numerous papers where the conclusions are not supported by the data. Amidst debates over the role of p-values and significance testing in science, MBI also provides an important natural experiment: we find no evidence that moving researchers away from p-values or null hypothesis significance testing makes them less prone to dichotomization or over-interpretation of findings.

Tests examining skill outcomes in sport: a systematic review of measurement properties and feasibility

BACKGROUND

A high level of participant skill is influential in determining the outcome of many sports. Thus, tests assessing skill outcomes in sport are commonly used by coaches and researchers to estimate an athlete's ability level, to evaluate the effectiveness of interventions or for the purpose of talent identification.

OBJECTIVE

The objective of this systematic review was to examine the methodological quality, measurement properties and feasibility characteristics of sporting skill outcome tests reported in the peer-reviewed literature.

DATA SOURCES

A search of both SPORTDiscus and MEDLINE databases was undertaken.

STUDY SELECTION

Studies that examined tests of sporting skill outcomes were reviewed. Only studies that investigated measurement properties of the test (reliability or validity) were included. A total of 22 studies met the inclusion/exclusion criteria.

STUDY APPRAISAL AND SYNTHESIS METHODS

A customised checklist of assessment criteria, based on previous research, was utilised for the purpose of this review.

RESULTS

A range of sports were the subject of the 22 studies included in this review, with considerations relating to methodological quality being generally well addressed by authors. A range of methods and statistical procedures were used by researchers to determine the measurement properties of their skill outcome tests. The majority (95%) of the reviewed studies investigated test-retest reliability, and where relevant, inter and intra-rater reliability was also determined. Content validity was examined in 68% of the studies, with most tests investigating multiple skill domains relevant to the sport. Only 18% of studies assessed all three reviewed forms of validity (content, construct and criterion), with just 14% investigating the predictive validity of the test. Test responsiveness was reported in only 9% of studies, whilst feasibility received varying levels of attention.

LIMITATIONS

In organised sport, further tests may exist which have not been investigated in this review. This could be due to such tests firstly not being published in the peer-review literature and secondly, not having their measurement properties (i.e., reliability or validity) examined formally.

CONCLUSIONS

Of the 22 studies included in this review, items relating to test methodological quality were, on the whole, well addressed. Test-retest reliability was determined in all but one of the reviewed studies, whilst most studies investigated at least two aspects of validity (i.e., content, construct or criterion-related validity). Few studies examined predictive validity or responsiveness. While feasibility was addressed in over half of the studies, practicality and test limitations were rarely addressed. Consideration of study quality, measurement properties and feasibility components assessed in this review can assist future researchers when developing or modifying tests of sporting skill outcomes.

Twenty-five years of sport performance research in the Journal of Sports Sciences

In this historical review covering the past 25 years, we reflect on the content of manuscripts relevant to the Sport Performance section of the Journal of Sports Sciences. Due to the wide diversity of sport performance research, the remit of the Sport Performance section has been broad and includes mathematical and statistical evaluation of competitive sports performances, match- and notation-analysis, talent identification, training and selection or team organization. In addition, due to the academic interests of its section editors, they adopted a quality-assurance role for the Sport Performance section, invariably communicated through key editorials that subsequently shaped the editorial policy of the Journal. Key high-impact manuscripts are discussed, providing readers with some insight into what might lead an article to become a citation "classic". Finally, landmark articles in the areas of "science and football" and "notation analysis" are highlighted, providing further insight into how such articles have contributed to the development of sport performance research in general and the Journal of Sports Sciences in particular.