Advancing kinesiology through improved peer review

Cognitive Bias in the Peer Review Process

In a recent critique of reviewers, Ralph (2016) stated that "Peer review is prejudiced, capricious, inefficient, ineffective and generally unscientific" (p. 274). Our research proposes that one way the peer review process could appear flawed is if those involved had different beliefs about what was important in evaluating research. We found evidence for a cognitive bias where respondents to a survey asking about the importance of particular validity and reliability method practices gave different answers depending on whether they were asked to answer the survey as a researcher or as a reviewer. Because researchers have higher motivation to publish research than reviewers do to review research, we theorize that motivational differences between researchers and reviewers leads to this bias and contributes to the perception that the review process is flawed. We discuss the implications of our findings for improving the peer review process in MIS.

A slight glance at peer review

In order to ensure that the highest quality of literature is published, most journals utilize a peer review process for manuscripts submitted. Although the primary purpose for this process is to filter out ”bad science”, the process is not perfect. While there is a general consensus among researchers and clinicians that something must be done to improve upon the method for properly vetting manuscripts, there are conflicting opinions on how to best implement new policies. In this paper, we discuss the most well-supported suggestions to improve the process, with the hopes of increasing rigor and reproducibility, ensuring double-blinding, developing set guidelines, offering early training to reviewers, and giving reviewers better feedback and recognition for their work.

Citation metrics of excellence in sports biomechanics research

This study extended research on key citation metrics of winners of two career scholar awards in sports biomechanics. Google Scholar (GS) was searched using Harzing's Publish or Perish software for the 13 most recent winners of the ISBS Geoffrey Dyson Award and the ASB Jim Hay Memorial Award. Returned records were corrected for author, and publications excluded for all but peer-reviewed journal articles, proceedings articles, chapters and books in English. These recent award winners had published about 150 publications that had been cited typically 4,082 and 6,648 times over a 26- and 28-year period before receiving these career awards for sports biomechanics research. Estimated median citations at time of their awards were 2,927 and 4,907 for the Dyson and Hay awards, respectively. Award winners had mean Hirsh indexes of 32-45 and mean h_i of 19-28. Their mean g indexes (59-84) and their numerous citation classics (C > 100) indicated that they had many influential publications. The citation metrics of these scholars were outstanding and consistent with recent studies of top scholars in biomechanics and kinesiology/exercise science. Careful searching, cleaning and interpretation of several scholar-level citation metrics may provide useful confirmatory evidence for evaluations of awards committees.

The importance of peer review: thoughts on Knudson, Morrow, and Thomas (2014)

Knudson, Morrow, and Thomas ( 2014 ) recently summarized a number of important issues related to the quality of peer review and current peer-review practice in kinesiology. This writer endorses their six recommendations for improving peer review in kinesiology journals. The purpose of this commentary is to further highlight the importance of reviewer and associate editor recommendations to the editor, and the importance of editor decisions, in determining what becomes part of a discipline's body of knowledge. These recommendations and decisions ultimately affect what students learn, and what professionals practice, in their disciplines. This kind of power over the control of knowledge and the application of that knowledge is cause for all involved in science to do everything possible to improve our stewardship over peer review.