Capturing the Four-Phase Team Adaptation Process With Behaviorally Anchored Rating Scales (BARS)

Abstract. As a response to the lack of quantitative and reliable measures of the team adaptation process, the aim of the present study was to develop and validate an instrument for assessing the four phases of the team adaptation process as described by Rosen and colleagues (2011) . Two trained raters and two subject matter expert groups contributed to the development of four behaviorally anchored rating scales (BARS) that span across the spectrum of team processes involved in each team adaptation phase. To validate the four BARS, two different trained raters assessed independently the team adaptation phases of 66 four-person teams. The validation study provided empirical support for the BARS’ psychometric adequacy. The BARS measures overcame the common middle anchor problem, showed sensitivity in differentiating between teams and between the four phases, showed evidence for acceptable reliability, construct, and criterion validity, and supported the theoretical team adaptation process assumptions. The study contributes to research and praxis by enabling the direct assessment of the overall team adaptation process, thereby facilitating our understanding of this complex phenomenon. This allows the identification of behavioral strengths and weaknesses for targeted team development and comprehensive team adaptation studies.

Evaluating non-technical skills and mission essential competencies of pilots in military aviation environments

To develop and validate a classification of non-technical skills (NTS) in military aviation, a study was conducted, using data from real operations of F16 aircraft formations. Phase 1 developed a NTS classification based on the literature review (e.g. NOTECHS) and a workshop with pilots. The Non-TEChnical-MILitary-Skills (NOTEMILS) scheme was tested in Phase 2 in a series of Principal Component Analysis with data from After-Action-Review sessions (i.e. 900 records from a wide range of operations). The NTS were found to make a good prediction of Mission Essential Components (R² > 0.80) above the effect of experience. Phase 3 undertook a reliability analysis where three raters assessed the NOTEMILS scheme with good results (i.e. all r_wg > 0.80). To look into the consistency of classifications, another test indicated that, at least, two out of three raters were in agreement in over 70% of the assessed flight segments. Practitioner Summary: A classification scheme of Non-Technical Skills (NTS) was developed and tested for reliability in military aviation operations. The NTS scheme is a valuable tool for assessing individual and team skills of F-16 pilots in combat. It is noteworthy that the tool had a good capability of predicting Mission Essential Competencies.

Team Performance: Evidence for Validity of a Measure

Abstract This study aimed to obtain validity evidences of a teamwork performance scale. Team performance was understood as a meso-level characteristic, resulting of an emerging process. Due to that understanding the proposed instrument should take that aspect into consideration. The empirical data were collected from 276 Ecuadorian teachers organized in 70 educational teams. They answered nine sentences of the scale. Results of exploratory factor analysis showed a unifactorial solution explaining 65.84% of the variance. The measure also has adequate values of reliability (Cronbach Alpha = .93). In addition to these analyses, patterns of variance within and between the groups were verified. The results showed that the variance at the individual level was small when answers of team members were analyzed and was significant when teams were compared. We consider it is important that additional studies be performed in order to identify stability of the factor solution.