Clinical psychology trainees' self versus supervisor assessments of practitioner competencies

Supervisor versus self-assessment of trainee competence: Differences across developmental stages and competency domains

OBJECTIVES

This research aimed to systematically examine supervisor-trainee differences in assessments of trainee competencies across domains and developmental stages.

METHODS

Trainees and supervisors (N = 141 dyads) independently rated trainee performance at the end of placements using the Clinical Psychology Competencies Rating Scale. Based on the number of placement hours completed at the time competence was assessed, the 141 trainees were assigned to three developmental levels (61, 42, and 31 in the groups, respectively). Trajectories of 10 different competencies and trainee-supervisor differences for these competencies were examined across three developmental levels.

RESULTS

Compared to their supervisor ratings, trainees underestimated their competence during early stages of training, with this discrepancy reducing at Level 2 and reversing into an overestimation at Level 3. Compared to their own ratings for overall competence, trainees rated Relational and Communication, Reflective Practice, and Professionalism domains as relative strengths, and rated their competence on assessment and intervention domains as relative weaknesses.

CONCLUSION

Growth trajectories derived from supervisor assessments were much flatter than trajectories derived from trainee assessments. As predicted by the impostor theory of practitioner development, trainees significantly underestimated their competence early in training. The trend for trainees to overestimate their competence toward the end of their training is a potential concern that warrants further research.

Reliability and validity of the Self-Reflection and Insight Scale for psychologists and the development and validation of the revised short version

Self-reflection is broadly considered a core competency for psychologists; however, there is an absence of measures of self-reflection, limiting the extent to which self-reflection can be assessed in both research and practice contexts. Whilst the Self-Reflection and Insight Scale (Grant et al., 2002) has been validated in a range of formats with different populations, it has not yet been validated with psychologists. Further, the psychometric properties of a short version of the scale (Silvia, 2021) have not been examined for use with psychologists. This study tested the factor structure, internal consistency and convergent and divergent validity of the Self-Reflection and Insight Scale with registered psychologists (N = 123), finding both the full scale and short version to have sound psychometrics. However, as there were low loading items across both versions of the measure, and the short version also excluded high-loading items, the SRIS-Revised (SRIS-R) was formed through model improvement, retaining a total of 14 items. This revised version of the scale captures high loading items without redundancy of low-loading items, resulting in a measure that parsimoniously captures the construct of self-reflection as relevant to psychologists. The SRIS-R demonstrated good internal consistency (α = .882), convergent, divergent and construct validity. Scores on the SRIS-R were used to test whether there was a correlation between self-reflection and years of professional registration, with this not being significant.

Examining Skills and Abilities During the Pandemic - Psychology Students' and Examiners' Perceptions of a Digital OSCE

Finding valid and reliable ways to assess complex clinical skills within psychology is a challenge. Recently, there have been some examples of applying Objective Structured Clinical Examinations (OSCEs) in psychology for making such assessments. The aim of this study was to examine students' and examiners' perceptions of a digital OSCE in psychology regarding quality and students' feelings about the OSCE. Participants were 51 students enrolled in the Programme for Master of Science in Clinical Psychology during two semesters and nine examiners assessing each OSCE occasion, at Umeå University, Sweden. Web-based questionnaires were used for data collection. Psychometric analyses indicated that the subscales in the student questionnaire had adequate or close to adequate levels of item and scale reliability. Both students and examiners felt that the digital OSCE was realistic, valid and well-aligned with professional practice. Although students perceived the digital OSCE as stressful, the results showed that they were focused and concentrated and found the OSCE to be a positive learning experience, implying that the stress did not affect performance to any significant extent. Based on the examiners' experiences, it can be concluded that there are both advantages and disadvantages which need to be considered when planning future digital OSCEs.

Introducing the QACP: development and preliminary validation of an instrument to measure psychotherapist's core competencies

The movement towards the conceptualization, description and evaluation of psychotherapists' competencies has been widely developed in the last years and has relevant implications for psychotherapy, training, and continuous education. In Italy, this movement has been supported by the Committee for Psychotherapists' Competencies established in 2010 by FIAP (Italian Federation of Psychotherapy's Associations) and CNSP (National Association of Psychotherapy's Training Institutes) and has involved more than 1000 psychotherapists from different approaches, by means of conferences, expert meetings, workshops, and focus groups. One of the outcomes of this process has been the development of a new self-assessment tool for core competencies (i.e., those that are shared by therapists from all modalities): the QACP (Questionario per l'Autovalutazione delle Competenze dello Psicoterapeuta). The present study aims to present the process of development and the preliminary proofs of the validity of this tool. Construct and knowngroup validity of the questionnaire were examined, and reliability was estimated by computing the internal consistency reliability coefficients for both the overall and the subscale scores. Overall, the instrument showed satisfactory psychometric characteristics. The limitations of the study and the results are discussed and directions for further research are proposed.

Self-Coding of Fidelity as a Potential Active Ingredient of Consultation to Improve Clinicians' Fidelity

A key goal for implementation science is the identification of evidence-based consultation protocols and the active ingredients within these protocols that drive clinician behavior change. The current study examined clinicians' self-coding of fidelity as a potential active ingredient of consultation for the Attachment and Biobehavioral Catch-up (ABC) intervention. It also examined two other potential predictors of clinician fidelity in response to consultation: dosage of consultation and working alliance. Twenty-nine clinicians (97% female, 62% White, M age = 34 years) participated in a year of weekly fidelity-focused ABC consultation sessions, for which clinicians self-coded fidelity and received consultant feedback on both their coding and their fidelity. Data from the ABC fidelity measure were available for 1067 sessions coded by consultants, and clinicians' self-coding accuracy was calculated from 1044 sessions coded by both clinicians and consultants. Alliance was measured with the Working Alliance Inventory-Trainee and Supervisor Versions. The study was observational, and fidelity and self-coding accuracy were modeled across time using hierarchical linear modeling. Clinicians' ABC fidelity, as well as their self-coding accuracy, increased over the course of consultation. Clinicians' self-coding accuracy predicted their initial fidelity and growth in fidelity. Working alliance was also linked to fidelity and self-coding accuracy. These results suggest that clinician self-coding should be further examined as an active ingredient of consultation. The study has important implications for the design of consultation procedures and fidelity assessments.

Meta-analysis of the impact of cross-cultural training on adjustment, cultural intelligence, and job performance

Purpose

This research aims to provide an updated meta-analysis of cross-cultural training effectiveness through quantitatively reviewing the links between cross-cultural training and three outcomes: adjustment, cultural intelligence and job performance.

Design/methodology/approach

Drawing from 37 independent samples from 1991 through August 2021 involving a total of 7,040 trainees, the author adopt random-effects meta-analyses method to assess the relationships between cross-cultural training and three outcomes and examine potential moderators (i.e. training phase, training method and rating source) that influence these relationships.

Findings

The meta-analytical results mainly support our hypotheses. Cross-cultural training -adjustment exhibits a medium and significant positive effect, whereas cross-cultural training is positive significantly related to job performance and cultural intelligence of small effect sizes. Training method and data source significantly moderate the cross-cultural training's impact on cultural intelligence and performance correspondingly.

Originality/value

This study considers longitudinal statistics for measuring cross-cultural training effectiveness, breaking down the inadequacy of cross-sectional measurement and covers sub-dimensions of cross-cultural training outcomes to provide more comprehensive and advanced evidence of cross-cultural training effectiveness for the latest 30 years.