Is subjective outcome evaluation related to objective outcome evaluation? Insights from a longitudinal study in Hong Kong

Law abidance leadership education for university students in Hong Kong: Post-lecture evaluation

Law abidance is very important for effective leaders. Without law abidance, abuse of power and corruption would easily happen, which would eventually erode organizational health. To promote law abidance leadership in university students in Hong Kong, we developed a law abidance leadership program with 3 h of face-to-face lecture and 7 h of self-study of materials disturbed to students. To understand students' perception of the 3-h lecture, we conducted a post-lecture evaluation study using a 26-item measure (N = 914). Results showed that the scale possessed good reliability and validity. Confirmatory factor analyses showed that the assessment tool has three internally consistent and factorial invariant dimensions: program attributes, appreciation of law abidance, and teacher attributes. Regarding students' perception of the program, students were generally satisfied with the lecture attributes, including design, content, lecture atmosphere, teaching quality, and benefits to students. In particular, students agreed that the lecture helped them understand the importance and value of law abidance and national security; over 95% of the students indicated that they would try their best to serve as law-abiding citizens and socially responsible leaders. Multiple regression analyses showed that program attributes, appreciation of law abidance, and teacher attributes predicted overall satisfaction with the lecture. Qualitative comments of the student echoed the quantitative findings, with most of the comments being positive in nature. The present study replicated the findings reported previously. Local and international contextual factors relevant to the interpretations of the findings are highlighted.

The Effectiveness of a Leadership Subject Using a Hybrid Teaching Mode during the Pandemic: Objective Outcome and Subjective Outcome Evaluation

Positive youth development (PYD) is an innovative approach to protect students from mental health problems and promote their positive and holistic development. Although there are many studies on the beneficial effects of PYD programs on youth in high school and community contexts, it is not clear whether subjects adopting PYD principles can promote positive development for university students. Moreover, it is unclear whether such subjects are effective under COVID-19, where subjects are commonly taught via the "hybrid" mode (i.e., face-to-face plus online teaching). The present study examined students' changes in the PYD, wellbeing, and desired graduate attributes after they had taken a leadership subject utilizing PYD principles taught by the "hybrid" mode (N = 630). Adopting the one-group pre-test and post-test design (i.e., objective outcome evaluation), we found that students showed significant positive improvement in PYD indicators, wellbeing, as well as desired graduate attributes. Additionally, students had high satisfaction with the course design and teaching staff, and perceived many benefits from this subject based on the subjective outcome evaluation conducted at the end of the subject. Results also showed that students' satisfaction with the curriculum significantly and positively predicted their positive change in PYD indicators, indicating the convergence of subjective outcome evaluation and objective outcome evaluation. The results highlight the positive impacts of the hybrid mode leadership subject with PYD principles in higher education.

Who benefits and appreciates more? An evaluation of Online Service-Learning Projects in Mainland China during the COVID-19 pandemic

Previous studies have reported the feasibility and benefits of online service learning, but little is known about who benefits more from online SL and who is more satisfied. This study addressed these questions based on an evaluation of online service learning projects implemented in Xi'an and Chengdu, China, during the COVID-19 pandemic. Pretest-posttest comparison showed significant positive changes in two intended learning outcomes (i.e., positive youth development qualities and service leadership qualities) and life satisfaction before and after the service among participating students in the Xi'an project. Cross-lagged modeling based on Xi'an and Chengdu data revealed that students with better initial positive youth development qualities tended to show increased service leadership qualities and life satisfaction after the service, while initial service leadership qualities did not predict positive youth development qualities and life satisfaction after the service. However, the two-line test indicated that some of these relationships might be curvilinear. Finally, Pearson correlation analyses demonstrated that students who experienced greater changes in positive youth development qualities and service leadership qualities reported better appraisal of course qualities, teacher performance, and course effectiveness, while multiple regression analyses showed the unique effects of change in service leadership qualities (but not change in positive youth development qualities) on the perception of teacher performance and course effectiveness. Altogether, this study not only showcases the potential benefits of online SL, but also provides initial evidence suggesting the variation in (perceived) benefits by students' psychosocial competencies and learning experiences.

Evaluation of Electronic Service-Learning (e-Service-Learning) Projects in Mainland China under COVID-19

The use of electronic service-learning (e-Service-Learning or e-SL) is valuable under COVID-19 because we can provide the service without physical contact. Unfortunately, evaluation of e-SL is not widespread and there is no known study in different Chinese societies. Besides, there are many methodological limitations of the existing studies in the field. In this paper, we evaluated e-SL projects implemented in summer 2020 and 2021 in three sites in China. First, we examined service providers' changes based on pretest and posttest scores (i.e., objective outcome evaluation) and their perceptions of the e-SL projects (i.e., subjective outcome evaluation based on the service providers). Second, graduate student assessors in Chinese mainland universities and teachers of primary school students (i.e., service recipients) rated the SL program quality, service providers' performance and benefits to the service recipients after program completion (i.e., subjective outcome evaluation of SL projects based on other stakeholders). Third, trained graduate student assessors evaluated service quality during the implementation process (i.e., process evaluation). We found that university students (i.e., service providers) showed higher posttest scores in positive youth development attributes, leadership attributes and life satisfaction relative to pretest scores. Besides, service providers showed positive perceptions of their learning experience, own performance, benefits to the service recipients and themselves in the SL projects. Similarly, other stakeholders also had positive evaluation of the SL projects and related benefits. Finally, trained graduate student assessors had positive assessment of the quality of program implementation. The findings underscore the utility of e-SL involving both online teaching and learning as well as online service, particularly in a Chinese context.

Promoting Service Leadership Qualities and Well-Being among University Students through an Online Course during COVID-19 Pandemic

The COVID-19 pandemic is a great challenge to leadership education in universities. Although previous findings provide support for the effectiveness of online learning, the impact of online leadership course on students' learning outcomes and well-being has not been well documented. Using objective outcome and subjective outcome evaluation strategies, the present study examined students' perceived qualities and effectiveness of an online credit-bearing service leadership course adopting asynchronous mode (primarily online learning) and synchronous mode under COVID-19. Regardless of teaching modes, the subject yielded positive impacts. Specifically, pretest-posttest (N = 228) showed that there were positive changes in students' service leadership qualities, life satisfaction and psychological well-being. For students' perception of the course (N = 219), results indicated that most students were positive in their learning experience and satisfied with course design, lecturer quality and the benefits of the course to their development. Students' changes and subjective perceptions were positively correlated, but with a low effect size. The findings reflected that online service leadership course adopting asynchronous or synchronous mode was effective, and students were positive about their learning experience.

Evaluation of Service-Learning in Project WeCan under COVID-19 in a Chinese Context

Service-learning is a widely adopted educational pedagogy and philosophy. With the support from the Wharf (Holdings) Limited (Group), service-learning was conducted in the "Project WeCan" in Hong Kong. Prior to COVID-19, traditional service-learning was implemented with students learning in the classroom and applying their knowledge and skills to the community through providing direct face-to-face service. With the COVID-19 outbreak in the 2019-20 academic year, school lockdown measures appeared. Students had to learn online and to design and implement service offsite. As the impacts of this rapid shift in paradigm remain unknown, this study examined changes in university students using a pretest-posttest design (n = 124) and perceptions of service-learning experience via the subjective outcome evaluation design (n = 192) under COVID-19. The authors also investigated service recipients' (n = 56) satisfaction with service activities they participated in during the pandemic. Both objective outcome evaluation and subjective outcome evaluation findings revealed that service providers (university students) and recipients (secondary school students) experienced benefits from the Project. Findings support the benefits of online service-learning in "Project WeCan" even during unprecedented times such as COVID-19.

Evaluation of programs for adolescents with greater psychosocial needs: community-based Project P.A.T.H.S. in Hong Kong

This study examined the perceptions of the Tier 2 Program of the Project P.A.T.H.S. (Positive Adolescent Training through Holistic Social programmes) among Chinese secondary school students displaying greater psychosocial needs. Results showed that participants held positive views of program qualities, implementer qualities, as well as the perceived effectiveness of the program. Significant grade differences in terms of various indicators of satisfaction related to program qualities, implementer qualities, and program effectiveness were found. Correlation analyses showed that there were significant inter-relationships amongst program qualities, implementer qualities and program effectiveness. In line with the predictions, both program qualities and implementer qualities were significant predictors of perceived program effectiveness. The present study provides support for the perceived effectiveness of the Tier 2 Program of the Project P.A.T.H.S. in the community-based project context.