Presidential address: Preparing for permanent test centers and computerized adaptive testing

The role of web-based flipped learning in EFL learners’ critical thinking and learner engagement

The flipped learning approach with the use of social media as an emerging technology has changed the quality of learning in English as a Foreign Language (EFL) educational contexts. This review probed the effect of the web-based flipped learning approach on learners’ engagement and critical thinking. The earlier studies revealed the significance of social media in developing learner engagement and critical thinking. Studies indicated that the provision of opportunities for more cooperative and collaborative learning activities, and high-quality interaction through the use of social media can be influential in developing learners’ engagement. Moreover, social media platforms can provide a context for feedback, and various types of challenging tasks that can improve EFL learners’ critical thinking. However, this review implicated that social media in flipped learning approach may be beneficial for instructors, learners, teacher educators, curriculum designers, educational policy-makers, and advisors to be aware of this valuable learner-centered approach.

Introduction to the LIVECAT web-based computerized adaptive testing platform

This study introduces LIVECAT, a web-based computerized adaptive testing platform. This platform provides many functions, including writing item content, managing an item bank, creating and administering a test, reporting test results, and providing information about a test and examinees. The LIVECAT provides examination administrators with an easy and flexible environment for composing and managing examinations. It is available at http://www.thecatkorea.com/. Several tools were used to program LIVECAT, as follows: operating system, Amazon Linux; web server, nginx 1.18; WAS, Apache Tomcat 8.5; database, Amazon RDMS-Maria DB; and languages, JAVA8, HTML5/CSS, Javascript, and jQuery. The LIVECAT platform can be used to implement several item response theory (IRT) models such as the Rasch and 1-, 2-, 3-parameter logistic models. The administrator can choose a specific model of test construction in LIVECAT. Multimedia data such as images, audio files, and movies can be uploaded to items in LIVECAT. Two scoring methods (maximum likelihood estimation and expected a posteriori) are available in LIVECAT and the maximum Fisher information item selection method is applied to every IRT model in LIVECAT. The LIVECAT platform showed equal or better performance compared with a conventional test platform. The LIVECAT platform enables users without psychometric expertise to easily implement and perform computerized adaptive testing at their institutions. The most recent LIVECAT version only provides a dichotomous item response model and the basic components of CAT. Shortly, LIVECAT will include advanced functions, such as polytomous item response models, weighted likelihood estimation method, and content balancing method.

Post-hoc simulation study of computerized adaptive testing for the Korean Medical Licensing Examination

PURPOSE

Computerized adaptive testing (CAT) has been adopted in licensing examinations because it improves the efficiency and accuracy of the tests, as shown in many studies. This simulation study investigated CAT scoring and item selection methods for the Korean Medical Licensing Examination (KMLE).

METHODS

This study used a post-hoc (real data) simulation design. The item bank used in this study included all items from the January 2017 KMLE. All CAT algorithms for this study were implemented using the 'catR' package in the R program.

RESULTS

In terms of accuracy, the Rasch and 2-parametric logistic (PL) models performed better than the 3PL model. The 'modal a posteriori' and 'expected a posterior' methods provided more accurate estimates than maximum likelihood estimation or weighted likelihood estimation. Furthermore, maximum posterior weighted information and minimum expected posterior variance performed better than other item selection methods. In terms of efficiency, the Rasch model is recommended to reduce test length.

CONCLUSION

Before implementing live CAT, a simulation study should be performed under varied test conditions. Based on a simulation study, and based on the results, specific scoring and item selection methods should be predetermined.