Stat-Knowlab. Assessment and Learning of Statistics with Competence-based Knowledge Space Theory

Interactive, computer-based, and situated design for innovative formative assessment approaches

Social and teamwork skills are essential for today's teachers, yet their assessment in authentic contexts is challenging. This study presents the design, development and validation of an innovative computer-based test, designed to assess collaborative and teamwork skills in preprimary and primary school teachers and referred to as the 'CoTeSt' (Collaborative and Teamwork Skill test). Based on Situated Action Theory, the test immerses participants in realistic team interactions using conversational agents within a narrative framework. Teachers are asked to solve problems, collaborate with virtual colleagues, and provide feedback. The test consists of 20 dichotomously scored items in both multiple-choice and short answer, and it was administered to 139 Italian teachers. Test validation involved qualitative and quantitative methods, confirming that the items actually assess the skills they were assumed to evaluate. Post-test interviews and group discussions highlighted the tool's user-friendly design and its potential to foster self-reflection, professional dialogue, and continuous skill development. The CoTeSt represents a meaningful step toward empowering teachers with critical social skills and fostering a culture of collaboration and growth in education.

Extracting preference relations from data: Clustering with transitive centroids

A clustering algorithm, named k-orders, is proposed to extract transitive relations from a data set. The k-orders algorithm differs from the original k-modes only in the adjustment step. Two adjustment procedures, named transitive centroid adjustment (TCA) and greedy TCA, are proposed that can be used to find clusters with transitive centroids. The proposed clustering approach finds application, especially in studies on preference, where this last may be heterogeneous across individuals, although transitive. The set of cluster centroids extracted by the algorithm from a data set can then be empirically tested via the estimation of a latent class model. The performance of the two versions of k-orders were compared to one another and with the canonical k-modes, in simulation studies. Results show that when centroids are transitive relations, both versions of k-orders outperform k-modes. Moreover, in experimental designs in which two-component options are considered, the TCA algorithm performs better than the greedy TCA. An empirical application was also carried out for exemplifying how k-orders can be useful for studying individual preferences.

PsycAssist: A Web-Based Artificial Intelligence System Designed for Adaptive Neuropsychological Assessment and Training

Assessing executive functions in individuals with disorders or clinical conditions can be challenging, as they may lack the abilities needed for conventional test formats. The use of more personalized test versions, such as adaptive assessments, might be helpful in evaluating individuals with specific needs. This paper introduces PsycAssist, a web-based artificial intelligence system designed for neuropsychological adaptive assessment and training. PsycAssist is a highly flexible and scalable system based on procedural knowledge space theory and may be used potentially with many types of tests. We present the architecture and adaptive assessment engine of PsycAssist and the two currently available tests: Adap-ToL, an adaptive version of the Tower of London-like test to assess planning skills, and MatriKS, a Raven-like test to evaluate fluid intelligence. Finally, we describe the results of an investigation of the usability of Adap-ToL and MatriKS: the evaluators perceived these tools as appropriate and well-suited for their intended purposes, and the test-takers perceived the assessment as a positive experience. To sum up, PsycAssist represents an innovative and promising tool to tailor evaluation and training to the specific characteristics of the individual, useful for clinical practice.

Algorithms for the adaptive assessment of procedural knowledge and skills

Procedural knowledge space theory (PKST) was recently proposed by Stefanutti (British Journal of Mathematical and Statistical Psychology, 72(2) 185-218, 2019) for the assessment of human problem-solving skills. In PKST, the problem space formally represents how a family of problems can be solved and the knowledge space represents the skills required for solving those problems. The Markov solution process model (MSPM) by Stefanutti et al. (Journal of Mathematical Psychology, 103, 102552, 2021) provides a probabilistic framework for modeling the solution process of a task, via PKST. In this article, three adaptive procedures for the assessment of problem-solving skills are proposed that are based on the MSPM. Beside execution correctness, they also consider the sequence of moves observed in the solution of a problem with the aim of increasing efficiency and accuracy of assessments. The three procedures differ from one another in the assumption underlying the solution process, named pre-planning, interim-planning, and mixed-planning. In two simulation studies, the three adaptive procedures have been compared to one another and to the continuous Markov procedure (CMP) by Doignon and Falmagne (1988a). The last one accounts for dichotomous correct/wrong answers only. Results show that all the MSP-based adaptive procedures outperform the CMP in both accuracy and efficiency. These results have been obtained in the framework of the Tower of London test but the procedures can also be applied to all psychological and neuropsychological tests that have a problem space. Thus, the adaptive procedures presented in this paper pave the way to the adaptive assessment in the area of neuropsychological tests.