Development of a parent-report computer-adaptive test to assess physical functioning in children with cerebral palsy I: lower-extremity and mobility skills

Scoping Review of Judgment-Based Measures of Ambulation with Assistive Devices for Children and Youth

AIM

To identify available judgment-based measures of ambulation with assistive devices for the purpose of examining item content and responses to aid in the expansion of the Pediatric Evaluation of Disability Inventory Computer Adaptive Test (PEDI-CAT) Mobility Domain.

METHODS

PubMed and CINAHL databases were used to identify measures meeting the following criteria: 1) applicable for children/youth; 2) self-report, proxy-report, or interview administration; and 3) assistive device (walker, cane, crutches, gait trainer) use specified or considered with responses. Population, administration, respondent(s), items, and responses were compiled. Item content was categorized and response scales grouped by type.

RESULTS

Fifteen measures met inclusion criteria. Measures included child and proxy-report. Item categories included Surfaces, Steps/Stairs, Dual Tasks, Negotiation of Environment, Distance, and Time. Only two measures distinguished between device type within items. One measure specified gait trainers. "Difficulty" and "Assistance" were the most frequently used response scales.

CONCLUSIONS

Available measures have content examining device use; however, none of the measures are comprehensive, devices are not consistently specified, and responses are imprecise. Items with well-defined responses for measuring a child's ambulation with an assistive device are needed for clinical practice, research, and program evaluation.

Development of a proxy-reported scale to assess motor function in infants and young children with early-onset neuromuscular disorders

AIM

To develop a novel proxy-reported scale of motor function in infants and young children with early-onset neuromuscular disorders (NMD), entitled the Proxy Motor Outcome Measure (PMOM).

DESIGN

A mixed method design was employed, applying both qualitative and quantitative research.

METHODS

A framework technique using sensitivity analyses guided the development of the most appropriate and relevant subset of items, modelled after 30 neuromuscular disease instruments/scales. The PMOM was designed based on semi-structured interviews with 16 proxies; a focus group of 11 experts in neuromuscular diseases and scale development, 10 of whom also gave quantitative data using a two-round Delphi method survey; and cognitive interviews with five proxies. These processes were conducted between January 2014-March 2019.

RESULTS

Nine themes and 32 subthemes were derived from the semi-structured interviews. Five domains and three subdomains of potential items were identified by the focus group. An initial version of the PMOM scale was created with 121 items. Using the two-round Delphi method, 43 items met agreement on pre-defined requirements. The second version of the PMOM scale included these 43 and two additional items based on expert feedback. Proxies gave 114 suggestions on cognitive interviews, 99 of which were successfully addressed by the research team. The final version of the PMOM scale included 43 items.

CONCLUSION

We developed a preliminary proxy-reported instrument, the PMOM, to evaluate motor function in infants and young children with early-onset NMD.

IMPACT

Proxies hold a wealth of knowledge on their child's motor function during early development, which may complement clinic-based motor function testing. However, there is no validated measure of motor function that incorporates the observation of proxies of infants and young children with NMD. Future work will be focused on assessing the reliability, validity and responsiveness of the PMOM scale and implementing this tool in clinical studies.

The Cerebral Palsy Profile of Health and Function: Upper-Extremity Domain's Sensitivity to Change Following Musculoskeletal Surgery

PURPOSE

The Cerebral Palsy Profile of Health and Function (CP-PRO) Computerized Adaptive Tests (CAT) are quality of life measures developed specifically for use in children with cerebral palsy. This study examined the ability of the upper-extremity (UE) CP-PRO CAT to detect change in function after UE surgery compared with the Pediatric Outcomes Data Collection Instrument (PODCI), ABILHAND-Kids, and Box and Blocks test.

METHODS

From 2009 to 2013, children with cerebral palsy who had UE musculoskeletal surgery completed the UE CP-PRO CAT, PODCI-UE, ABILHAND-Kids, and Box and Blocks tests before surgery (97 children) and at 3 postoperative intervals: 6 months (80 children), 12 months (73 children), and 24 months (52 children). Mean, SD, effect size (ES), and standardized response mean (SRM) values for each measure at each time interval and each level of the Manual Ability Classification System were calculated and compared. Finally, the minimal detectable change at the 90% confidence level was determined.

RESULTS

Values for the ES (0.40) and SRM (0.53) for the UE CP-PRO CAT at baseline to 6 months were moderate and significantly greater than the PODCI-UE (ES, 0.18; SRM, 0.25). The ES and SRM for the PODCI-UE, ABILHAND-Kids, and Box and Blocks tests were not significantly greater than for the UE CP-PRO CAT at any period. From baseline to 6 months, the UE CP-PRO CAT detected a large and significant improvement for Manual Ability Classification System level II (SRM, 0.70; ES, 0.70). The minimal detectable change for the UE CP-PRO CAT was 5.20.

CONCLUSIONS

The UE CP-PRO CAT is significantly better in detecting change in UE function in the first 6 months after surgery and is comparable to other measures at 12 and 24 months.

TYPE OF STUDY/LEVEL OF EVIDENCE

Diagnostic II.

Functional Gains in Children With Spastic Hemiplegia Following a Tendon Achilles Lengthening Using Computerized Adaptive Testing-A Pilot Study

Purpose:

This pilot study evaluated the outcomes of tendon Achilles lengthening in 12 children (mean age: 11.2 years) with spastic hemiplegia.

Methods:

Cerebral Palsy Computer Adaptive Tests, the timed up-and-go, the Gross Motor Function Measure, the Gillette Functional Assessment Questionnaire, and the Pediatric Outcomes Data Collection Instrument were administered at baseline and at 6, 12, and 24 months postsurgery.

Results:

Significant improvement at the latest follow-up (12-24 months following surgery) was seen in all domains of the Cerebral Palsy Computer Adaptive Test: activity (P = .017), lower extremity (P = .005), global (P = .005), pain (P = .005), and fatigue (P = .028), as well as in the Gross Motor Function Measure-standing domain (P = .02) and the mobility domain of the Pediatric Outcomes Data Collection Instrument (P = .04).

Conclusion:

These findings indicate that the tendon Achilles lengthening improved functional outcome in these children as measured by tests of physical function, walking speed, and activity performance.

A new computerized adaptive test advancing the measurement of health-related quality of life (HRQoL) in children: the Kids-CAT

PURPOSE

Assessing health-related quality of life (HRQoL) via Computerized Adaptive Tests (CAT) provides greater measurement precision coupled with a lower test burden compared to conventional tests. Currently, there are no European pediatric HRQoL CATs available. This manuscript aims at describing the development of a HRQoL CAT for children and adolescents: the Kids-CAT, which was developed based on the established KIDSCREEN-27 HRQoL domain structure.

METHODS

The Kids-CAT was developed combining classical test theory and item response theory methods and using large archival data of European KIDSCREEN norm studies (n = 10,577-19,580). Methods were applied in line with the US PROMIS project. Item bank development included the investigation of unidimensionality, local independence, exploration of Differential Item Functioning (DIF), evaluation of Item Response Curves (IRCs), estimation and norming of item parameters as well as first CAT simulations.

RESULTS

The Kids-CAT was successfully built covering five item banks (with 26-46 items each) to measure physical well-being, psychological well-being, parent relations, social support and peers, and school well-being. The Kids-CAT item banks proved excellent psychometric properties: high content validity, unidimensionality, local independence, low DIF, and model conform IRCs. In CAT simulations, seven items were needed to achieve a measurement precision between .8 and .9 (reliability). It has a child-friendly design, is easy accessible online and gives immediate feedback reports of scores.

CONCLUSIONS

The Kids-CAT has the potential to advance pediatric HRQoL measurement by making it less burdensome and enhancing the patient-doctor communication.

Development of items that assess physical function in children who use wheelchairs

PURPOSE

To assess the content, format, and comprehension of the Patient Reported Outcomes Measurement Information System (PROMIS) pediatric physical function related to mobility items for children who use wheelchairs (WCs).

METHODS

During a cognitive interview, 14 children, aged 8 to 12 years, who use WCs, verbalized their thoughts when answering PROMIS items. The questionnaire appraisal system was used to code summarized text from the interviews.

RESULTS

The children requested items be more specific and include options for reporting adaptive ways of performing and participating. How they would answer the item depended on the situation and specific environmental supports and constraints they may have experienced.

CONCLUSIONS

As rehabilitation professionals develop and use self-reported outcome measures, they should explore what is important to children who use WCs regarding their views on physical functioning, the influences of the environment, and variability in the use of devices to assist with functional mobility.

Evaluation of the box and blocks test, stereognosis and item banks of activity and upper extremity function in youths with brachial plexus birth palsy

BACKGROUND

One of the greatest limitations to measuring outcomes in pediatric orthopaedics is the lack of effective instruments. Computer adaptive testing, which uses large item banks, select only items that are relevant to a child's function based on a previous response and filters items that are too easy or too hard or simply not relevant to the child. In this way, computer adaptive testing provides for a meaningful, efficient, and precise method to evaluate patient-reported outcomes. Banks of items that assess activity and upper extremity (UE) function have been developed for children with cerebral palsy and have enabled computer adaptive tests that showed strong reliability, strong validity, and broader content range when compared with traditional instruments. Because of the void in instruments for children with brachial plexus birth palsy (BPBP) and the importance of having an UE and activity scale, we were interested in how well these items worked in this population.

METHODS

Cross-sectional, multicenter study involving 200 children with BPBP was conducted. The box and block test (BBT) and Stereognosis tests were administered and patient reports of UE function and activity were obtained with the cerebral palsy item banks. Differential item functioning (DIF) was examined. Predictive ability of the BBT and stereognosis was evaluated with proportional odds logistic regression model. Spearman correlations coefficients (rs) were calculated to examine correlation between stereognosis and the BBT and between individual stereognosis items and the total stereognosis score.

RESULTS

Six of the 86 items showed DIF, indicating that the activity and UE item banks may be useful for computer adaptive tests for children with BPBP. The penny and the button were strongest predictors of impairment level (odds ratio=0.34 to 0.40]. There was a good positive relationship between total stereognosis and BBT scores (rs=0.60). The BBT had a good negative (rs=-0.55) and good positive (rs=0.55) relationship with the clinical category of BPBP and Raimondi classification, respectively. There was a moderate negative (rs=-0.36) and moderate positive (rs=0.47) relationship between total stereognosis and clinical category of BPBP and Raimondi, respectively. Individual stereognosis items had moderate (rs=0.34 to 0.45) to good (rs=0.52 to 0.74) correlation with total stereognosis score.

CONCLUSIONS

In children with BPBP, there was little to no DIF with item banks of activity and UE functioning. The BBT is a good predictor of degree of impairment. Stereognosis with 2 items may provide comparable information as 12 items.

LEVELS OF EVIDENCE

Level II.

Accuracy and precision of the Pediatric Evaluation of Disability Inventory computer-adaptive tests (PEDI-CAT)

AIM

The aims of the study were to: (1) build new item banks for a revised version of the Pediatric Evaluation of Disability Inventory (PEDI) with four content domains: daily activities, mobility, social/cognitive, and responsibility; and (2) use post-hoc simulations based on the combined normative and disability calibration samples to assess the accuracy and precision of the PEDI computer-adaptive tests (PEDI-CAT) compared with the administration of all items.

METHOD

Parents of typically developing children (n = 2205) and parents of children and adolescents with disabilities (n = 703) between the ages of 0 and 21 years, stratified by age and sex, participated by responding to PEDI-CAT surveys through an existing Internet opt-in survey panel in the USA and by computer tablets in clinical sites.

RESULTS

Confirmatory factor analyses supported four unidimensional content domains. Scores using the real data post hoc demonstrated excellent accuracy (intraclass correlation coefficients ≥ 0.95) with the full item banks. Simulations using item parameter estimates demonstrated relatively small bias in the 10-item and 15-item CAT versions; error was generally higher at the scale extremes.

INTERPRETATION

These results suggest the PEDI-CAT can be an accurate and precise assessment of children's daily performance at all functional levels.

Factor- and item-level analyses of the 38-item activities scale for kids-performance

AIM

Children and adolescents highly value their ability to participate in relevant daily life and recreational activities. The Activities Scale for Kids-performance (ASKp) instrument measures the frequency of performance of 30 common childhood activities, and has been shown to be valid and reliable. A revised and expanded 38-item ASKp (ASKp38) version has been reported in recent literature and is currently used in clinical research. The aim of this paper is to assess the factor structure and item-level statistics of the ASKp38.

METHOD

Our study used factor analyses and Rasch analyses to determine the item-set dimensionality and to calculate item-level statistics respectively, for existing ASKp38 data from 200 children (104 males; 96 females; mean age 12y 7mo; SD 2y 8mo; range 6-20y) with physical disabilities. The children had a variety of physical impairments including cerebral palsy (n = 105; range 8-13 y), limb salvage (n = 18; range 11-20y), arthrogryposis (n = 13; 6-17y), and other, including individuals with spina bifida and spinal cord injury (n = 64; 8-19 y).

RESULTS

A two-factor model, with components of activities of daily living and play, most optimally fit the data. Item-fit statistics based on this two-factor model demonstrated adequate fit and content coverage.

INTERPRETATION

The ASKp38 appears to consist of two factors, defined as (1) activities of daily living and (2) play, and may be used to measure the frequency of activity performance on two corresponding subscales.

Content range and precision of a computer adaptive test of upper extremity function for children with cerebral palsy

This article reports on the content range and measurement precision of an upper extremity (UE) computer adaptive testing (CAT) platform of physical function in children with cerebral palsy. Upper extremity items representing skills of all abilities were administered to 305 parents. These responses were compared with two traditional standardized measures: Pediatric Outcomes Data Collection Instrument and Functional Independence Measure for Children. The UE CAT correlated strongly with the upper extremity component of these measures and had greater precision when describing individual functional ability. The UE item bank has wider range with items populating the lower end of the ability spectrum. This new UE item bank and CAT have the capability to quickly assess children of all ages and abilities with good precision and, most importantly, with items that are meaningful and appropriate for their age and level of physical function.

Item bank development for a revised pediatric evaluation of disability inventory (PEDI)

The Pediatric Evaluation of Disability Inventory (PEDI) is a useful clinical and research assessment, but it has limitations in content, age range, and efficiency. The purpose of this article is to describe the development of the item bank for a new computer adaptive testing version of the PEDI (PEDI-CAT). An expanded item set and response options were reviewed by clinician experts and examined at parent and clinician focus groups. Eleven parents participated in 32 cognitive interviews to examine content, format, and comprehension of items and responses. A set of 76 self-care, 78 mobility, and 64 social function items with pictures and a four-point "Difficulty" scale were developed. The PEDI's Caregiver Assistance scale was replaced by a "Responsibility Scale" with 53 items. Content validity was established incorporating input from clinicians and parents. The new item bank covers a broad range of functional activities for children of all ages and abilities.

Measuring physical function in children with airway support: a pilot study using computer adaptive testing

OBJECTIVE

To assess the responsiveness, examine the scoring range and determine the efficiency of a multidimensional computer adaptive testing version of the Pediatric Evaluation of Disability Inventory (PEDI-MCAT) for children admitted to inpatient pulmonary rehabilitation.

METHODS

The PEDI-MCAT was completed by clinician report for 30 infants and children. Mean self-care and mobility admission scores were compared with discharge scores for the total group and two diagnostic sub-groups (prematurity and congenital/neurological conditions). The scoring range of the mobility and self-care scales was examined to determine placement of the scores along the overall PEDI-MCAT scale. Efficiency was determined using an internal clock and average number of items required for score generation.

RESULTS

Mean changes for the total group and both sub-groups were significant for both self-care and mobility, except for the prematurity group's mobility scores. Effect sizes were small-to-moderate. Scores for both groups were at the low end of the scoring ranges. Average time to complete the PEDI-MCAT was 1.57 minutes. Average number of items administered was nine for self-care and 11 for mobility.

CONCLUSION

The PEDI-MCAT was responsive to change in physical function, although only low-ability items were needed. The PEDI-MCAT can potentially minimize clinician burden in inpatient settings.

Precision and content range of a parent-reported item bank assessing lower extremity and mobility skills in children with cerebral palsy

AIM

The aim of this study was to determine the psychometric properties, content range, and measurement precision of a lower extremity physical functioning and mobility skills item bank (LE85) in children with cerebral palsy (CP).

METHOD

Lower extremity functioning and mobility skill items were administered to 308 parents of children (169 males, 139 females; mean age 10y 8mo, SD 4y) with spastic CP (145 diplegia, 73 hemiplegia, 89 quadriplegia; [for one person type of CP was unknown]) classified using the Gross Motor Function Classification System (75 level I, 91 level II, 79 level III, 37 level IV, 26 level V). Additional legacy measures were administered to assess concurrent validity. Psychometric characteristics, differential item functioning, content range, and score precision were examined.

RESULTS

The LE85 had acceptable psychometric properties. Content range matched the ability range of the sample population and exceeded legacy measures with minimal differential item functioning. The LE85 had good correlation with the Paediatric Outcomes Data Collection Instrument, Functional Independence Measure for Children, Gillette Functional Assessment Questionnaire, and Paediatric Quality of Life Inventory-CP module (range r=0.63-0.86). Precision of the LE85 and 10-item simulated computer adaptive test scores outperformed legacy measures.

INTERPRETATION

The LE85 appears to be suitable to administer as a computer adaptive test to measure lower extremity physical functioning and mobility in children with CP.

Validity and reliability of physical functioning computer-adaptive tests for children with cerebral palsy

BACKGROUND

The purpose of this study was to assess the concurrent validity and reliability of scores from 4 new parent-report computer-adapted testing (CAT) programs developed to measure the physical functioning of children with cerebral palsy (CP). The Shriners Hospitals for Children CP-CAT battery includes upper-extremity skills, lower-extremity and mobility skills, activity, and global physical health.

METHODS

This was a prospective study of 91 children with CP who were tested cross-sectionally and 27 children with CP who were administered the CP-CAT programs twice within approximately a 1-month interval. We examined the concurrent validity of the 4 Shriners Hospitals for Children CP-CAT programs by Pearson correlations with comparative parent-report instruments. The scale reliability was tested by developing estimates of marginal reliability; test-retest reliability was assessed by intraclass correlations.

RESULTS

Pearson correlations were moderate to high in matching content domains of the CATs with the comparison measures. Marginal reliability estimates were always better for the CAT program than the comparative instruments. Average test-retest reliability using Intraclass correlations across the 4 CATs was ICC3,1=0.91 with a range of 0.88 to 0.94.

CONCLUSIONS

We found the CAT scores to be related to expected domains from external instruments, to have good scale reliability, and to have stable scores as determined by test-retest reliability. These results support the use of parent-report CATs in the assessment of physical functioning in children with CP.

Evaluation of an item bank for a computerized adaptive test of activity in children with cerebral palsy

BACKGROUND

Contemporary clinical assessments of activity are needed across the age span for children with cerebral palsy (CP). Computerized adaptive testing (CAT) has the potential to efficiently administer items for children across wide age spans and functional levels.

OBJECTIVE

The objective of this study was to examine the psychometric properties of a new item bank and simulated computerized adaptive test to assess activity level abilities in children with CP.

DESIGN

This was a cross-sectional item calibration study.

METHODS

The convenience sample consisted of 308 children and youth with CP, aged 2 to 20 years (X=10.7, SD=4.0), recruited from 4 pediatric hospitals. We collected parent-report data on an initial set of 45 activity items. Using an Item Response Theory (IRT) approach, we compared estimated scores from the activity item bank with concurrent instruments, examined discriminate validity, and developed computer simulations of a CAT algorithm with multiple stop rules to evaluate scale coverage, score agreement with CAT algorithms, and discriminant and concurrent validity.

RESULTS

Confirmatory factor analysis supported scale unidimensionality, local item dependence, and invariance. Scores from the computer simulations of the prototype CATs with varying stop rules were consistent with scores from the full item bank (r=.93-.98). The activity summary scores discriminated across levels of upper-extremity and gross motor severity and were correlated with the Pediatric

OUTCOMES

Data Collection Instrument (PODCI) physical function and sports subscale (r=.86), the Functional Independence Measure for Children (Wee-FIM) (r=.79), and the Pediatric Quality of Life Inventory-Cerebral Palsy version (r=.74).

LIMITATIONS

The sample size was small for such IRT item banks and CAT development studies. Another limitation was oversampling of children with CP at higher functioning levels.

CONCLUSIONS

The new activity item bank appears to have promise for use in a CAT application for the assessment of activity abilities in children with CP across a wide age range and different levels of motor severity.