Intraobserver and interobserver reliability of assessments of impairments and disabilities

The Mental Disability Military Assessment Tool: A Reliable Tool for Determining Disability in Veterans with Post-traumatic Stress Disorder

PURPOSE

An assessment tool was developed to assess disability in veterans who suffer from post-traumatic stress disorder (PTSD) due to a military mission. The objective of this study was to determine the reliability, intra-rater and inter-rater variation of the Mental Disability Military (MDM) assessment tool.

METHODS

Twenty-four assessment interviews of veterans with an insurance physician were videotaped. Each videotaped interview was assessed by a group of five independent raters on limitations of the veterans using the MDM assessment tool. After 2 months the raters repeated this procedure. Next the intra-rater and inter-rater variation was assessed with an adjusted version of AG09 computing weighted percentage agreement.

RESULTS

The results of this study showed that both the intra-rater variation and inter-rater variation on the ten subcategories of the MDM assessment tool were small, with an agreement of 84-100% within raters and 93-100% between raters.

CONCLUSIONS

The MDM assessment tool proves to be a reliable instrument to measure PTSD limitations in functioning in Dutch military veterans who apply for disability compensation. Further research is needed to assess the validity of this instrument.

The reliability of a 10-test package for patients with prolonged back and neck pain: could an examiner without formal medical education be used without loss of quality? A methodological study

Background

In the rehabilitation of patients with prolonged back and neck pain, the physical impairment should be assessed. Previous research has exclusively engaged medically educated examiners, mostly physiotherapists. However, less biased evaluations of efforts at rehabilitation might be achieved by personnel standing outside the treatment work itself. Therefore, if medically untrained examiners could be used without cost to the quality, this might produce a better evaluation at defensible cost and could also be useful in a research context. The aim of this study was to answer the question: given a 10-test package for patients with prolonged back and neck pain, could an examiner without formal medical education be used without loss of quality? Five of the ten tests required the examiner to keep a firm hold against the foundation of those parts of the participant's body that were not supposed to move during the test.

Methods

Examination by an experienced physiotherapist (A) in performing the package was compared with that by a research assistant (B) without formal medical education. The reliability, including inter- and intra-rater reliability, was assessed. In the inter-rater reliability study, 50 participants (30 patients + 20 healthy subjects) were tested once each by A and B. In the intra-rater reliability study, the 20 healthy subjects were tested twice by A or B. One-way ANOVA intra-class-correlation coefficient (ICC) was calculated and its possible systematic error was determined using a t-test.

Results

All five tests that required no manual fixation had acceptable reliability (ICC > .60 and no indication of systematic error). Only one of the five tests that required fixation had acceptable reliability. The difference (five vs. one) was significant (p = .01).

Conclusion

In a 10-test package for patients with prolonged back and neck pain, an examiner without formal medical education could be used without loss of quality, at least for the five tests requiring no manual fixation. To make our results more generalizable and their implications more searching, a similar study should be conducted with two or more examiners with and without formal medical education, and the intra-rater reliability study should also include patients and involve more participants.

Impairment measures in rheumatic disorders for rehabilitation medicine and allied health care: a systematic review

The objective of this study is to provide a critical overview of available instruments to assess impairments in patients with rheumatic disorders, and to recommend reliable and valid instruments for use in allied health care and rehabilitation medicine. A computer-aided literature search (1982-2004) in several databases was performed to identify studies focusing on the clinimetric properties of instruments designed to assess impairments in function in patients with rheumatic disorders. Data on intra-rater reliability, inter-rater reliability and construct validity were extracted in a standardized way. Explicit criteria were applied for reliability and validity.

RESULTS

The search identified a total of 49 instruments to assess impairments in functions in patients with rheumatic disorders; 19 met the criteria for reliability, 22 met the criteria for validity, and 11 out of the 49 appeared to meet the criteria for both reliability and validity. In summary, evidence of both reliability and validity was only found for 11 out of 49 instruments for the assessment of impairments in patients with rheumatic disorders. Only a limited number of the identified instruments for the assessment of impairments is both reliable and valid. Allied health care professionals should be cautious in the selection of measurement instruments to assess their patients.

Clinical physiotherapy documentation in stroke rehabilitation: an ICIDH-2 beta-2 based analysis

PURPOSE

To explore the focus of physiotherapists practising in stroke rehabilitation, through identification of central elements in clinical physiotherapy documentation.

METHOD

A descriptive study in which the Motor Assessment Scale for Stroke (MAS) and 20 physiotherapy records were collected from the stroke unit of a rehabilitation hospital and classified according to the ICIDH-2 beta-2. The text was broken down into single units of functional descriptions and assigned classification codes. The coded units were counted and the distribution of the units on the ICIDH-2 beta-2 examined.

RESULTS

In the physiotherapy records the majority of coded units belonged to the dimension of Body Functions, particularly to b7 Neuromusculoskeletal and movement related functions. In the MAS two thirds of the units belonged to Activities, with a3 Movement activities as the major group. There were no units related to Participation or Environmental Factors.

CONCLUSIONS

Physical functioning and movement were the central elements in this sample of clinical documentation, thus confirming earlier perception of physiotherapy. Lack of documented information on participation and environment could have several reasons. However, physiotherapists in stroke rehabilitation may consider if patients' environment and participation in life situations should have a larger place in clinical physiotherapy.

Clinical practice guideline for the physiotherapy of patients with whiplash-associated disorders

STUDY DESIGN

A clinical practice guideline.

OBJECTIVES

To assist physiotherapists in decision making and to improve the efficacy and uniformity of care for patients with whiplash-associated disorders Grades I and II.

SUMMARY OF BACKGROUND DATA

Whiplash constitutes a considerable problem in health care. Many interventions are used in physiotherapy practice, despite increasing evidence for the use of active interventions. There is still no clinical practice guideline for the management of patients with whiplash-associated disorders. METHOD OF DEVELOPMENT: A computerized literature search of Medline, Cinahl, Cochrane Controlled Trial Register, Cochrane Database of Systematic Reviews, and the Database of the Dutch National Institute of Allied Health Professions was performed to search for information about the diagnostic process and the therapeutic process in whiplash patients. When no evidence was available, consensus between experts was achieved to develop the guideline. Practicing physiotherapists reviewed the clinical applicability and feasibility of the guideline, and their comments were used to improve it.

RECOMMENDATIONS

The diagnostic process consists of systematic history taking and a physical examination supported by reliable and valid assessment tools to document symptoms and functional disabilities. The primary goals of treatment are a quick return to normal activities and the prevention of chronicity. Active interventions such as education, exercise therapy, training of functions, and activities are recommended according to the length of time since the accident and the rate of recovery. The biopsychosocial model is used to address the consequences of whiplash trauma.

CONCLUSIONS

Scientific evidence for the diagnosis and physiotherapeutic management of whiplash is sparse; therefore, consensus is used in different parts of the guideline. The guideline reflects the current state of knowledge of the effective and appropriate physiotherapy in whiplash patients. More and better research is necessary to validate this guideline in the future.

Interobserver reliability in physical examination of the cervical spine in patients with headache

OBJECTIVE

To assess interobserver reliability of two expert headache neurologists when examining the cervical spine of patients with headache.

BACKGROUND

The diagnosis of cervicogenic headache involves the physical examination of the cervical spine.

METHODS

Twenty-four patients diagnosed as having migraine, tension-type headache, or cervicogenic headache were included in the study. After interview, each patient's cervical spine was examined in a structured way. Reliability was assessed by Cohen's kappa.

RESULTS

Reduced range of motion in the cervical spine showed kappa scores indicating moderate agreement. Provocation of headache revealed moderate-to-substantial agreement. Assessment of zygapophyseal joint pressure pain showed slight-to-fair agreement. The kappa values of the circumscribed characteristic tender points showed agreement ranging from "not better than chance" to "substantial agreement."

CONCLUSIONS

Our study showed that the interobserver reliability of expert headache neurologists was satisfactory in the majority of the physical examination tests of the cervical spine in patients with different headache syndromes. However, standardization of the clinical tests in order to improve their reliability is recommended.

Intra- and inter-rater reliability of an 11-test package for assessing dysfunction due to back or neck pain

BACKGROUND AND PURPOSE

The intra- and inter-rater reliability of 11 tests assembled by physiotherapists for clinical purposes was investigated. Forty-five patients and 23 healthy volunteers participated in the study.

METHOD

Twenty-one patients were tested simultaneously and independently by two physiotherapists to determine inter-rater reliability for two raters. Twenty-four patients and 11 healthy volunteers were tested by one physiotherapist three times in a week to determine intra-rater reliability over time. Twelve healthy volunteers were tested by three different physiotherapists in a week to determine inter-rater reliability for three raters.

RESULTS

Inter-rater agreement for two simultaneous raters was clinically acceptable. Repeatability on three test occasions was clinically acceptable in six of the 11 tests. There were no systematic differences between occasions.

CONCLUSIONS

Intra- and inter-rater reliability was acceptable for six of the 11 tests in the form described here: three gait tests, two functional lifting tests and a functional muscular endurance test in the right leg. If these tests are to be used as outcome measures, account must be taken of the size of the typical fluctuation in measurements shown. The repeatability figures given may be used as guidelines for interpreting the clinical value of possible changes in test values.

Development of a uniform record for patients with burns of the hand

This study reports the development of a uniform record for patients with burns of the hands. The steps in the development of the record are outlined and discussed. The use of a uniform standardized record is intended to improve communication within a network of three burn centers and to facilitate joint research projects. A data entry method to facilitate pooling and subsequent data analysis is also described.