Inter-rater reliability of formally trained and self-trained raters using the Edmonton Functional Assessment Tool

Psychometric Properties of the Japanese Version of the Edmonton Functional Assessment Tool 2

CONTEXT

Physical symptoms such as pain and cancer-related fatigue limit physical function and activities of daily living among patients with terminal cancer, which can lead to a decline in quality of life. Therefore, comprehensive functional impairments should be evaluated to determine the progression of the disease and the effectiveness of palliative treatment.

OBJECTIVE

To validate the psychometric properties of the Japanese version of the Edmonton Functional Assessment Tool 2 (EFAT2-J).

METHODS

We developed a Japanese version of the EFAT-2 in accordance with international guidelines. To verify the reliability and validity of the EFAT2-J, patients were evaluated by a physiotherapist and a nurse separately, and correlations with existing evaluation scales for physical function, physical symptoms, and quality of life were analyzed, respectively. The significance level was set at 5%.

RESULTS

Twenty patients participated in the reliability measurement. The average EFAT2-J scores were 7.95 ± 4.12 for physical therapists and 7.20 ± 4.23 for nurses, and the intraclass correlation coefficient was 0.95. The weighted kappa coefficient (κ) for each item was 0.57-1.00. Fifty-five patients participated in the validity measurement. The EFAT2-J showed significant correlations with Eastern Cooperative Oncology Group Performance Status and the Karnofsky Performance Scale, Barthel Index, and the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire-Core 15-Palliative Care sub-item "physical function."

CONCLUSION

These results indicate that the EFAT2-J has robust psychometric properties and is useful for evaluating physical function in patients with terminal cancer, and thus may be an acceptable clinical instrument in research and practice.

Interrater Agreement between Bedside and Video Raters Using the CPOT-Neuro for Pain Assessment in Critically Ill Patients with a Brain Injury

This study aimed to examine the interrater agreement of Critical-Care Pain Observation Tool-Neuro (CPOT-Neuro) scores as a newly developed tool for pain assessment in patients with critical illness and brain injury between raters using two methods of rating (bedside versus video) during standard care procedures (i.e., non-invasive blood pressure and turning). The bedside raters were research staff, and the two video raters had different backgrounds (health and non-health disciplines). Raters received standardized 45 min training by the principal investigator. Video recordings of 56 patient participants with a brain injury at different levels of consciousness were included. Interrater agreement was supported with an Intraclass Correlation Coefficient (ICC) > 0.65 for all pairs of raters and for each procedure. Interrater agreement was highest during turning in the conscious group, with ICCs ranging from 0.79 to 0.90. The use of video recordings was challenging for the observation of some behaviors (i.e., tearing, face flushing), which were influenced by factors such as lighting and the angle of the camera. Ventilator alarms were also challenging to distinguish from other sources for the video rater from a non-health discipline. Following standardized training, video technology was useful in achieving an acceptable interrater agreement of CPOT-Neuro scores between bedside and video raters for research purposes.

Evaluation of the Edmonton Functional Assessment Tool (EFAT2) within palliative care: a pilot study

BACKGROUND

There is increasing need for rehabilitation in both cancer and palliative care. However, there are few validated outcome measures that are suitable for measuring functional performance in this population. The present study evaluated the validity, sensitivity and reliability of the Edmonton Functional Assessment Tool (EFAT2) within a UK palliative care setting.

METHODS

Eleven participants aged 65 years and over (mean age 76.5 ± 6.7 years) receiving rehabilitation in a palliative care inpatient setting were studied. Concurrent validity was assessed by comparing EFAT2 with the Barthel Index. Inter-rater reliability of EFAT2 was examined using a sample of four participants recruited from a cancer care ward.

RESULTS

A significant negative correlation was observed between the Barthel Index and EFAT2 (r = -0.765, p = 0.01) and both measures were found to be sensitive as determined by Cohen's effect size (EFAT2 = 0.60, Barthel Index = 0.72). High inter-rater reliability was noted for EFAT2 (ICC3, 1 = 0.85) and the agreement between scores was confirmed by Bland-Altman analysis.

CONCLUSIONS

EFAT2 showed concurrent validity with the Barthel Index when used to assess the effects of rehabilitation on participants with advanced cancer. The tool was sensitive to change and was found to be reliable when used by different raters. The findings indicate that EFAT2 might be an appropriate outcome measure to use within the palliative care setting. However, the feasibility of using EFAT2 needs to be explored and larger studies are required to confirm its reliability.

Can the Dyskinesia Impairment Scale be used by inexperienced raters? A reliability study

BACKGROUND

The Dyskinesia Impairment Scale (DIS) is a new scale for measuring dystonia and choreoathetosis in dyskinetic Cerebral Palsy (CP). Previously, reliability of this scale has only been assessed for raters highly experienced in discriminating between dystonia and choreoathetosis.

AIMS

The aims of this study are to examine the reliability of the DIS used by inexperienced raters, new to discriminating between dystonia and choreoathetosis and to determine the effect of clinical expertise on reliability.

METHODS

Twenty-five patients (17 males; 8 females; age range 5-22 years; mean age = 13 years 6 months; SD = 5 years 4 months) with dyskinetic CP were filmed with the DIS standard video protocol. Two junior physiotherapists (PTs) and three senior PTs, all of whom were new to discriminating between dystonia and choreoathetosis, were trained in scoring the DIS. Afterward, they independently scored all patients from the video recordings using the DIS. Reliability was assessed by (1) Intraclass Correlation Coefficient (ICC), (2) Standard Error of Measurement (SEM) and Minimal Detectable Difference (MDD) and (3) Cronbach's alpha for internal consistency.

RESULTS

Interrater reliability for the total DIS, and for the dystonia and choreoathetosis subscales was good for the junior PTs and moderately high to excellent for the senior PTs. SEM and MDD values for the total DIS were 6% and 15% respectively for the junior PTs and 4% and 12% respectively for the senior PTs. Cronbach's alpha ranged between 0.87 and 0.95 for the junior PTs and between 0.76 and 0.93 for the senior PTs.

CONCLUSIONS

Reliability of the DIS scores for the inexperienced junior and senior PTs was sufficient in comparison with scores from the experienced raters in the previous study, indicating that the DIS can be used by inexperienced PTs new to discriminating between dystonia and choreoathetosis, and also that its reliability is not dependent on clinical expertise. However, based on the measurement errors and questionnaire data, familiarity with operational definitions of dystonia and choreoathetosis is crucial to improve scoring reliability.

Palliative care symptom assessment for patients with cancer in the emergency department: validation of the Screen for Palliative and End-of-life care needs in the Emergency Department instrument

OBJECTIVE

We sought to develop and validate a novel palliative medicine needs assessment tool for patients with cancer in the emergency department.

METHODS

An expert panel trained in palliative medicine and emergency medicine reviewed and adapted a general palliative medicine symptom assessment tool, the Needs at the End-of-Life Screening Tool. From this adaptation a new 13-question instrument was derived, collectively referred to as the Screen for Palliative and End-of-life care needs in the Emergency Department (SPEED). A database of 86 validated symptom assessment tools available from the palliative medicine literature, totaling 3011 questions, were then reviewed to identify validated test items most similar to the 13 items of SPEED; a total of 107 related questions from the database were identified. Minor adaptations of questions were made for standardization to a uniform 10-point Likert scale. The 107 items, along with the 13 SPEED items were randomly ordered to create a single survey of 120 items. The 120-item survey was administered by trained staff to all patients with cancer who met inclusion criteria (age over 21 years, English-speaking, capacity to provide informed consent) who presented to a large urban academic emergency department between 8:00 am and 11:00 pm over a 10-week period. Data were analyzed to determine the degree of correlation between SPEED items and the related 107 selected items from previously validated tools.

RESULTS

A total of 53 subjects were enrolled, of which 49 (92%) completed the survey in its entirety. Fifty-three percent of subjects were male, age range was 24-88 years, and the most common cancer diagnoses were breast, colon, and lung. Cronbach coefficient α for the SPEED items ranged from 0.716 to 0.991, indicating their high scale reliability. Correlations between the SPEED scales and related assessment tools previously validated in other settings were high and statistically significant.

CONCLUSION

The SPEED instrument demonstrates reliability and validity for screening for palliative care needs of patients with cancer presenting to the emergency department.

The PEACE project review of clinical instruments for hospice and palliative care

BACKGROUND

Hospice and palliative care organizations are expanding their use of standardized instruments and other approaches to measure quality. We undertook a systematic review and evaluation of published patient-level instruments for potential application in hospice and palliative care clinical quality measurement.

METHODS

We searched prior reviews and computerized reference databases from 1990 through February 2007 for studies of instruments relevant to physical, psychological, social, cultural, spiritual, or ethical aspects of palliative care, or measuring prognosis, function or continuity of care. Publications were selected for full review if they provided evidence of psychometric properties or practical application of an instrument tested in or appropriate for a hospice or palliative care population. Selected instruments were evaluated and scored for scientific soundness and potential application in clinical quality measurement.

RESULTS

The search found 1427 publications, with 229 selected for full manuscript review. Manuscripts provided information on 129 instruments which were evaluated using a structured scoring guide for psychometric properties. Thirty-nine instruments scoring near or above the 75th percentile were recommended. Most instruments covered multiple domains or focused on care for physical symptoms, psychological or social aspects of care. Few instruments were available to measure cultural aspects of care, structure and process of care, and continuity of care.

CONCLUSION

Numerous patient-level instruments are available to measure physical, psychological and social aspects of palliative care with adequate evidence for scientific soundness and practical clinical use for quality improvement and research. Other aspects of palliative care may benefit from further instrument development research.

Methods to improve reliability of video-recorded behavioral data

Behavioral observation is a fundamental component of nursing practice and a primary source of clinical research data. The use of video technology in behavioral research offers important advantages to nurse scientists in assessing complex behaviors and relationships between behaviors. The appeal of using this method should be balanced, however, by an informed approach to reliability issues. In this article, we focus on factors that influence reliability, such as the use of sensitizing sessions to minimize participant reactivity and the importance of training protocols for video coders. In addition, we discuss data quality, the selection and use of observational tools, calculating reliability coefficients, and coding considerations for special populations based on our collective experiences across three different populations and settings.

A systematic review of measures of end-of-life care and its outcomes

OBJECTIVE

To identify psychometrically sound measures of outcomes in end-of-life care and to characterize their use in intervention studies.

DATA SOURCES

English language articles from 1990 to November 2005 describing measures with published psychometric data and intervention studies of end-of-life care.

STUDY DESIGN

Systematic review of end-of-life care literature.

EXTRACTION METHODS

Two reviewers organized identified measures into 10 major domains. Eight reviewers extracted and characterized measures from intervention studies.

PRINCIPAL FINDINGS

Of 24,423 citations, we extracted 200 articles that described 261 measures, accepting 99 measures. In addition to 35 measures recommended in a prior systematic review, we identified an additional 64 measures of the end-of-life experience. The most robust measures were in the areas of symptoms, quality of life, and satisfaction; significant gaps existed in continuity of care, advance care planning, spirituality, and caregiver well-being. We also reviewed 84 intervention studies in which 135 patient-centered outcomes were assessed by 97 separate measures. Of these, 80 were used only once and only eight measures were used in more than two studies.

CONCLUSIONS

In general, most measures have not undergone rigorous development and testing. Measure development in end-of-life care should focus on areas with identified gaps, and testing should be done to facilitate comparability across the care settings, populations, and clinical conditions. Intervention research should use robust measures that adhere to these standards.

Symptom control in palliative care--Part I: oncology as a paradigmatic example

Achieving the best quality of life for patients and their families when a disease becomes progressive and no longer remains responsive to curative therapy is the primary goal of palliative care. A comprehensive care plan focusing on control of physical symptoms as well as psychological, social, and spiritual issues then becomes paramount in that context. Symptom assessment and treatment are a principle part of palliative care. This paper is the first of three in a series addressing non-pain symptoms, which are frequently encountered in the palliative care populations. The most frequent non-pain symptoms are constipation, chronic nausea and vomiting, anorexia, dyspnea, fatigue, and delirium. As symptoms are subjective, their expression varies from patient to patient, depending on the individual patient's perception and on other factors such as psychosocial issues. While symptoms are addressed individually, patients frequently have multiple coexisting symptoms. Generally told, once the intensity of a symptom has been assessed, it is necessary to assess the symptom in the context of other symptoms such as pain, appetite, fatigue, depression, and anxiety. Given that fact, adopting a multidimensional assessment allows for formulation of a more effective therapeutic strategy. More pertinently, this paper highlights the management of non-pain symptoms as an integral part of patient care and reviews the pathophysiologies, causes, assessment, and management of constipation, chronic nausea, and vomiting, each of which is common among the palliative care population.

Effect of fish oil on appetite and other symptoms in patients with advanced cancer and anorexia/cachexia: a double-blind, placebo-controlled study

PURPOSE

To determine whether high doses of fish oil, administered over 2 weeks, improve symptoms in patients with advanced cancer and decreased weight and appetite.

PATIENTS AND METHODS

Sixty patients were randomly assigned to fish oil capsules or placebo. Appetite, tiredness, nausea, well-being, caloric intake, nutritional status, and function were prospectively assessed at days 1 and 14.

RESULTS

The baseline weight loss was 16 +/- 11 and 16 +/- 8 kg in the fish oil (n = 30) and placebo (n = 30) group respectively, whereas the baseline appetite (0 mm = best and 10 mm = worst) was 58 +/- 24 mm and 67 +/- 19 mm, respectively (P = not significant). The mean daily dose was 10 +/- 4 (fish oil group) and 9 +/- 3 (placebo group) capsules, which provided 1.8 g of eicosapentaenoic acid and 1.2 g of docosahexaenoic acid in the fish oil group. No significant differences in symptomatic or nutritional parameters were found (P <.05), and there was no correlation between changes in different variables between days 1 and 14 and the fish oil doses. Finally, the majority of the patients were not able to swallow more than 10 fish oil capsules per day, mainly because of burping and aftertaste.

CONCLUSION

Fish oil did not significantly influence appetite, tiredness, nausea, well-being, caloric intake, nutritional status, or function after 2 weeks compared with placebo in patients with advanced cancer and loss of both weight and appetite.

Update on anorexia and cachexia

Declining physical, emotional, and social function as a result of anorexia and cachexia are considerable contributors to discomfort for cancer patients and their families, and they impair the patient's ability to express optimal physical and psychosocial potential as long as possible. This decline no longer has to be accepted as an indispensable sequel to advanced cancer, just as pain is no longer considered to be unavoidable. A routine screening for anorexia and cachexia and associated symptoms is necessary, as is a careful, comprehensive assessment, because the condition is not always obvious. Decisions about anorexia and cachexia treatment are guided by prioritizing the different, concurrent physical, psychosocial, and existential problems and by considering the natural course of the cancer and the effects of antineoplastic therapies. Reversible causes for anorexia and cachexia need to be identified and treated, if appropriate. Nutritional interventions are often indicated; patients with a predominant starvation component and without inflammation may profit the most. New pharmacologic therapies for primary anorexia and cachexia syndrome are expected to enter clinical practice soon; however, until then, treatment with corticosteroids, progestins, or prokinetics may be indicated for some patients. To understand a multicausal syndrome, multimodal and interdisciplinary therapy is required. Specialist palliative care services can be helpful to provide, hand-in-hand with the disease specialists [172], assessment and management of psychophysical symptoms and sociospiritual needs of patients during the course of the illness and at the end of life [173]. Research efforts aim to better characterize subgroups of patients suffering from secondary causes of anorexia and cachexia and to elucidate the mechanisms involved in the primary anorexia and cachexia syndrome. Increasingly individualized treatments are expected with combination treatments that involve different mechanisms including nutrition.