Searching for information on outcomes: do you need to be comprehensive?

Measures of social connectedness in adult populations: a systematic review

BACKGROUND

Poor social connectedness has been identified as a risk factor for poor mental health but there is a lack of standardisation in how it is measured. This systematic review aimed to identify suitable measures of social connectedness for use in UK adult general populations.

METHODS

Searches were undertaken in two stages to identify: (1) measures of social connectedness from review articles and grey literature and (2) studies reporting on the psychometric properties of the identified measures. Grey literature and five databases were searched: MEDLINE, Embase and PsycINFO; CINAHL and Web of Science. Studies based on UK adult general populations (16-65 years) or other English language speaking countries with similar cultures (US, Canada, Ireland, Australia and New Zealand) were included. Psychometric evidence was extracted relating to six general domains: conceptual model, content validity, reliability, construct validity, scoring and interpretability, and respondent burden and presentation. A narrative synthesis summarised these psychometric properties.

RESULTS

Stage (1) 2,396 studies were retrieved and, 24 possible measures of social connectedness were identified; stage (2) 6,218 studies were identified reporting on psychometrics of identified measures and 22 studies were included. These studies provided psychometric evidence for 10 measures, and we did not find psychometric studies for the other identified measures. Six measures (6/10, 60%) reported assessing loneliness and four (4/10, 40%) reported assessing social support but there was a degree of overlap between the assessments of each concept. There was good evidence of reliability across measures, 90% (9/10) had adequate internal consistency, but evidence of content validity was only available for one scale. Five measures (5/10, 50%) reported on at least half of the psychometric criteria, and these were: UCLA-3 (for loneliness), and MSPSS, F-SozU K-6, SPS-10 and SPS-5 (for social support).

CONCLUSIONS

This review identified ten social connectedness measures, and identified UCLA-3, MSPSS, F-SozUK-6, SPS-10, and SPS-5 as having the most robust psychometric properties for the UK adult population. Further testing is required to establish content validity, and to clarify the definition and conceptualisation of social connectedness, to enable standardisation in the approach to measuring social connectedness.

Search strategies (filters) to identify systematic reviews in MEDLINE and Embase

BACKGROUND

Bibliographic databases provide access to an international body of scientific literature in health and medical sciences. Systematic reviews are an important source of evidence for clinicians, researchers, consumers, and policymakers as they address a specific health-related question and use explicit methods to identify, appraise and synthesize evidence from which conclusions can be drawn and decisions made. Methodological search filters help database end-users search the literature effectively with different levels of sensitivity and specificity. These filters have been developed for various study designs and have been found to be particularly useful for intervention studies. Other filters have been developed for finding systematic reviews. Considering the variety and number of available search filters for systematic reviews, there is a need for a review of them in order to provide evidence about their retrieval properties at the time they were developed.

OBJECTIVES

To review systematically empirical studies that report the development, evaluation, or comparison of search filters to retrieve reports of systematic reviews in MEDLINE and Embase.

SEARCH METHODS

We searched the following databases from inception to January 2023: MEDLINE, Embase, PsycINFO; Library, Information Science & Technology Abstracts (LISTA) and Science Citation Index (Web of Science).

SELECTION CRITERIA

We included studies if one of their primary objectives is the development, evaluation, or comparison of a search filter that could be used to retrieve systematic reviews on MEDLINE, Embase, or both.

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted data using a pre-specified and piloted data extraction form using InterTASC Information Specialist Subgroup (ISSG) Search Filter Evaluation Checklist.

MAIN RESULTS

We identified eight studies that developed filters for MEDLINE and three studies that developed filters for Embase. Most studies are very old and some were limited to systematic reviews in specific clinical areas. Six included studies reported the sensitivity of their developed filter. Seven studies reported precision and six studies reported specificity. Only one study reported the number needed to read and positive predictive value. None of the filters were designed to differentiate systematic reviews on the basis of their methodological quality. For MEDLINE, all filters showed similar sensitivity and precision, and one filter showed higher levels of specificity. For Embase, filters showed variable sensitivity and precision, with limited study reports that may affect accuracy assessments. The report of these studies had some limitations, and the assessments of their accuracy may suffer from indirectness, considering that they were mostly developed before the release of the PRISMA 2009 statement or due to their limited scope in the selection of systematic review topics. Search filters for MEDLINE Three studies produced filters with sensitivity > 90% with variable degrees of precision, and only one of them was developed and validated in a gold-standard database, which allowed the calculation of specificity. The other two search filters had lower levels of sensitivity. One of these produced a filter with higher levels of specificity (> 90%). All filters showed similar sensitivity and precision in the external validation, except for one which was not externally validated and another one which was conceptually derived and only externally validated. Search filters for Embase We identified three studies that developed filters for this database. One of these studies developed filters with variable sensitivity and precision, including highly sensitive strategies (> 90%); however, it was not externally validated. The other study produced a filter with a lower sensitivity (72.7%) but high specificity (99.1%) with a similar performance in the external validation.

AUTHORS' CONCLUSIONS

Studies reporting the development, evaluation, or comparison of search filters to retrieve reports of systematic reviews in MEDLINE showed similar sensitivity and precision, with one filter showing higher levels of specificity. For Embase, filters showed variable sensitivity and precision, with limited information about how the filter was produced, which leaves us uncertain about their performance assessments. Newer filters had limitations in their methods or scope, including very focused subject topics for their gold standards, limiting their applicability across other topics. Our findings highlight that consensus guidance on the conduct of search filters and standardized reporting of search filters are needed, as we found highly heterogeneous development methods, accuracy assessments and outcome selection. New strategies adaptable across interfaces could enhance their usability. Moreover, the performance of existing filters needs to be evaluated in light of the impact of reporting guidelines, including the PRISMA 2009, on how systematic reviews are reported. Finally, future filter developments should also consider comparing the filters against a common reference set to establish comparative performance and assess the quality of systematic reviews retrieved by strategies.

Assessment of the quality of services of reception with risk classification in obstetrics: a scoping review protocol

INTRODUCTION

Quality assessment in the context of maternal health services in Brazil has been the object of study due to maternal mortality rates that remain high in the country, in addition to the high costs of healthcare and the increased level of complexity in care. To change this situation, several strategies have been proposed to improve care for women. One of them is the improvement of women's access to maternal and child services through the Reception with Obstetric Risk Classification (ACCRO).

OBJECTIVE

To map and synthesise scientific evidence in the literature of studies that assess the quality of ACCRO services from the perspective of users, professionals and managers, as well as map tools that assess the quality of these services and their results.

METHOD AND ANALYSIS

It is a scoping protocol that follows the method recommended by the Joanna Briggs Institute, other authors and guided by the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) guidelines. The Population, Concept and Context strategy will systematise the search in PubMed, Cochrane, Embase, LILACS, SCOPUS and Web of Science and PsycINFO databases. For the grey literature search: Google Scholar, Theses and Dissertations Bank of the Brazilian Institute of Information in Science and Technology, Federated Network of Institutional Repositories of Scientific Publications, Online Theses Electronic Service, Open Access Scientific Repository of Portugal, Academic Archive Online. Searches and application of selection criteria will be performed according to the PRISMA-ScR flow approach. The articles will be selected from the Web Rayyan platform, the results will be analysed and presented in a narrative and thematically organised way. This scoping review is expected to contribute to the feasibility of assessing the quality of ACCRO services.

ETHICS AND DISSEMINATION

Ethics approval is not required. Results will be disseminated through professional networks, conference presentations and publication in a scientific journal. This protocol has been registered with OSF (https://osf.io/sp5df).

Systematic review of guidance for the collection and use of patient-reported outcomes in real-world evidence generation to support regulation, reimbursement and health policy

Background

Real-world evidence (RWE) plays an increasingly important role within global regulatory and reimbursement processes. RWE generation can be enhanced by the collection and use of patient-reported outcomes (PROs), which can provide valuable information on the effectiveness, safety, and tolerability of health interventions from the patient perspective. This systematic review aims to examine and summarise the available PRO-specific recommendations and guidance for RWE generation.

Methods and findings

Medical Literature Analysis and Retrieval System Online, Excerpta Medica Database, and websites of selected organisations were systematically searched to identify relevant publications. 1,249 articles were screened of which 7 papers met the eligibility criteria and were included in the review. The included publications provided PRO-specific recommendations to facilitate the use of PROs for RWE generation and these were extracted and grouped into eight major categories. These included: (1) instrument selection, (2) participation and engagement, (3) burden to health care professionals and patients, (4) stakeholder collaboration, (5) education and training, (6) PRO implementation process, (7) data collection and management, and (8) data analysis and presentation of results. The main limitation of the study was the potential exclusion of relevant publications, due to poor indexing of the databases and websites searched.

Conclusions

PROs may provide valuable and crucial patient input in RWE generation. Whilst valuable insights can be gained from guidance for use of PROs in clinical care, there is a lack of international guidance specific to RWE generation in the context of use for regulatory decision-making, reimbursement, and health policy. Clear and appropriate evidence-based guidance is required to maximise the potential benefits of implementing PROs for RWE generation. Unique aspects between PRO guidance for clinical care and other purposes should be differentiated. The needs of various stakeholder groups (including patients, health care professionals, regulators, payers, and industry) should be considered when developing future guidelines.

Supplementary Information

The online version contains supplementary material available at 10.1186/s41687-022-00466-7.

Development and validation of search filters to identify articles on deprescribing in Medline and Embase

Background

Deprescribing literature has been increasing rapidly. Our aim was to develop and validate search filters to identify articles on deprescribing in Medline via PubMed and in Embase via Embase.com.

Methods

Articles published from 2011 to 2020 in a core set of eight journals (covering fields of interest for deprescribing, such as geriatrics, pharmacology and primary care) formed a reference set. Each article was screened independently in duplicate and classified as relevant or non-relevant to deprescribing. Relevant terms were identified by term frequency analysis in a 70% subset of the reference set. Selected title and abstract terms, MeSH terms and Emtree terms were combined to develop two highly sensitive filters for Medline via Pubmed and Embase via Embase.com. The filters were validated against the remaining 30% of the reference set. Sensitivity, specificity and precision were calculated with their 95% confidence intervals (95% CI).

Results

A total of 23,741 articles were aggregated in the reference set, and 224 were classified as relevant to deprescribing. A total of 34 terms and 4 MeSH terms were identified to develop the Medline search filter. A total of 27 terms and 6 Emtree terms were identified to develop the Embase search filter. The sensitivity was 92% (95% CI: 83–97%) in Medline via Pubmed and 91% (95% CI: 82–96%) in Embase via Embase.com.

Conclusions

These are the first deprescribing search filters that have been developed objectively and validated. These filters can be used in search strategies for future deprescribing reviews. Further prospective studies are needed to assess their effectiveness and efficiency when used in systematic reviews.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01515-x.

"It has no meaning to me." How do researchers understand the effectiveness of literature searches? A qualitative analysis and preliminary typology of understandings

This study aimed to address the question: what does "effectiveness" mean to researchers in the context of literature searching for systematic reviews? We conducted a thematic analysis of responses to an e-mail survey. Eighty-nine study authors, whose studies met inclusion in a recent review (2018), were contacted via e-mail and asked three questions; one directly asking the question: in literature searching, what does effective (or effectiveness in) literature searching mean to you? Thirty-eight (46%) responses were received from diverse professional groups, including: literature searchers, systematic reviewers, clinicians and researchers. A shared understanding of what effectiveness means was not identified. Instead, five themes were developed from data: (a) effectiveness is described as a metric; (b) effectiveness is a balance between metrics; (c) effectiveness can be categorized by search purpose; (d) effectiveness is an outcome; and, (e) effectiveness is an experimental concept. We propose that these themes constitute a preliminary typology of understandings. No single definition of effectiveness was identified. The proposed typology suggests that different researchers have differing understandings of effectiveness. This could lead to uncertainty as to the aim and the purpose of literature searches and confusion about the outcomes. The typology offers a potential route for further exploration.

Outcome measurement instruments for erythema associated with incontinence-associated dermatitis: Systematic review

AIM

To: (a) examine which outcome measurement instruments for erythema associated with incontinence-associated dermatitis (IAD) with supporting evidence about measurement properties are available; (b) evaluate the methodological quality of the studies and the quality of the measurement properties; and (c) identify eligible instruments to measure erythema in incontinence-associated dermatitis research.

DESIGN

Systematic review.

DATA SOURCES

MEDLINE, EMBASE, CINAHL and CENTRAL were systematically searched until July 2018 (update December 2018). Additional input was gathered from 151 incontinence-associated dermatitis experts. Cited and citing references of included studies were screened.

REVIEW METHODS

The COSMIN Risk of Bias checklist was applied to evaluate the methodological quality of the studies. Reported measurement properties were rated against criteria for good measurement properties.

RESULTS

Fourteen studies, describing 10 measurement instruments, were included. In five instruments, erythema was captured as a separate concept, two studies provided empirical evidence about the measurement properties. The most studied measurement properties were reliability (9 studies), measurement error (4 studies) and criterion validity (4 studies). In one study, internal consistency was examined.

CONCLUSION

No instrument measuring exclusively erythema associated with incontinence-associated dermatitis exists. There is no single composite incontinence-associated dermatitis measurement instrument that outperforms others. Development or adaption of an instrument to measure erythema associated with incontinence-associated dermatitis is one option to solve this challenge.

IMPACT

The evidence about measurement properties of instruments measuring erythema associated with incontinence-associated dermatitis has not been summarized to date. The lack of an instrument should trigger activities to measure this domain accurately in future clinical trials.

Interventions for cancer-related fatigue: a scoping review

Cancer-related fatigue (CRF) is common and can be distressing for some survivors. There is increasing interest in measuring levels of CRF, highlighting its impact on quality of life. This review describes the nature and scope of evidence relating to interventions for CRF. Scoping review methodology was used to identify studies, extract data, collate and summarise results. Data were collated according to cancer tumour streams, stage of illness and the types of trial interventions. A total of 447 trials and 37 systematic reviews met the inclusion criteria. Nine papers reported longitudinal results. Populations studied were predominantly of mixed cancer diagnoses and breast cancer. The most frequent interventions were exercise, pharmacological, psycho-education and mind-body interventions. Fatigue was identified as a primary outcome measure (OM) in 58% of studies, with 58 different fatigue measures reported. Emerging evidence exists for the effectiveness of fatigue interventions for some cancer types. More research on interventions with participants with the same cancer type and illness phase is needed. Measurement of severity and impact of CRF using fewer, robust OMs will permit comparisons across studies.

Patient-reported outcome use in oncology: a systematic review of the impact on patient-clinician communication

PURPOSE

Patient-reported outcomes (PROs) are an increasingly popular tool to optimize care and bridge the gap between patient experience and clinician understanding. The aim of this review was to identify mechanisms through which PROs facilitate patient-clinician communication in the adult oncology population.

METHODS

We conducted a systematic review of the published literature using the following data sources: MEDLINE, EMBASE, CINAHL, PsycINFO, Cab Direct, and CDSR. Studies included in this review reported on the outcomes of PRO use, used PROs as an intervention and not as a study outcome measurement tool, included cancer patients or survivors as study participants, and analyzed patient-clinician communication.

RESULTS

We identified 610 unique records, of which 43 publications met the inclusion and exclusion criteria. Synthesis of the reviewed studies provided evidence of the usefulness of PROs in facilitating patient-clinician communication on a variety of topics. We identified mechanisms though which PROs influenced patient-clinician communication to include increasing symptom awareness, prompting discussion, streamlining consultations, and facilitating inter-professional communication. Barriers to PRO use in communication improvement include technical problems impeding its administration and completion, compliance issues due to lack of incentive or forgetfulness, and use of PROs that do not appropriately assess issues relevant to the patient. Facilitators include increased education on PRO use, using PRO tools that patients find more acceptable, and providing patient data summaries in an easily accessible format for clinicians.

CONCLUSIONS

Our review suggests that PROs facilitate patient-clinician communication through various mechanisms that could perhaps contribute to improvements in symptom management and survival. The impact of PROs on clinical outcomes, however, remains poorly studied.

Patient-reported outcome measures in advanced musculoskeletal physiotherapy practice: a systematic review

OBJECTIVE

Advanced practice physiotherapists (APPs), also known as extended scope physiotherapists, provide a new model of service delivery for musculoskeletal (MSK) disorders. Research to date has largely focused on health service efficiencies, with less emphasis on patient outcomes. The present systematic review aimed to identify the patient-reported outcome measures (PROMs) being utilized by APPs.

METHOD

A wide search strategy was employed, including the PubMed, Embase, CINAHL, CENTRAL and PEDro databases, to identify studies relating to PROMs utilized by APPs in MSK healthcare settings. PROMs identified were classified into predetermined outcome domains, with additional contextual data extracted.

RESULTS

Of the initial 12,302 studies, 38 met the inclusion criteria. These involved APPs across different settings, utilizing 72 different PROMs and most commonly capturing: Patient Satisfaction, Quality of Life (QoL), Functional Status, and Pain; and, less frequently: Global Status (i.e. overall improvement), Psychological Well-Being, Work ability, and Healthcare Consumption and Costs. The quality of the PROMs varied greatly, with Satisfaction most commonly measured utilizing non-standardized locally-devised tools; the EuroQol five-dimensions questionnaire (EuroQoL-5D) and 36-Item Short-Form (SF-36) cited most frequently to capture QoL; and the Visual Analogue Scale (VAS) to capture Pain. No key measure was identified to capture Functional Status, with 15 different tools utilized.

CONCLUSION

APPs utilized a multiplicity of PROMs across a range of MSK disorders. The present review will act as an important resource, informing the selection of outcomes for MSK disorders, with a view to greater standardization of outcome measurement in MSK clinical practice, service evaluation and research.

Health-related quality-of-life outcome measures in paediatric palliative care: A systematic review of psychometric properties and feasibility of use

BACKGROUND

The number of children worldwide requiring palliative care services is increasing due to advances in medical care and technology. The use of outcome measures is important to improve the quality and effectiveness of care.

AIM

To systematically identify health-related quality-of-life outcome measures that could be used in paediatric palliative care and examine their feasibility of use and psychometric properties.

DESIGN

A systematic literature review and analysis of psychometric properties.

DATA SOURCES

PsychInfo, Medline and EMBASE were searched from 1 January 1990 to 10 December 2014. Hand searches of the reference list of included studies and relevant reviews were also performed.

RESULTS

From 3460 articles, 125 papers were selected for full-text assessment. A total of 41 articles met the eligibility criteria and examined the psychometric properties of 22 health-related quality-of-life measures. Evidence was limited as at least half of the information on psychometric properties per instrument was missing. Measurement error was not analysed in any of the included articles and responsiveness was only analysed in one study. The methodological quality of included studies varied greatly.

CONCLUSION

There is currently no 'ideal' outcome assessment measure for use in paediatric palliative care. The domains of generic health-related quality-of-life measures are not relevant to all children receiving palliative care and some domains within disease-specific measures are only relevant for that specific population. Potential solutions include adapting an existing measure or developing more individualized patient-centred outcome and experience measures. Either way, it is important to continue work on outcome measurement in this field.

How to prepare a systematic review of economic evaluations for informing evidence-based healthcare decisions: a five-step approach (part 1/3)

INTRODUCTION

Systematic reviews of economic evaluations are useful for synthesizing economic evidence about health interventions and for informing evidence-based decisions. Areas covered: As there is no detailed description of the methods for performing a systematic review of economic evidence, this paper aims to provide an overview of state-of-the-art methodology. This is laid out in a 5-step approach, as follows: step 1) initiating a systematic review; step 2) identifying (full) economic evaluations; step 3) data extraction, risk of bias and transferability assessment; step 4) reporting results; step 5) discussion and interpretation of findings. Expert commentary: The paper aims to help inexperienced reviewers and clinical practice guideline developers, but also to be a resource for experts in the field who want to check on current methodological developments.

A checklist to assess database-hosting platforms for designing and running searches for systematic reviews

BACKGROUND

Systematic reviews require literature searches that are precise, sensitive and often complex. Database-hosting platforms need to facilitate this type of searching in order to minimise errors and the risk of bias in the results.

OBJECTIVES

The main objective of the study was to create a generic checklist of criteria to assess the ability of host platforms to cope with complex searching, for example, for systematic reviews, and to test the checklist against three host platforms (EBSCOhost, OvidSP and ProQuest).

METHOD

The checklist was developed as usual review work was carried out and through discussion between the two authors. Attributes on the checklist were designated as 'desirable' or 'essential'. The authors tested the checklist independently against three host platforms and graded their performance from 1 (insufficient) to 3 (performs well).

RESULTS

Fifty-five desirable or essential attributes were identified for the checklist. None of the platforms performed well for all of the attributes on the checklist.

CONCLUSIONS

Not all database-hosting platforms are designed for complex searching. Librarians and other decision-makers who work in health research settings need to be aware of the different limitations of host platforms for complex searching when they are making purchasing decisions or training others.

Core Outcome Measures in Effectiveness Trials (COMET) initiative: protocol for an international Delphi study to achieve consensus on how to select outcome measurement instruments for outcomes included in a 'core outcome set'

BACKGROUND

The Core Outcome Measures in Effectiveness Trials (COMET) initiative aims to facilitate the development and application of 'core outcome sets' (COS). A COS is an agreed minimum set of outcomes that should be measured and reported in all clinical trials of a specific disease or trial population. The overall aim of the Core Outcome Measurement Instrument Selection (COMIS) project is to develop a guideline on how to select outcome measurement instruments for outcomes included in a COS. As part of this project, we describe our current efforts to achieve a consensus on the methods for selecting outcome measurement instruments for outcomes to be included in a COS.

METHODS/DESIGN

A Delphi study is being performed by a panel of international experts representing diverse stakeholders with the intention that this will result in a guideline for outcome measurement instrument selection. Informed by a literature review, a Delphi questionnaire was developed to identify potentially relevant tasks on instrument selection. The Delphi study takes place in a series of rounds. In the first round, panelists were asked to rate the importance of different tasks in the selection of outcome measurement instruments. They were encouraged to justify their choices and to add other relevant tasks. Consensus was reached if at least 70% of the panelists considered a task 'highly recommended' or 'desirable' and if no opposing arguments were provided. These tasks will be included in the guideline. Tasks that at least 50% of the panelists considered 'not relevant' will be excluded from the guideline. Tasks that were indeterminate will be taken to the second round. All responses of the first round are currently being aggregated and will be fed back to panelists in the second round. A third round will only be performed if the results of the second round require it.

DISCUSSION

Since the Delphi method allows a large group of international experts to participate, we consider it to be the preferred consensus-based method for our study. Based upon this consultation process, a guideline will be developed on instrument selection for outcomes to be included in a COS.

Comparison of the efficacy of three PubMed search filters in finding randomized controlled trials to answer clinical questions

OBJECTIVE

The aim of this study was to compare the performance of three search methods in the retrieval of relevant clinical trials from PubMed to answer specific clinical questions.

METHODS

Included studies of a sample of 100 Cochrane reviews which recorded in PubMed were considered as the reference standard. The search queries were formulated based on the systematic review titles. Precision, recall and number of retrieved records for limiting the results to clinical trial publication type, and using sensitive and specific clinical queries filters were compared. The number of keywords, presence of specific names of intervention and syndrome in the search keywords were used in a model to predict the recalls and precisions.

RESULTS

The Clinical queries-sensitive search strategy retrieved the largest number of records (33) and had the highest recall (41.6%) and lowest precision (4.8%). The presence of specific intervention name was the only significant predictor of all recalls and precisions (P = 0.016).

CONCLUSION

The recall and precision of combination of simple clinical search queries and methodological search filters to find clinical trials in various subjects were considerably low. The limit field strategy yielded in higher precision and fewer retrieved records and approximately similar recall, compared with the clinical queries-sensitive strategy. Presence of specific intervention name in the search keywords increased both recall and precision.

Literature searching for social science systematic reviews: consideration of a range of search techniques

BACKGROUND

Literature for a systematic review on the student experience of e-learning is located across a range of subject areas including health, education, social science, library and information science.

OBJECTIVES

To assess the merits and shortcomings of using different search techniques in retrieval of evidence in the social science literature.

METHODS

A conventional subject search was undertaken as the principal method of identifying the literature for the review. Four supplementary search methods were used including citation searching, reference list checking, contact with experts and pearl growing.

RESULTS

The conventional subject search identified 30 of 41 included references; retrieved from 10 different databases. References were missed by this method and a further 11 references were identified via citation searching, reference list checking and contact with experts. Pearl growing was suspended as the nominated pearls were dispersed across numerous databases, with no single database indexing more than four pearls.

CONCLUSIONS

Searching within the social sciences literature requires careful consideration. Conventional subject searching identified the majority of references, but additional search techniques were essential and located further high quality references.

Evaluation of five search strategies in retrieving qualitative patient-reported electronic data on the impact of pressure ulcers on quality of life

AIM

This paper is a report of a study conducted to compare the effectiveness of qualitative methodology search strategies with subject-specific (health-related quality of life) search strategies in the retrieval of qualitative patient-reported data of the impact of pressure ulcers on health-related quality of life.

BACKGROUND

Methods to locate qualitative patient-reported health-related quality of life research data electronically have undergone little replication and validation. A major problem in searching for this type of data is that it is reported in accounts of both primary qualitative research as well as mixed methods research.

DATA SOURCES

We combined five search strategies with terms for pressure ulcer and searched seven electronic databases from inception to October 2007.

METHODS

The sensitivity, specificity, precision and accuracy for each search strategy were assessed.

RESULTS

A subject-specific (health-related quality of life) search strategy, developed by us, had a high yield (100% sensitivity), but low specificity (<50%). The research methodology-based strategies had lower yields (sensitivity 72-83%) but high specificity (79-83%). Importantly, subject-specific search strategies identified all studies reporting qualitative patient-reported health-related quality of life data, whereas, research methodology-based strategies did not identify qualitative data reported in mixed method studies, making subject-based strategies more effective in retrieving qualitative patient-reported health-related quality of life research.

CONCLUSION

An important consideration in the health-related quality of life field is that qualitative data are reported in both qualitative and mixed methodology research and searching for this type data involves trade-offs between yield, sensitivity and specificity. Accurate indexing of subject-specific outcomes and methodology used in electronic databases and publications is also needed.

Development of a methodological PubMed search filter for finding studies on measurement properties of measurement instruments

OBJECTIVES

For the measurement of patient-reported outcomes, such as (health-related) quality of life, often many measurement instruments exist that intend to measure the same construct. To facilitate instrument selection, our aim was to develop a highly sensitive search filter for finding studies on measurement properties of measurement instruments in PubMed and a more precise search filter that needs less abstracts to be screened, but at a higher risk of missing relevant studies.

METHODS

A random sample of 10,000 PubMed records (01-01-1990 to 31-12-2006) was used as a gold standard. Studies on measurement properties were identified using an exclusion filter and hand searching. Search terms were selected from the relevant records in the gold standard as well as from 100 systematic reviews of measurement properties and combined based on sensitivity and precision. The performance of the filters was tested in the gold standard as well as in two validation sets, by calculating sensitivity, precision, specificity, and number needed to read.

RESULTS

We identified 116 studies on measurement properties in the gold standard. The sensitive search filter was able to retrieve 113 of these 116 studies (sensitivity 97.4%, precision 4.4%). The precise search filter had a sensitivity of 93.1% and a precision of 9.4%. Both filters performed very well in the validation sets.

CONCLUSION

The use of these search filters will contribute to evidence-based selection of measurement instruments in all medical fields.

Adequacy of health-related quality of life measures in children under 5 years old: systematic review

AIM

This paper is a report of a systematic review to assess the adequacy of measures evaluating generic health-related quality of life outcomes in children less than 5 years old.

BACKGROUND

Evaluating generic health-related quality of life outcomes in children is important to assess the effectiveness of interventions, nursing care and services.

METHODS

A methodological systematic review was carried out for the period 1980-2005 in accord with the UK Centre of Reviews and Dissemination guidelines on systematic reviews. We searched multiple electronic databases, hand-searched key texts, and contacted investigators. We included all English language publications describing primary empirical research of generic health-related quality of life, health status, functional status or wellbeing measures, with published psychometric evidence. All measures (proxy and/or self-complete) for use in children under 5 years were included; single dimension measures were excluded.

RESULTS

Seventy-six papers (70 studies) referring to 16 generic health-related quality of life measures met our inclusion criteria. None of the 16 measures were adequate in terms of their conceptual content or psychometric criteria; quality scores were poor (0-8). No current, generic health-related quality of life measure is both psychometrically and conceptually robust, although the Health Utilities Index has the most comprehensive psychometric data published. However, not all dimensions of health are assessed, with little evidence of reliability in children under 5 years of age.

CONCLUSION

There is a need to develop empirically robust and conceptually comprehensive health-related quality of life measures, particularly in the context of proxy-completion measures for very young children.

Effectiveness and efficiency of a literature search strategy to answer questions on the etiology of occupational diseases: a controlled trial

OBJECTIVES

To evaluate the effectiveness and efficiency of a search strategy to find evidence-based answers to questions related to the possible occupational etiology of diseases.

METHODS

A controlled trial of 70 occupational health physicians and 55 insurance physicians who were asked to answer one out of four 'occupational disease case-vignettes' following the steps of evidence-based medicine (EBM). The intervention group were given the search strategy as a tool.

RESULTS

The intervention group scored significantly better than the control group in answering the main question of the case-vignette correctly (57% versus 37%) using more adequate search terms. The intervention group scored significantly better regarding satisfaction with the applied search strategy (28% very satisfied versus 8%). We found no differences in time spent in solving the case or in the intention of future practice of EBM.

CONCLUSIONS

The introduction and application of specific search strategies can have a positive effect on the effectiveness of searching literature. Future initiatives for developing and testing specific search strategies in the field of occupational health should be encouraged.

Populating decision-analytic models: The feasibility and efficiency of database searching for individual parameters

Objectives: The aim of the study was to investigate the feasibility and effectiveness of searching selected databases to identify information required to populate a decision-analytic model.

Methods: Methods of searching for information to populate a decision-analytic model were piloted using a case study of prophylactic antibiotics to prevent recurrent urinary tract infections in children. This study explored how the information requirements for a decision-analytic model could be developed into searchable questions and how search strategies could be derived to answer these questions. The study also assessed the usefulness of three published search filters and explored which resources might produce relevant information for the various model parameters.

Results: Based on the data requirements for this case study, 42 questions were developed for searching. These questions related to baseline event rates, health-related quality of life and outcomes, relative treatment effects, resource use and unit costs, and antibiotic resistance. A total of 1,237 records were assessed by the modeler, and of these, 48 were found to be relevant to the model. Search precision ranged from 0 percent to 38 percent, and no single database proved the most useful for all the questions.

Conclusions: The process of conducting specific searches to address each of the model questions provided information that was useful in populating the case study model. The most appropriate resources to search were dependent on the question, and multiple database searching using focused search strategies may prove more effective in finding relevant data than thorough searches of a single database.

Optimal search strategies for detecting health services research studies in MEDLINE

BACKGROUND

Evidence from health services research (HSR) is currently thinly spread through many journals, making it difficult for health services researchers, managers and policy-makers to find research on clinical practice guidelines and the appropriateness, process, outcomes, cost and economics of health care services. We undertook to develop and test search terms to retrieve from the MEDLINE database HSR articles meeting minimum quality standards.

METHODS

The retrieval performance of 7445 methodologic search terms and phrases in MEDLINE (the test) were compared with a hand search of the literature (the gold standard) for each issue of 68 journal titles for the year 2000 (a total of 25,936 articles). We determined sensitivity, specificity and precision (the positive predictive value) of the MEDLINE search strategies.

RESULTS

A majority of the articles that were classified as outcome assessment, but fewer than half of those in the other categories, were considered methodologically acceptable (no methodologic criteria were applied for cost studies). Combining individual search terms to maximize sensitivity, while keeping specificity at 50% or more, led to sensitivities in the range of 88.1% to 100% for several categories (specificities ranged from 52.9% to 97.4%). When terms were combined to maximize specificity while keeping sensitivity at 50% or more, specificities of 88.8% to 99.8% were achieved. When terms were combined to maximize sensitivity and specificity while minimizing the differences between the 2 measurements, most strategies for HSR categories achieved sensitivity and specificity of at least 80%.

INTERPRETATION

Sensitive and specific search strategies were validated for retrieval of HSR literature from MEDLINE. These strategies have been made available for public use by the US National Library of Medicine at www.nlm.nih.gov/nichsr/hedges/search.html.

Evaluation of methodological search filters--a review

OBJECTIVES

The objectives of the study were: to provide an overview of approaches to methodological search filter development; to identify and critically review the stages of methodological search filter development; to devise a search filter appraisal checklist based on the review.

METHODS

An iterative approach to searching was employed utilizing health and library databases, the world wide web and citation searching. Further systematic methods included hand searching of key journals in the field of search filter development, contacting known experts in the field and scanning reference lists of relevant papers to identify additional studies. Altogether, 51 potentially relevant papers were found, of which 20 met the inclusion criteria.

RESULTS

Four stages of search filter development were identified from the literature (search term selection, identification of a gold standard, evaluation and validation). Variations in the methods used to approach these four stages were identified, most importantly in the extent to which search filters are tested and validated.

CONCLUSION

Awareness of the process and limitations involved in search filter development is essential to make an informed decision on the applicability and validity of search filters. The findings of this review indicate a considerable agenda for future research, in particular, to improve the quality of reporting of search filters and to inform users on their use and application. Based on the review, guidance in the appraisal process of search filters is given in the form of a checklist.

How does your searching grow? A survey of search preferences and the use of optimal search strategies in the identification of qualitative research

The objective was to gain an overview of researchers experiences of searching the literature, with particular reference to the use of optimal search strategies (OSSs) and searching for qualitative research studies. A 13-item semi-structured questionnaire investigating search behaviour was distributed to members of the Cochrane Qualitative Methods Network. Follow-up interviews were conducted with a subset of respondents to explore issues raised and clarify points of ambiguity. Findings were analysed using data reduction, data displays and verification techniques. Eighty-six per cent of distributed questionnaires were returned. All respondents reported searching electronic databases as part of their literature search, with 80% expressing a preference for searching alone or with colleagues. Forty-one per cent indicated that they consider a database search to be only one aspect of a comprehensive literature search. The rigour and availability of OSSs was a concern for 30% of respondents. Twenty-five per cent of respondents had searched for qualitative studies, although the difficulty of locating this type of literature was considered problematic because of the varied use of the term 'qualitative'. Whilst the majority of respondents reported using OSSs in some capacity, reservations were expressed about their ability to facilitate a comprehensive search. Replies indicated a belief that OSSs can reduce the sensitivity of a search, and might limit the breadth of coverage required. A greater appreciation of the availability and purpose of OSSs-including the ability to optimize either sensitivity or recall-is needed if this enhanced approach to accurate data retrieval and potential improvement in time management is to become widespread.

Evaluation of outcomes in community-acquired pneumonia: a guide for patients, physicians, and policy-makers

Community-acquired pneumonia (CAP) is a key target for research and quality improvement in acute medicine. However, many of the outcome measures used in prognostic and antibiotic studies are not validated and do not capture features of outcome that are important to patients. Substitutes for traditional outcome measures include a recently validated patient-based symptom questionnaire (the CAP-Sym) and process-of-care measures. The interpretation of outcomes also depends on the quality of the study design and methods used. This paper discusses the advantages and disadvantages of outcome, process-of-care, and economic measures in CAP and the interpretation of these measures in randomised and observational studies. A core set of measures for use in clinical CAP research and performance measurement is proposed.

BEME Guide no 3: systematic searching for evidence in medical education--Part 1: Sources of information

Searching for evidence to inform best practice in medical education is a complex undertaking. With very few information sources dedicated to medical education itself, one is forced to consult a wide range of often enormous sources--and these are dedicated to either medicine or education, making a medical education search all the more challenging. This guide provides a comprehensive overview of relevant information sources and methods (including bibliographic databases, grey literature, hand searching and the Internet) and describes when they should be consulted. The process of constructing a search is explained: identifying and combining core concepts, using Boolean algebra and search syntax, limiting results sets, and making best use of databases' controlled vocabularies. This process is illustrated with images from search screens and is followed by numerous examples designed to reinforce skills and concepts covered. The guide has been developed from the ongoing experience gained from the systematic searches conducted for the Best Evidence Medical Education Collaboration, and concludes by looking ahead to initiatives that will shape future searching for medical education evidence.

Comparison of bibliographic databases for information on the rehabilitation of people with severe mental illness

OBJECTIVE

The research sought to examine the overlap in coverage between several health-related databases, thus enabling the identification of the most important sources for searching for information on the rehabilitation of people with severe mental illness.

METHODS

The literature was searched within a systematic review. Several health-related databases were retrieved (Cumulative Index to Nursing and Allied Health Literature, The Cochrane Library, MEDLINE, PsycLIT, Sociofile, and Social Science Citation Index), noting their source and comparing results retrieved from each database.

FINDINGS

The total number of studies retrieved from each database varied. Almost a third of the papers retrieved from each database were unique to that source. Forty-two percent of the papers were only found in one database. Restricting a search to one database alone would miss many papers and could affect the results of a systematic review. PsycLIT was the most useful database for this topic area, containing 44% of the papers. MEDLINE, the database of first choice for many health professionals, held only 29%.

CONCLUSIONS

No database was determined to be significantly more useful than any other--each warranted inclusion in the study. Reliance cannot be placed on one database alone, and other methods such as hand searching should also be used. Although this may not be new information for information professionals, it is likely to be new for health professionals and researchers who are increasingly performing their own literature searches. Information professionals have an important role to play in conveying this message to those outside their profession.

Selecting a mental health needs assessment scale: guidance on the critical appraisal of standardized measures

The assessment of need has become a central element of the NHS policy on mental health care. Despite the importance placed on a needs-led mental health service, little guidance has been provided as to the most appropriate means of assessing need. The use of standardized instruments can facilitate a systematic approach to the needs assessment process. However, available mental health needs assessment scales (MHNAS) vary considerably. Care is required to ensure that the scale selected is appropriate to the proposed purpose. This paper presents practical guidance on instrument selection for potential users of MHNAS including: (1) a review of the key issues to be considered when selecting an assessment scale, (2) a critical appraisal checklist for use in the review of needs assessment scales, and (3) a comparative analysis of three of the most commonly cited needs assessment scales. In addition, recommendations are made of ways to improve both the accessibility of MNHAS and to facilitate their appraisal prior to application in a clinical or research setting.