Publication bias in qualitative research: what becomes of qualitative research presented at conferences?

BACKGROUND

Less than half of studies presented at conferences remain unpublished two years later, and these studies differ systematically from those that are published. In particular, the unpublished studies are less likely to report statistically significant findings, and this introduces publication bias. This has been well documented for quantitative studies, but has never been explored in relation to qualitative research.

METHODS

We reviewed the abstracts of qualitative research presented at the 1998 (n = 110) and 1999 (n = 114) British Sociological Association (BSA) Medical Sociology meetings, and attempted to locate those studies in databases or by contacting authors. We also appraised the quality of reporting in each abstract.

RESULTS

We found an overall publication rate for these qualitative studies of 44.2%. This is nearly identical to the publication rate for quantitative research. The quality of reporting of study methods and findings in the abstract was positively related to the likelihood of publication.

CONCLUSION

Qualitative research is as likely to remain unpublished as quantitative research. Moreover, non-publication appears to be related to the quality of reporting of methodological information in the original abstract, perhaps because this is a proxy for a study with clear objectives and clear findings. This suggests a mechanism by which "qualitative publication bias" might work: qualitative studies that do not show clear, or striking, or easily described findings may simply disappear from view. One implication of this is that, as with quantitative research, systematic reviews of qualitative studies may be biased if they rely only on published papers.

The psychosocial and health effects of workplace reorganisation. 1. A systematic review of organisational-level interventions that aim to increase employee control

OBJECTIVE

Systematic review of the health and psychosocial effects of increasing employee participation and control through workplace reorganisation, with reference to the "demand-control-support" model of workplace health.

DESIGN

Systematic review (QUORUM) of experimental and quasi-experimental studies (any language) reporting health and psychosocial effects of such interventions.

DATA SOURCES

Electronic databases (medical, social science and economic), bibliographies and expert contacts.

RESULTS

We identified 18 studies, 12 with control/comparison groups (no randomised controlled trials). Eight controlled and three uncontrolled studies found some evidence of health benefits (especially beneficial effects on mental health, including reduction in anxiety and depression) when employee control improved or (less consistently) demands decreased or support increased. Some effects may have been short term or influenced by concurrent interventions. Two studies of participatory interventions occurring alongside redundancies reported worsening employee health.

CONCLUSIONS

This systematic review identified evidence suggesting that some organisational-level participation interventions may benefit employee health, as predicted by the demand-control-support model, but may not protect employees from generally poor working conditions. More investigation of the relative impacts of different interventions, implementation and the distribution of effects across the socioeconomic spectrum is required.

The health impacts of housing-led regeneration: a prospective controlled study

STUDY OBJECTIVE

To evaluate self-reported changes in housing quality and health associated with housing-led area regeneration.

DESIGN

A prospective study over 1 year using structured interviews with 50 households who moved to new housing and with 50 matched controls who did not move.

SETTING AND PARTICIPANTS

Residents of two social rented housing schemes in the West of Scotland.

RESULTS

Small but not statistically significant increases in levels of "excellent" or "good" self-reported health status were found in both groups. Both intervention and control groups experienced reductions in problems related to warmth, but no significant change in how they felt about their house.

CONCLUSIONS

It is feasible to conduct prospective controlled studies to evaluate the health effects of housing improvement using matched control groups. The absence of marked improvement in health after moving to new housing might be due to the small sample size or to the limited potential to improve health through this intervention alone.

Housing and health

Heating improvements may hold most promise for developing healthy housing policy

Randomized clinical trial of the effect of quilting latissimus dorsi flap donor site on seroma formation

Background

Latissimus dorsi (LD) flap breast reconstruction is associated with a high incidence of donor site seromas, despite the use of surgical drains. The aim of this study was to evaluate the use of donor site quilting sutures, as well as drains, on the incidence, volume and frequency of seroma aspiration.

Methods

The trial randomized 108 women undergoing LD breast reconstruction to quilting procedures (54) or control group (52) for intention-to-treat analysis; two were excluded. Outcome measures were the incidence and volume of postoperative seroma. Secondary outcome measures included postoperative back pain, analgesic consumption, shoulder movement and duration of hospital stay.

Results

Quilting significantly reduced the overall incidence of seroma from 46 of 48 (96 per cent) to 43 of 52 (83 per cent) (P = 0·036), including the 38 women who had extended LD flap (with or without implants). There were further significant reductions in seroma volume (P = 0·004), frequency of aspiration (P = 0·001) and overall seroma volumes, including surgical drainage and symptomatic seromas (P = 0·013). Subset analyses for LD–implant (60 women) and extended LD (with or without implant) showed similar significance. Quilting did not affect back pain or compromise shoulder mobility.

Conclusion

Quilting significantly reduced overall seroma volumes after LD breast reconstruction including extended LD, and is recommended in combination with surgical drains.

Do urban regeneration programmes improve public health and reduce health inequalities? A synthesis of the evidence from UK policy and practice (1980-2004)

OBJECTIVES

To synthesise data on the impact on health and key socioeconomic determinants of health and health inequalities reported in evaluations of national UK regeneration programmes.

DATA SOURCES

Eight electronic databases were searched from 1980 to 2004 (IBSS, COPAC, HMIC, IDOX, INSIDE, Medline, Urbadisc/Accompline, Web of Knowledge). Bibliographies of located documents and relevant web sites were searched. Experts and government departmental libraries were also contacted.

REVIEW METHODS

Evaluations that reported achievements drawing on data from at least two target areas of a national urban regeneration programme in the UK were included. Process evaluations and evaluations reporting only business outcomes were excluded. All methods of evaluation were included. Impact data on direct health outcomes and direct measures of socioeconomic determinants of health were narratively synthesised.

RESULTS

19 evaluations reported impacts on health or socioeconomic determinants of health; data from 10 evaluations were synthesised. Three evaluations reported health impacts; in one evaluation three of four measures of self reported health deteriorated, typically by around 4%. Two other evaluations reported overall reductions in mortality rates. Most socioeconomic outcomes assessed showed an overall improvement after regeneration investment; however, the effect size was often similar to national trends. In addition, some evaluations reported adverse impacts.

CONCLUSION

There is little evidence of the impact of national urban regeneration investment on socioeconomic or health outcomes. Where impacts have been assessed, these are often small and positive but adverse impacts have also occurred. Impact data from future evaluations are required to inform healthy public policy; in the meantime work to exploit and synthesise "best available" data is required.

Venous Ulcer Services in the United Kingdom

OBJECTIVES

To obtain comprehensive information about venous ulcer services throughout the United Kingdom (UK).

DESIGN

Questionnaire based survey.

MATERIALS

Questionnaire.

METHODS

Letters about venous ulcer services were sent to consultant vascular surgeons in all areas of the UK (total 181). Questionnaires were then directed to the appropriate clinician in each area.

RESULTS

Responses were received from 177 (98%) areas. Fifteen (8%) had no dedicated service. Completed questionnaires were returned for 112 (63%) areas. Fifty-six (54%) services were managed by acute hospitals, 29 (28%) by primary care (community) and 19 (18%) jointly. Doctors supervised services in 65 (64%) (vascular surgeons 49, dermatologists 12, both 4) and nurses in 31 (33%). New referrals per week were 1-50 based on audit (33%) or estimates (67%). Sixty-three (58%) services had no database. Written guidelines existed for 76% services (90% nurse-supervised and 64% doctor-supervised services--p<0.02).

CONCLUSIONS

Dedicated venous ulcer services have been shown to improve healing rates and quality of life. Although now present in most areas of the UK, their organisation varies considerably and many are not based in the community, near patients homes. This survey provides a benchmark for comparison with venous ulcer services in other countries.

Evaluation of the health effects of a neighbourhood traffic calming scheme

STUDY OBJECTIVE

To assess the secondary health impacts of a traffic calming scheme on a community.

METHODS

Prospective cohort study of a randomly selected sample of the local community using postal questionnaires and pedestrian counts on the affected road six months before and six months after the implementation of the scheme. The setting was a community in which a traffic calming scheme was built in the main road (2587 households). The Short Form 36 version 2 was included in the questionnaire and summary measures of physical health (physical component summary) and mental health (mental component summary) calculated. A random sample of 750 households was initially posted the pre-intervention questionnaire.

MAIN RESULTS

There were increases in observed pedestrian activity in the area after the introduction of the traffic calming scheme. Physical health improved significantly but mental health did not change. Traffic related problems improved, while other local nuisances were reported to be worse.

CONCLUSIONS

The introduction of a traffic calming scheme is associated with improvements in health and health related behaviours. It is feasible to prospectively evaluate broader health impacts of similar transport interventions although poor response rates may limit the validity of results.

What does GP out of hours care cost? An analysis of different models of out of hours care in Scotland

BACKGROUND

The changes to out of hours care provided by General Practitioners have led to wide variation in the types and costs of out of hours care across the country.

AIM

To examine the costs of different models of service delivery for GP out of hours organisations.

METHODS

This was a prospective cross-sectional survey of eight GP out of hours organisations and samples of their patients. A deputising service, rotas, and various types of GP co-operative across Scotland were surveyed. Information on the quantities of resources used by each organisation was combined with unit costs. Costs incurred by patients and other NHS costs subsequent to the out of hours contact were also calculated.

RESULTS

Annual costs incurred by the GP out of hours organisation per 1,000 population ranged from pounds sterling 2,916 to pounds sterling 12,120. There was no relationship between costs and type and size of organisation. There was a three-fold variation in total costs per out of hours contact (pounds sterling 15 to pounds sterling 51). Costs per phone contact were lowest (pounds sterling 6 to pounds sterling 11), followed by cost per centre contact (pounds sterling 10 to pounds sterling 16) and cost per home contact (pounds sterling 21 to pounds sterling 60). Total costs per episode ranged from pounds sterling 78 to pounds sterling 136 for centre contacts, from pounds sterling 130 to pounds sterling 303 for home contacts, and from pounds sterling 70 to pounds sterling 553 for telephone contacts. Home contacts had the highest average cost per episode (pounds sterling 212), followed by telephone contacts (pounds sterling 117) and centre contacts (pounds sterling 85).

CONCLUSIONS

There are wide variations in the costs of operating GP out of hours services, It is likely that the context in which organisations were set up and local geography infuence variations in costs, as well as the level of GP cover.

Modelling consultation rates in infancy: influence of maternal and infant characteristics, feeding type and consultation history

BACKGROUND

Severity of illness, sociodemographic factors, and breastfeeding have been identified as predictors of consultation rates in infants, and prescriptions for antibiotics have been found to increase future consultation rates in older children. The Baby Check trial (1996-1998) provided detailed information about consultations for 935 babies during their first 6 months.

AIMS

To investigate potential predictors of consultation rates in babies.

DESIGN OF STUDY

A 6-month cohort study of newborn babies originally enrolled into a randomised controlled trial. Maternal and infant characteristics were collected from hospital discharge records. Primary care consultation data for each baby were collected by case note review.

SETTING

Thirteen general practices in Glasgow.

METHOD

Multilevel models were used to analyse the number of consultations for each baby during its first 26 weeks, dependent upon the baby's age, the calendar month, maternal and infant characteristics, and previous consultations.

RESULTS

The strongest predictors of consultation rates were previous consultations, particularly during the preceding week. Breastfed babies and those with older mothers consulted less often. A multilevel model was better than a fixed effects model, with considerable variation in consultation rates between babies.

CONCLUSION

Infants' consultation rates over time can be analysed using multilevel models, if details of primary care consultations are available. These models can incorporate the effects of fixed variables and those that change during the follow-up period. Our findings add to previous research linking breastfeeding with reduced morbidity in infancy, and for that reason breastfeeding should continue to be promoted in primary care.

Evaluating the health effects of social interventions

Is no evidence better than any evidence when controlled studies are unethical?

Assessing the health impact of local amenities: a qualitative study of contrasting experiences of local swimming pool and leisure provision in two areas of Glasgow

STUDY OBJECTIVE

To assess the health impacts of local public swimming pool and leisure provision.

DESIGN

Retrospective qualitative study using focus groups. Reports from two areas with contrasting experience of provision of a public swimming pool (opening and closure) were compared within the context of general reports about health and neighbourhood.

SETTING

Two deprived neighbourhoods in south Glasgow.

PARTICIPANTS

Local adult residents of mixed ages, accessed through local community groups.

MAIN RESULTS

In both areas the swimming pool was reported as an important amenity that was linked to health and wellbeing. However, few residents reported regular use of the pool for physical activity. Use of the pool facility for social contact was directly linked to reports of relief of stress and isolation, and improved mental health. Pool closure was one in a series of amenity closures and area decline and was used to represent other area changes. Health impacts were strongly linked to the pool closure. The pool opening was associated with local area regeneration, similar but less prominent links between swimming pool provision and health were reported. Health benefits of social contact were diffuse and linked to other local amenities as well as the new pool facility.

CONCLUSIONS

Although theoretically linked to increased physical activity, the health benefits conveyed by the swimming pool may be more closely linked to the facilitation of social contact, and a supervised facility for young children. The use of qualitative work to investigate area based change provides rich contextual data to strengthen and explain the reported health impacts.

What are the most effective ways of improving population health through transport interventions? Evidence from systematic reviews

STUDY OBJECTIVE

To review systematic review literature that describes the effectiveness of transport interventions in improving population health.

METHODS

Systematic review methodology was used to evaluate published and unpublished systematic reviews in any language that described the measured health effects of any mode of transport intervention.

MAIN RESULTS

28 systematic reviews were identified. The highest quality reviews indicate that the most effective transport interventions to improve health are health promotion campaigns (to prevent childhood injuries, to increase bicycle and motorcycle helmet use, and to promote children's car seat and seatbelt use), traffic calming, and specific legislation against drink driving. Driver improvement and education courses are associated with increases in crash involvement and violations.

CONCLUSIONS

Systematic reviews are able to provide evidence about effective ways of improving health through transport related interventions and also identify well intentioned but harmful interventions. Valuable additional information may exist in primary studies and systematic reviews have a role in evaluating and synthesising their findings.

Health impact assessment of housing improvements: incorporating research evidence

BACKGROUND

Health impact assessment (HIA) has been widely recommended for future social policies and investment, such as housing improvement. However, concerns have been raised about the utility and predictive value of an HIA. Use of existing research data would add more weight to forecasts by an HIA. METHODS, RESULTS, and

CONCLUSIONS

A recent systematic review of housing intervention studies found a lack of research. The authors recommended that a broader evidence base would be needed to support HIA. In response to consultation with policymakers and HIA practitioners this paper presents a way in which research can be used to inform HIA. Based on the systematic review, the authors have developed a table of synthesised findings indicating the expected health effects of specific housing improvements. The authors also reviewed observational data of housing associated health risks to highlight the key impacts to consider when doing a housing HIA. The findings are presented and the authors discuss how they should be used to inform evidence based housing HIA. In addition to considering the existing research, HIA must consider the local relevance of research. Consultation with local stakeholders also needs to be incorporated to the final assessment. The lack of data and the difficulties in gathering and reviewing data mean that not all HIAs will be able to be informed by research evidence. Well conducted prospective validation of HIAs would contribute to the development of healthy housing investment by informing future housing HIA.

Mothers' use of and attitudes to BabyCheck

A copy of BabyCheck was sent to 497 mothers shortly after the birth of their baby. Six months later they were sent a questionnaire asking about their use of and attitudes to, BabyCheck. Questionnaires were returned by 323 (65%) mothers; 215 (67%) of them reported reading BabyCheck, the majority found it easy to understand (74%) and agreed with the advice (67%). Eighty-four (26%) of the mothers who returned the questionnaires reported using BabyCheck when their baby was ill; of these, 71% agreed with the advice and 65% trusted the advice. None of the mothers had used the complete range of tests. This lead to our conclusion that BabyCheck is well accepted among mothers but is not used routinely as part of mothers' response to illness in their babies. BabyCheck requires introductory education to ensure that parents score all the tests and that the predictive power of BabyCheck is not compromised. Further evaluation of BabyCheck is needed to find its ideal role in the assessment and monitoring of infant illness.

Dissecting aneurysm of the carotid artery as a cause of respiratory distress in adult cattle

A two-and-a-half-year-old Friesian cow and a five-year-old Charolais cow developed severe respiratory distress and palpable swellings to the left of the larynx as a result of a dissecting aneurysm of the common carotid artery. Neither cow responded to medical treatment. The underlying pathogenesis of the condition was uncertain, but direct trauma to the carotid artery was a possible contributory factor. Aneurysms of the common carotid artery should be considered when swelling occurs in the region of the larynx or when respiratory distress is due to laryngeal compression.

Health effects of housing improvement: systematic review of intervention studies

OBJECTIVE

To review the evidence on the effects of interventions to improve housing on health.

DESIGN

Systematic review of experimental and non-experimental housing intervention studies that measured quantitative health outcomes.

DATA SOURCES

Studies dating from 1887, in any language or format, identified from clinical, social science, and grey literature databases, personal collections, expert consultation, and reference lists.

MAIN OUTCOME MEASURES

Socioeconomic change and health, illness, and social measures.

RESULTS

18 completed primary intervention studies were identified. 11 studies were prospective, of which six had control groups. Three of the seven retrospective studies used a control group. The interventions included rehousing, refurbishment, and energy efficiency measures. Many studies showed health gains after the intervention, but the small study populations and lack of controlling for confounders limit the generalisability of these findings.

CONCLUSIONS

The lack of evidence linking housing and health may be attributable to pragmatic difficulties with housing studies as well as the political climate in the United Kingdom. A holistic approach is needed that recognises the multifactorial and complex nature of poor housing and deprivation. Large scale studies that investigate the wider social context of housing interventions are required.