Patients' experience of primary care: James Mackenzie Lecture 2017

The good, the bad and the ugly: What do we really do when we identify the best and the worst organisations?

Identifying high and poorly performing organisations is common practice in healthcare. Often this is done within a frequentist inferential framework where statistical techniques are used that acknowledge that observed performance is an imperfect measure of underlying quality. Various methods are employed for this purpose, but the influence of chance on the degree of misclassification is often underappreciated. Using simulations, we show that the distribution of underlying performance of organisations flagged as the worst performers, using current best practices, was highly dependent on the reliability of the performance measure. When reliability was low, flagged organisations were likely to have an underlying performance that was near the population average. Reliability needs to reach at least 0.7 for 50% of flagged organisations to be correctly flagged and 0.9 to nearly eliminate incorrectly flagging organisations close to the overall mean. We conclude that despite their widespread use, techniques for identifying the best and worst performing organisations do not necessarily identify truly good and bad performers and even with the best techniques, reliable data are required.

Elliott M, A. Abel G. The underlying structure of the English Cancer Patient Experience Survey: Factor analysis to support survey reporting and design

BACKGROUND

The English Cancer Patient Experience Survey (CPES) is a regularly conducted survey measuring the experience of cancer patients. We studied the survey's underlying structure using factor analysis to identify potential for improvements in reporting or questionnaire design.

METHODS

Cancer Patient Experience Survey 2015 respondents (n = 71,186, response rate 66%) were split into two random subgroups. Using exploratory factor analysis (EFA) on the first subgroup, we identified the survey's latent structure. EFA was then applied to 12 sets of items. A first ("core") set was formed by questions that applied to all participants. The subsequent sets contained the "core set" plus questions corresponding to specific care pathways/patient groups. We used confirmatory factor analysis (CFA) on the second data subgroup for cross-validation.

RESULTS

The EFA suggested that five latent factors underlie the survey's core questions. Analysis on the remaining 11 care pathway/patient group items also indicated the same five latent factors, although additional factors were present for questions applicable to patients with an overnight stay or those accessing specialist nursing. The five factors models had an excellent fit (comparative fit index = 0.95, root mean square error of approximation = 0.045 for core set of questions). Items loading on each factor generally corresponded to a specific section or subsection of the questionnaire. CFA findings were concordant with the EFA patterns.

CONCLUSION

The findings suggest five coherent underlying sub-constructs relating to different aspects of cancer health care. The findings support the construction of evidence-based composite indicators for different domains of experience and provide options for survey re-design.

Sociodemographic inequalities in patients' experiences of primary care: an analysis of the General Practice Patient Survey in England between 2011 and 2017

OBJECTIVE

Younger people, minority ethnic groups, sexual minorities and people of lower socioeconomic status report poorer experiences of primary care. In light of NHS ambitions to reduce unwarranted variations in care, we aimed to investigate whether inequalities in patient experience of primary care changed between 2011 and 2017, using data from the General Practice Patient Survey in England.

METHODS

We considered inequalities in relation to age, sex, deprivation, ethnicity, sexual orientation and geographical region across five dimensions of patient experience: overall experience, doctor communication, nurse communication, access and continuity of care. We used linear regression to explore whether the magnitude of inequalities changed between 2011 and 2017, using mixed models to assess changes within practices and models without accounting for practice to assess national trends.

RESULTS

We included 5,241,408 responses over 11 survey waves from 2011-2017. There was evidence that inequalities changed over time (p < 0.05 for 27/30 models), but the direction and magnitude of changes varied. Changes in gaps in experience ranged from a 1.6 percentage point increase for experience of access among sexual minorities, to a 5.6 percentage point decrease for continuity, where experience worsened for older ages. Inequalities in access in relation to socio-economic status remained reasonably stable for individuals attending the same GP practice; nationally inequalities in access increased 2.1 percentage points (p < 0.0001) between respondents living in more/less deprived areas, suggesting access is declining fastest in practices in more deprived areas.

CONCLUSIONS

There have been few substantial changes in inequalities in patient experience of primary care between 2011 and 2017.