Usability, accuracy, and cost-effectiveness of a medical software for early pregnancies: a retrospective study

STUDY QUESTION

Can early pregnancies be accurately and cost-effectively diagnosed and managed using a new medical computerized tool?

SUMMARY ANSWER

Compared to the standard clinical approach, retrospective implementation of the new medical software in a gynaecological emergency unit was correlated with more accurate diagnosis and more cost-effective management.

WHAT IS KNOWN ALREADY

Early pregnancy complications are responsible for a large percentage of consultations, mostly in emergency units, with guidelines becoming complex and poorly known/misunderstood by practitioners.

STUDY DESIGN, SIZE, DURATION

A total of 780 gynaecological emergency consultations (446 patients), recorded between November 2018 and June 2019 in a tertiary university hospital, were retrospectively encoded in a new medical computerized tool. The inclusion criteria were a positive hCG test result, ultrasonographical visualization of gestational sac, and/or embryo corresponding to a gestational age of 14 weeks or less. Diagnosis and management suggested by the new computerized tool are named eDiagnoses, while those provided by a gynaecologist member of the emergency department staff are called medDiagnoses.

PARTICIPANTS/MATERIALS, SETTING, METHODS

Usability was the primary endpoint, with accuracy and cost reduction, respectively, as secondary and tertiary endpoints. Identical eDiagnoses/medDiagnoses were considered as accurate. During follow-up visits, if the updated eDiagnoses and medDiagnoses became both identical to a previously discrepant eDiagnosis or medDiagnosis, this previous eDiagnosis or medDiagnosis was also considered as correct. Four double-blinded experts reviewed persistent discrepancies, determining the accurate diagnoses. eDiagnoses/medDiagnoses accuracies were compared using McNemar's Chi square test, sensitivity, specificity, and predictive values.

MAIN RESULTS AND THE ROLE OF CHANCE

Only 1 (0.1%) from 780 registered medical records lacked data for full encoding. Out of the 779 remaining consultations, 675 eDiagnoses were identical to the medDiagnoses (86.6%) and 104 were discrepant (13.4%). From these 104, 60 reached an agreement during follow-up consultations, with 59 medDiagnoses ultimately changing into the initial eDiagnoses (98%) and only one discrepant eDiagnosis turning later into the initial medDiagnosis (2%). Finally, 24 remained discrepant at all subsequent checks and 20 were not re-evaluated. Out of these 44, the majority of experts agreed on 38 eDiagnoses (86%) and 5 medDiagnoses (11%, including four twin pregnancies whose twinness was the only discrepancy). No majority was reached for one discrepant eDiagnosis/medDiagnosis (2%). In total, the accuracy of eDiagnoses was 99.1% (675 + 59 + 38 = 772 eDiagnoses out of 779), versus 87.4% (675 + 1 + 5 = 681) for medDiagnoses (P < 0.0001). Calculating all basic costs of extra consultations, extra-medications, extra-surgeries, and extra-hospitalizations induced by incorrect medDiagnoses versus eDiagnoses, the new medical computerized tool would have saved 3623.75 Euros per month. Retrospectively, the medical computerized tool was usable in almost all the recorded cases (99.9%), globally more accurate (99.1% versus 87.4%), and for all diagnoses except twinning reports, and it was more cost-effective than the standard clinical approach.

LIMITATIONS, REASONS FOR CAUTION

The retrospective study design is a limitation. Some observed improvements with the medical software could derive from the encoding by a rested and/or more experienced physician who had a better ultrasound interpretation. This software cannot replace clinical and ultrasonographical skills but may improve the compliance to published guidelines.

WIDER IMPLICATIONS OF THE FINDINGS

This medical computerized tool is improving. A new version considers diagnosis and management of multiple pregnancies with their specificities (potentially multiple locations, chorioamnionicity). Prospective evaluations will be required. Further developmental steps are planned, including software incorporation into ultrasound devices and integration of previously published predictive/prognostic factors (e.g. serum progesterone, corpus luteum scoring).

STUDY FUNDING/COMPETING INTEREST(S)

No external funding was obtained for this study. F.B. and D.G. created the new medical software.

TRIAL REGISTRATION NUMBER

NCT03993015.

P-404 Usability, accuracy and cost-effectiveness of “eDiagEPU”, a medical software for early pregnancies: a retrospective study

Study question

Can early pregnancies be accurately and cost-effectively diagnosed and managed using a new medical computerised tool, named “eDiagEPU”?

Summary answer

Compared to the standard clinical approach, the retrospective implementation of “eDiagEPU” in a gynaecological emergency unit was correlated with sharper diagnoses and more cost-effective managements.

What is known already

Early pregnancies complications are responsible for a large percentage of consultations, mostly in emergency units. Moreover, clinical guidelines updates for the management of Intrauterine Pregnancies of Uncertain Viability (IPUV) have become increasingly complex and seem to be unknown or misunderstood by several practitioners. Specifically, a recently published prospective multinational survey revealed a limited knowledge regarding early pregnancy guidelines, with 69.0% of the participants reporting incorrect managements of IPUV and 86.6% misinterpreting the evolution of serum human chorionic gonadotropin (hCG).

In an attempt to aid practitioners with the diagnosis and management of early pregnancies, a software, named “eDiagEPU”, was developed.

Study design, size, duration

A total of 780 consultations, recorded between November 2018 and June 2019 in the gynaecological emergency unit of a tertiary university hospital, were retrospectively encoded in eDiagEPU. Positive hCG, ultrasonographical visualisation of gestational sac or/and embryo corresponding to a gestational age of 14 weeks gestation or less were the inclusion criteria.

Diagnoses and managements suggested by eDiagEPU are named “eDiagnoses”. The ones provided by a gynaecologist member of the emergency department staff are called “medDiagnoses”.

Participants/materials, setting, methods

Identical eDiagnosis and medDiagnosis were considered as correct (gold standard). During follow-up examinations, if they became both identical to a previous discrepant eDiagnosis or medDiagnosis, this previous eDiagnosis/medDiagnosis was considered as correct. Persistent discrepancies were reviewed by four double-blinded experts whose majority defined the correct eDiagnosis/medDiagnosis.

The accuracies of eDiagnoses/medDiagnoses were compared using McNemar’s Chi square test, computing diagnostic values (Sensitivity, Specificity, and predictive values) and 95% Confidence Intervals (CI). Cost reduction was also analysed.

Main results and the role of chance

Only one datum (0.1%) from 780 registered medical records was missing to process using “eDiagEPU”. Out of the 779 consultations that could be fully encoded until obtaining an eDiagnosis, 675 eDiagnoses were identical to the medDiagnoses (86.6%) and 104 discrepant (13.4%). From these 104, 60 reached an agreement during follow-up controls with 59 medDiagnoses finally changing into the initial eDiagnoses (98%) while only one discrepant eDiagnosis turning later into the initial medDiagnosis (2%). Finally, 24 remained discrepant at all subsequent checks and 20 were not reevaluated. Out of these 44 discrepancies without identical diagnoses/managements during follow-up controls, the double-blinded experts majority chose 38 eDiagnoses (86%) and 5 medDiagnoses (11%) including 4 twin pregnancies whose twinness was the only discrepancy. One discrepant eDiagnosis/medDiagnosis reached no majority (2%).

In total, eDiagnoses accuracy was 99.1% (675 + 59 + 38=772 eDiagnoses out of 779 final diagnoses), vs 87.4% (675 + 1 + 5=681) for medDiagnoses accuracy (p &lt; 0.0001). Calculating all basic costs of consultations, medications, surgeries and hospitalisations induced by medDiagnoses versus eDiagnoses, “eDiagEPU” would have saved 3 623.75 € per month.

Retrospectively, “eDiaEPU” was usable (99.9%), more accurate for each diagnosis except twinning report and more cost-effective than standard clinical approach.

Limitations, reasons for caution

The retrospective design is a limitation, as well as the quality of ultrasound interpretation. Some improvements could not derive exclusively from “eDiagEPU” but also from the encoding by a rested or more experienced physician. This software cannot replace clinical and ultrasonographical skills but can improve the diagnostic and therapeutic reasoning.

Wider implications of the findings

An improved “eDiagEPU” version, considering the diagnosis and management of multiple pregnancies with their specificities (potentially multiple locations, chorioamnionicity) has been developed. Prospective evaluations will be required. Further development steps are considered, including software incorporation into ultrasound devices and integration of previously published predictive/prognostic factors (serum progesterone, corpus luteum scoring...).

Trial registration number

NCT03993015