A pragmatic methodology to extract anesthetic and physiological data from the electronic health record

BACKGROUND/AIMS

Traditional manual methods of extracting anesthetic and physiological data from the electronic health record rely upon visual transcription by a human analyst that can be labor-intensive and prone to error. Technical complexity, relative inexperience in computer coding, and decreased access to data warehouses can deter investigators from obtaining valuable electronic health record data for research studies, especially in under-resourced settings. We therefore aimed to develop, pilot, and demonstrate the effectiveness and utility of a pragmatic data extraction methodology.

METHODS

Expired sevoflurane concentration data from the electronic health record transcribed by eye was compared to an intermediate preprocessing method in which the entire anesthetic flowsheet narrative report was selected, copy-pasted, and processed using only Microsoft Word and Excel software to generate a comma-delimited (.csv) file. A step-by-step presentation of this method is presented. Concordance rates, Pearson correlation coefficients, and scatterplots with lines of best fit were used to compare the two methods of data extraction.

RESULTS

A total of 1132 datapoints across eight subjects were analyzed, accounting for 18.9 h of anesthesia time. There was a high concordance rate of data extracted using the two methods (median concordance rate 100% range [96%, 100%]). The median time required to complete manual data extraction was significantly longer compared to the time required using the intermediate method (240 IQR [199, 482.5] seconds vs 92.5 IQR [69, 99] seconds, p = .01) and was linearly associated with the number of datapoints (rmanual  = .97, p < .0001), whereas time required to complete data extraction using the intermediate approach was independent of the number of datapoints (rintermediate  = -.02, p = .99).

CONCLUSIONS

We describe a pragmatic data extraction methodology that does not require additional software or coding skills intended to enhance the ease, speed, and accuracy of data collection that could assist in clinician investigator-initiated research and quality/process improvement projects.

Design and Analysis Methods for Trials with AI-Based Diagnostic Devices for Breast Cancer

Imaging is important in cancer diagnostics. It takes a long period of medical training and clinical experience for radiologists to be able to accurately interpret diagnostic images. With the advance of big data analysis, machine learning and AI-based devices are currently under development and taking a role in imaging diagnostics. If an AI-based imaging device can read the image as accurately as experienced radiologists, it may be able to help radiologists increase the accuracy of their reading and manage their workloads. In this paper, we consider two potential study objectives of a clinical trial to evaluate an AI-based device for breast cancer diagnosis by comparing its concordance with human radiologists. We propose statistical design and analysis methods for each study objective. Extensive numerical studies are conducted to show that the proposed statistical testing methods control the type I error rate accurately and the design methods provide required sample sizes with statistical powers close to pre-specified nominal levels. The proposed methods were successfully used to design and analyze a real device trial.

Assessment of the Concordance Rate between Intraoperative Pathological Diagnosis and the Final Pathological Diagnosis of Spinal Cord Tumors

The intraoperative pathological diagnosis (IPD) plays an important role in determining the optimal surgical treatment for spinal cord tumors. The final pathological diagnosis (FPD) is sometimes different from the IPD. Here, we sought to identify the accuracy of the IPD of spinal cord tumors compared to the FPD. We retrospec-tively analyzed the cases of 108 patients with spinal cord tumors treated surgically in our institute; the IPD, FPD, mismatched cases, and concordance rate between the IPD and FPD were investigated. Five cases involved a mismatch between the IPD and FPD. The overall concordance rate was 95.4%, with 90.9% for extra-dural lesions, 98.5% for intradural extramedullary lesions, 84.2% for intramedullary lesions, and 100% for dumbbell-type tumors. The concordance rate of intramedullary lesions tended to be lower than that of other lesions (p = 0.096). A lower concordance rate was revealed for intramedullary lesions compared to the other lesions. Despite the IPD clearly remaining a valuable tool during operative procedures, surgeons should recog-nize the limitations of IPDs and make comprehensive decisions about surgical treatments.

Validation of digital microscopy: Review of validation methods and sources of bias

Digital microscopy (DM) is increasingly replacing traditional light microscopy (LM) for performing routine diagnostic and research work in human and veterinary pathology. The DM workflow encompasses specimen preparation, whole-slide image acquisition, slide retrieval, and the workstation, each of which has the potential (depending on the technical parameters) to introduce limitations and artifacts into microscopic examination by pathologists. Performing validation studies according to guidelines established in human pathology ensures that the best-practice approaches for patient care are not deteriorated by implementing DM. Whereas current publications on validation studies suggest an overall high reliability of DM, each laboratory is encouraged to perform an individual validation study to ensure that the DM workflow performs as expected in the respective clinical or research environment. With the exception of validation guidelines developed by the College of American Pathologists in 2013 and its update in 2021, there is no current review of the application of methods fundamental to validation. We highlight that there is high methodological variation between published validation studies, each having advantages and limitations. The diagnostic concordance rate between DM and LM is the most relevant outcome measure, which is influenced (regardless of the viewing modality used) by different sources of bias including complexity of the cases examined, diagnostic experience of the study pathologists, and case recall. Here, we review 3 general study designs used for previous publications on DM validation as well as different approaches for avoiding bias.

The predictive value of positive and suspicious urine cytology: Are they different?

BACKGROUND

Urine cytology evaluation is an effective test in the detection of high-grade urothelial carcinoma (HGUC). While the guideline distinguishes the 2 categories: "positive for HGUC" (PHGUC) and "suspicious for HGUC" (SHGUC), the association between these categories with their subsequent follow-up biopsies remains unclear. This study aims to determine and compare the positive predictive value (PPV) of the specimens in PHGUC and SHGUC categories with their respective histologic diagnoses.

METHODS

During the period of 03/01/2008 to 07/31/2018, urine cytology cases diagnosed as PHGUC and SHGUC with subsequent bladder biopsy within 12 months were retrieved. All cases were correlated with first biopsy obtained during 12 months of cytology specimen. Biopsy result with HGUC, carcinoma in situ, or non-urothelial carcinoma diagnoses were considered as concordance.

RESULTS

378 cases (229 SHGUC and 149 PHGUC) were identified from 263 patients. For the 229 SHGUC cases, the PPV was 72% (n = 166) and for the 149 PHGUC cases, the PPV was 85% (n = 127). While both categories have high PPV, they are statistically significant (p < 0.0001). Additionally, 33 cases were found to have low-grade urothelial carcinoma (LGUC), constituting a portion of discordant results.

CONCLUSION

PHGUC and SHGUC categories are both associated with a high risk of malignancy, however, there is a statistically significant difference between them in our study, supporting the PSRUC guidelines of two separate categories. In instances when urine cytology is discordant with biopsy results, further investigation and clinical follow up is warranted. LGUC appears to remain a common pitfall especially in the suspicious category.

The Diagnostic Process for the Evaluation of Acute Abdominal Pain by Resident Trainees in Japan: A Cross-sectional Study

Objective Acute abdominal pain (AAP) of diverse etiology is a common chief complaint of patients who present to the emergency department (ED). AAP may pose a diagnostic challenge to physicians in training. We aimed to evaluate whether or not resident trainee doctors examine patients presenting with AAP in a Japanese acute-care hospital following Kendall's diagnostic algorithm. Methods We conducted a retrospective medical chart review from January 2015 to December 2016. Patients Patients ≥50 years old who presented to the ED within 7 days of the onset of AAP who were evaluated by residents at the ED of an acute care hospital were enrolled in this study. Patients transported by ambulance and referred from other hospitals and classified as level 1 or 2 according to the Japanese version of the Canadian Triage and Acuity Scale were excluded. Data, including the clinical history, location and character of pain, and age and gender of patients as well as the level of experience of residents, were abstracted from charts. We evaluated the concordance rate between the actual diagnostic process followed by residents and Kendall's diagnostic algorithm for AAP. Results We analyzed 466 patients (mean age 67.6 years) in the study who were evaluated and diagnosed by 123 residents. The concordance rate between the diagnostic procedures performed by residents and those suggested by Kendall's diagnostic algorithm was 61.2%. A low concordance rate was observed among patients with peritoneal signs, shock or toxic appearance (25.0%), suggested acute coronary syndromes (ACS) (55.1%), epigastric or right upper-quadrant pain (52.8%), and left upper-quadrant pain (55.6%). Abdominal ultrasonography is one of the recommended examinations for patients with signs of peritoneal irritation, shock or toxic appearance, right lower-quadrant pain, and left upper- or lower-quadrant pain, but the rates were relatively low at 25.0%, 34.4%, 31.8%, and 26.7%, respectively. Conclusion Abdominal ultrasonography required by Kendall's diagnostic algorithm was not performed appropriately in patients with symptoms and signs of peritonitis, shock or toxic appearance, right lower-quadrant pain, and left upper- or lower-quadrant pain or in female patients by resident trainees. Our findings underscore the importance of providing resident doctors with focused training concerning ultrasonography by attending physicians.

Comparison of central and local serial CT assessments of metastatic renal cell carcinoma patients in a clinical phase IIB study

Background Clinical oncological studies attempt to improve precision of data by central radiological assessments. However, it is unclear, to which extent local and central assessments diverge. Purpose To quantify inter-reader variability and the deviation of local from central radiological assessments of computed tomography (CT) scans. Material and Methods This was a sub-study of a randomized clinical phase IIb trial in metastatic renal cell carcinoma (RCC), comparing first-line sorafenib with interferon-alpha-2a (IFN-α-2a). It analyzed agreements of local with central RECIST CT assessments by Cohen's kappa (κ), symmetry tests, deviations in waterfall plots, Bland-Altman plots, and parametric survival analyses. Results The concordance between local and central radiologic review was quantified by κ = 0.53. While local assessment yielded progressive disease (PD) in 18.6%, central assessment classified 22.5% of patient time points as PD exhibiting only a partial overlap with the 18.6% The tumor shrinkage rates in waterfall plots were 68.1% in local and 55.8% in central review (57.8% and 59% by Reader 1 and Reader 2). Bland-Altman plots identified a systematic shift of tumor change rates by -7.5% in local compared to central assessments, that may reflect a systematic tendency of more favorable results in local assessments. The discordance between local and central review was reflected by a time to progression (TTP) hazard ratio (HR) of 1.73 ( P = 0.0003). Conclusion These data suggest that central radiologic review may reduce technical measurement variability in clinical trials, which should be scrutinized in future studies compared to a volumetric reference.

Genetic and Environmental Factors in Invasive Cervical Cancer: Design and Methods of a Classical Twin Study

BACKGROUND

Persistent high-risk human papillomavirus (HPV) infection is a necessary prerequisite for development of cervical cancer and its precursor lesion, high-grade squamous intraepithelial lesion (HSIL). However, HPV infection is not sufficient to drive this process, and genetic and environmental factors may also play a role.

METHODS/DESIGN

The Cervical Cancer, Genetics and Environment Twin Study was established to investigate the environmental and genetic influences on variation in susceptibility to cervical pre-cancer in 25- to 69-year-old monozygotic (MZ) and dizygotic (DZ) twins recruited through the Australian Twin Registry. Reviews of Papanicolaou (Pap) screening histories were undertaken to identify individual women with a history of an abnormal Pap test. This was followed by detection of HPV in archival Pap smears of selected twin pairs to determine HPV persistence. Selected twin pairs also completed a detailed questionnaire on socio-demographic characteristics, sexual behavior, and HPV knowledge. In future analyses, under the assumptions of the classical twin design, case-wise concordance for persistent HPV infection and HSIL will be calculated for MZ and DZ twin pairs, and twin pairs (both MZ and DZ) who are discordant for the above outcomes will be used to assess the contributions of measured environmental risk factors.

DISCUSSION

The study examines factors related to HPV persistence and development of HSIL among female MZ and DZ twins. The results will contribute to our understanding of the natural history of cervical HPV infection and the relative contributions of genetic and environmental factors in disease progression.

International validation study for interim PET in ABVD-treated, advanced-stage hodgkin lymphoma: interpretation criteria and concordance rate among reviewers

At present, there are no standard criteria that have been validated for interim PET reporting in lymphoma. In 2009, an international workshop attended by hematologists and nuclear medicine experts in Deauville, France, proposed to develop simple and reproducible rules for interim PET reporting in lymphoma. Accordingly, an international validation study was undertaken with the primary aim of validating the prognostic role of interim PET using the Deauville 5-point score to evaluate images and with the secondary aim of measuring concordance rates among reviewers using the same 5-point score. This paper focuses on the criteria for interpretation of interim PET and on concordance rates.

METHODS

A cohort of advanced-stage Hodgkin lymphoma patients treated with doxorubicin, bleomycin, vinblastine, and dacarbazine (ABVD) were enrolled retrospectively from centers worldwide. Baseline and interim scans were reviewed by an international panel of 6 nuclear medicine experts using the 5-point score.

RESULTS

Complete scan datasets of acceptable diagnostic quality were available for 260 of 440 (59%) enrolled patients. Independent agreement among reviewers was reached on 252 of 260 patients (97%), for whom at least 4 reviewers agreed the findings were negative (score of 1-3) or positive (score of 4-5). After discussion, consensus was reached in all cases. There were 45 of 260 patients (17%) with positive interim PET findings and 215 of 260 patients (83%) with negative interim PET findings. Thirty-three interim PET-positive scans were true-positive, and 12 were false-positive. Two hundred three interim PET-negative scans were true-negative, and 12 were false-negative. Sensitivity, specificity, and accuracy were 0.73, 0.94, and 0.91, respectively. Negative predictive value and positive predictive value were 0.94 and 0.73, respectively. The 3-y failure-free survival was 83%, 28%, and 95% for the entire population and for interim PET-positive and -negative patients, respectively (P < 0.0001). The agreement between pairs of reviewers was good or very good, ranging from 0.69 to 0.84 as measured with the Cohen kappa. Overall agreement was good at 0.76 as measured with the Krippendorf α.

CONCLUSION

The 5-point score proposed at Deauville for reviewing interim PET scans in advanced Hodgkin lymphoma is accurate and reproducible enough to be accepted as a standard reporting criterion in clinical practice and for clinical trials.