Performance testing of medical echo/Doppler equipment

Comparative Approach to Performance Estimation of Pulsed Wave Doppler Equipment Based on Kiviat Diagram

Quality assessment of ultrasound medical systems is a demanding task due to the high number of parameters to quantify their performance: in the present study, a Kiviat diagram-based integrated approach was proposed to effectively combine the contribution of some experimental parameters and quantify the overall performance of pulsed wave Doppler (PWD) systems for clinical applications. Four test parameters were defined and assessed through custom-written measurement methods based on image analysis, implemented in the MATLAB environment, and applied to spectral images of a flow phantom, i.e., average maximum velocity sensitivity (AMVS), velocity measurements accuracy (VeMeA), lowest detectable signal (LDS), and the velocity profile discrepancy index (VPDI). The parameters above were scaled in a standard range to represent the four vertices of a Kiviat plot, whose area was considered the overall quality index of the ultrasound system in PWD mode. Five brand-new ultrasound diagnostic systems, equipped with linear array probes, were tested in two different working conditions using a commercial flow phantom as a reference. The promising results confirm the robustness of AMVS, VeMeA, and LDS parameters while suggesting further investigations on the VPDI.

A Comparative Study on a Novel Quality Assessment Protocol Based on Image Analysis Methods for Color Doppler Ultrasound Diagnostic Systems

Color Doppler (CD) imaging is widely used in diagnostics since it allows real-time detection and display of blood flow superimposed on the B-mode image. Nevertheless, to date, a shared worldwide standard on Doppler equipment testing is still lacking. In this context, the study herein proposed would give a contribution focusing on the combination of five test parameters to be included in a novel Quality Assessment (QA) protocol for CD systems testing. A first approach involving the use of the Kiviat diagram was investigated, assuming the diagram area, normalized with respect to one of the gold standards, as an index of the overall Doppler system performance. The QA parameters were obtained from the post-processing of CD data through the implementation of custom-written image analysis methods and procedures, here applied to three brand-new high-technology-level ultrasound systems. Experimental data were collected through phased and convex array probes, in two configuration settings, by means of a Doppler flow phantom set at different flow rate regimes. The outcomes confirmed that the Kiviat diagram might be a promising tool applied to quality controls of Doppler equipment, although further investigations should be performed to assess the sensitivity and specificity of the proposed approach.

Comprehensive Comparison of Image Quality Aspects Between Conventional and Plane-Wave Imaging Methods on a Commercial Scanner

Coherent plane-wave compound imaging (CPWCI) is used as alternative for conventional focused imaging (CFI) to increase frame rates linearly with the ratio number of imaging lines to steering angles. In this study, the image quality was compared between CPWCI and CFI, and the effect of steering angles (range and number) and beamforming strategies was evaluated in CPWCI. In automated breast volume scanners (ABVSs), which suffer from reduced volume rates, CPWCI might be an excellent candidate to replace CFI. Therefore, the image quality of CFI currently in ABVS and CPWCI was also compared in an in vivo breast lesion. Images were obtained by a Siemens Sequoia ultrasound system, and two transducers (14L5 and 10L4) in a CIRS multipurpose phantom (040GSE) and a breast lesion. Phantom results showed that contrast sensitivity and resolution, axial resolution, and generalized contrast-to-noise ratio (gCNR; imaging depths <45 mm) were similar for most imaging sequences. CNR (imaging depths ≥45 mm), penetration, and lateral resolution were significantly improved for CPWCI (15 angles) compared to CFI for both transducers. In CPWCI, certain combinations of steering angles and beamforming methods yielded improved gCNR (small angles and delay-and-sum) or lateral resolution (large angles and Lu's-fk). Image quality seemed similar between CPWCI and CFI (three angles incoherent compounded as in ABVS) by visual inspection of the in vivo breast lesion images.

Quantitative Ultrasound Imaging of Healthy and Reconstructed Cleft Lip: A Feasibility Study

Objective:

To investigate the feasibility of echographic imaging of healthy and reconstructed cleft lip and to estimate tissue dimensions and normalized echo level.

Methods:

Echographic images of the upper lip were made on three healthy subjects and two patients using a linear array transducer (7 to 11 MHz bandwidth) and a noncontact gel coupling. Tissue dimensions were measured using calipers. Echo levels were calibrated and were corrected for beam characteristics, gel path, and tissue attenuation using a tissue-mimicking phantom.

Results:

At the central position of the philtrum, mean thickness (SD) of lip loose connective tissue layer, orbicularis oris muscle, and dense connective layer was 4.0 (0.1) mm, 2.3 (0.7) mm, and 2.2 (0.7) mm, respectively, in healthy lip at rest; and 4.1 (0.9) mm, 3.8 (1.7) mm, and 2.6 (0.6) mm, respectively, in contracted lip. Mean (SD) echo level of muscle and dense connective tissue layer with respect to echo level of lip loose connective tissue layer was −19.3 (0.6) dB and −10.7 (4.0) dB, respectively, in relaxed condition and −20.7 (1.5) dB and −7.7 (2.3) dB, respectively, in contracted state. Color mode echo images were calculated, showing lip tissues in separate colors and highlighting details like discontinuity of the orbicularis oris muscle and presence of scar tissue.

Conclusions:

Quantitative assessment of thickness and echo level of various lip tissues is feasible after proper echographic equipment calibration. Diagnostic potentials of this method for noninvasive evaluation of cleft lip reconstruction outcome are promising.

Evaluation of Imaging Parameters of Ultrasound Scanners: Baseline for Future Testing

Background

Regular quality control is required in Poland only for those methods of medical imaging which involve the use of ionizing radiation but not for ultrasonography. It is known that the quality of ultrasound images may be affected by the wearing down or malfunctioning of equipment.

Material/Methods

An evaluation of image quality was carried out for 22 ultrasound scanners equipped with 46 transducers. The CIRS Phantom model 040GSE was used. A set of tests was established which could be carried out with the phantom, including: depth of penetration, dead zone, distance measurement accuracy, resolution, uniformity, and visibility of structures.

Results

While the dead zone was 0 mm for 89% of transducers, it was 3 mm for the oldest transducer. The distances measured agreed with the actual distances by 1 mm or less in most cases, with the largest difference of 2.6 mm. The resolution in the axial direction for linear transducers did not exceed 1 mm, but it reached even 5 mm for some of the convex and sector transducers, especially at higher depths and in the lateral direction. For 29% of transducers, some distortions of anechoic structures were observed. Artifacts were detected for several transducers.

Conclusions

The results will serve as a baseline for future testing. Several cases of suboptimal image quality were identified along with differences in performance between similar transducers. The results could be used to decide on the applicability of a given scanner or transducer for a particular kind of examination.

Breast Ultrasound Technology and Performance Evaluation of Ultrasound Equipment: B-Mode

Ultrasound (US) has become increasingly important in imaging and image-guided interventional procedures. In order to ensure that the imaging equipment performs to the highest level achievable and thus provides reliable clinical results, a number of quality control (QC) methods have been developed. Such QC is increasingly required by accrediting agencies and professional organizations; however, these requirements typically do not include detailed procedures for how the tests should be performed. In this paper, a detailed overview of QC methods for general and breast US imaging using computer-based objective methods is described. The application of QC is then discussed within the context of a common clinical application (US-guided needle biopsy) as well as for research applications, where QC may not be mandated, and thus is rarely discussed. The implementation of these methods will help in finding early stage equipment faults and in optimizing image quality, which could lead to better detection and classification of suspicious findings in clinical applications, as well as improving the robustness of research studies.

The effect of dead elements on the accuracy of Doppler ultrasound measurements

The objective of this study is to investigate the effect of multiple dead elements in an ultrasound probe on the accuracy of Doppler ultrasound measurements. For this work, we used a specially designed ultrasound imaging system, the Ultrasonix Sonix RP, that provides the user with the ability to disable selected elements in the probe. Using fully functional convex, linear, and phased array probes, we established a performance baseline by measuring the parameters of a laminar parabolic flow profile. These same parameters were then measured using probes with 1 to 10 disabled elements. The acquired velocity spectra from the functional probes and the probes with disabled elements were then analyzed to determine the overall Doppler power, maximum flow velocity, and average flow velocity. Color Flow Doppler images were also evaluated in a similar manner. The analysis of the Doppler spectra indicates that the overall Doppler power as well as the detected maximum and average velocities decrease with the increasing number of disabled elements. With multiple disabled elements, decreases in the detected maximum and average velocities greater than 20% were recorded. Similar results were also observed with Color Flow Doppler measurements. Our results confirmed that the degradation of the ultrasound probe through the loss of viable elements will negatively affect the quality of the Doppler-derived diagnostic information. We conclude that the results of Doppler measurements cannot be considered accurate or reliable if there are four or more contiguous dead elements in any given probe.

Reproducibility of phantom-based quality assurance parameters in real-time ultrasound imaging

BACKGROUND

In a large radiological center, the ultrasound (US) quality assurance (QA) program involves several professionals. Although the operator and the parameters utilized can contribute to the results, the selected QA parameters should still reflect the quality of the US scanner, not the measuring process.

PURPOSE

To evaluate the reproducibility of recommended phantom-based US QA parameters in a realistic environment.

MATERIAL AND METHODS

Six sonographers measured six high-end US scanners with 20 transducers using a general purpose phantom. Every transducer was measured altogether seven times, using one frequency per transducer. The QA parameters studied were homogeneity, visualization depth, vertical and horizontal distance measurements, axial and lateral resolution, and the correct visibility of anechoic and high-contrast masses. The evaluation of the homogeneity was based on visual observations. Inter-observer interquartile ranges were computed for the grading of the masses. For the other QA parameters, the mean inter- and intra-observer coefficients of variation (CoV) were calculated. In addition, the symmetry of the reverberations when imaging air with a clean transducer was checked.

RESULTS

The mean inter-observer CoVs were: visualization depth 11 ± 4%, vertical distance 1.7 ± 0.4%, horizontal distance 1.4 ± 0.6%, axial resolution 22 ± 7%, and lateral resolution 16 ± 8%. The mean intra-observer values were about half of these values with similar standard deviations. The visual evaluation of the homogeneity and the symmetry of the reverberations produced false-positive findings in 5% of the cases, but were found useful in detecting a defective transducer. The grading of the masses had mean interquartile ranges of 20-30% of the grading scale.

CONCLUSION

The inter-observer variability in measuring phantom-based QA parameters can be relatively high. This should be considered when implementing a phantom-based QA protocol and evaluating the results.

Measurements, phantoms, and standardization

The last 25 years has seen a number of significant developments in the establishment of a measurement infrastructure supporting medical applications of ultrasound. This has allowed manufacturers and users of medical ultrasonic equipment to undertake and compare measurements of key parameters describing the magnitude or strength of the applied ultrasonic field in a meaningful and traceable way: for equipment development, standards compliance, and quality assurance purposes. This paper describes the current state of the art for measurement techniques used to determine the key properties of an ultrasonic field, principally acoustic pressure and acoustic power. Measurement tools and methodologies are described in detail, alongside considerations of how these are likely to develop, shaped by user need. The way that these measurement methods underpin a range of international and national specification standards enabling equipment manufacturers to demonstrate that their equipment is safe and fit for purpose is covered.

Mechanical and clinical performance of pulse-inversion tissue harmonic imaging in the superficial region

PURPOSE

To compare image quality between fundamental imaging (FI; transmitted and received frequencies 8.0 and 9.0 MHz, respectively) and pulse-inversion tissue harmonic imaging (PI-THI; transmitted frequencies 4.0 and 4.5 MHz, respectively) within 40 mm of the surface of the probe, we tested axial, lateral, and contrast resolution.

METHODS

The axial and lateral resolution were tested by use of a tissue-mimicking test object. For contrast resolution, milk agar blocks of different concentration were used for the comparison test.

RESULTS

The lateral resolution was significantly improved by use of PI-THI at a depth of 20 to 40 mm from the surface of the phantom. Axial resolution was not reduced by use of PI-THI compared with FI. The value of the contrast index was higher in PI-THI than in FI. The researchers also evaluated the two modes clinically, by comparing the visibility of 170 cervical lymph nodes (identification of lymph node, echogenic hilus, parenchymal echogenicity). The results revealed significant advantages of PI-THI over FI for three features.

CONCLUSION

This study suggests that PI-THI might be mechanically and clinically useful in the superficial region.

The antral follicle count: practical recommendations for better standardization

OBJECTIVE

To provide recommendations for the standardized use of the Antral follicle count (AFC) which is used to predict ovarian response to gonadotrophin stimulation during assisted reproductive technology treatment. However, the nature of the follicles that are visualized by ultrasound and the competence of the oocytes held within are largely unknown. In addition, there is considerable variability in the clinical definitions and technical methods used to count and measure antral follicles in both published studies and clinical practice.

DESIGN AND SETTING

In December 2007, specialist reproductive medicine clinicians and scientists attended a workshop in an effort to address these issues. Literature concerning the physiology and measurement of ovarian antral follicles was reviewed, clinical and technical considerations regarding antral follicle measurement were discussed, and an operational definition of AFC was developed.

PATIENT(S)

None.

INTERVENTION(S)

None.

OUTCOME MEASURES

Simple recommendations were established for the standardization of AFC assessment in routine clinical practice. The basic clinical and technical requirements required for AFC evaluation were agreed upon, and a systematic method of measuring and counting antral follicles in routine practice was proposed.

CONCLUSION(S)

The use of a standardized approach according to the practical recommendations for antral follicle counting as presented is encouraged in future clinical trials and routine practice. The authors also advocate a systematic evaluation of these recommendations as standardized study data become available.

High incidence of defective ultrasound transducers in use in routine clinical practice

AIMS

The objective was to evaluate the function of ultrasound transducers in use in routine clinical practice and thereby estimating the incidence of defective transducers.

METHODS AND RESULTS

The study comprised a one-time test of 676 transducers from 7 manufacturers which were in daily use in clinical departments at 32 hospitals. They were tested with the Sonora FirstCall Test System; 39.8% exhibited a transducer error. Delamination was detected in 26.5% and break in the cable was detected in 8.4% of the tested transducers. Errors originating from the piezoelectrical elements were unusual. Delamination and short circuit occurred without significant differences between transducers from all tested manufacturers, but the errors break in the cable, weak and dead element showed a statistically significant higher frequency in transducers from certain manufacturers.

CONCLUSION

The high error frequency and the risk for incorrect medical decisions when using a defective transducer indicate an urgent need for increased testing of the transducers in clinical departments.

Computer-aided B-mode ultrasound diagnosis of hepatic steatosis: a feasibility study

Fatty liver (steatosis) occurs in obese patients, among others, and is related to the development of diabetes type-2. Timely diagnosis of steatosis is therefore of great importance. Steatosis is also the most common liver disease of high-yielding dairy cattle during early lactation. This makes it a suitable animal model for studying liver steatosis. Furthermore, reference of derived ultrasound parameters against a "gold standard" is possible in cattle by taking a liver biopsy for the assessment of fat concentration. The authors undertook this pilot study to investigate the hypothesis that quantitative, computer-aided B-mode ultrasound enables the noninvasive detection of hepatic steatosis. Echographic images were obtained postpartum from dairy cows (n = 12) in transcutaneous and direct (intraoperative) applications using a convex array transducer at 4.2 MHz. During surgery, a biopsy was taken from the caudate lobe to assess the liver fat content (fat score). A custom-designed software package for computer-aided ultrasound diagnosis (CAUS) was developed. After linearizing the post-processing look-up-table (LUT), the image gray levels were transferred into echo levels in decibels relative to the mean echo level in a tissue-mimicking phantom. The quantitative comparison of transcutaneous and intraoperative images enabled the correction for the attenuation effect of skin and subcutaneous fat layer on the mean echo level in the liver, as well as for the effects of the beam formation and attenuation of liver tissue on the echo level vs. depth. The residual attenuation coefficient (dB/cm) in fatty liver vs. normal liver was estimated and compensated for. Finally, echo level was estimated relative to the phantom used for calibration, and echo texture was characterized by the mean axial and lateral speckle size within the regions of interest. In the no fat/low fat group (n = 5) skin plus fat layer attenuation was 3.4 dB/cm. A correlation of skin layer thickness vs. fat score of r = 0.48 was found. The mean transcutaneous liver tissue echo level correlated well with fat score: r = 0.80. A residual liver attenuation coefficient of 0.76 dB/cm and 1.19 dB/cm was found in medium and high fat liver, respectively. In transcutaneous images, correlation of residual attenuation coefficient with fat score was r = 0.69. Axial and lateral speckle sizes were on the order of 0.2 and 1.0 cm, respectively, and no correlation was found with liver fat content. Results for transcutaneous and intraoperative images were similar. The authors conclude that this pilot study shows the feasibility of calibrated, computer-aided ultrasound for noninvasively diagnosing, possibly even screening, steatosis of the liver.

Objective performance testing and quality assurance of medical ultrasound equipment

There is an urgent need for a measurement protocol and software analysis for objective testing of the imaging performance of medical ultrasound equipment from a user's point of view. Methods for testing of imaging performance were developed. Simple test objects were used, which have a long life expectancy. First, the elevational focus (slice thickness) of the transducer was estimated and the in-plane transmit focus was positioned at the same depth. Next, the postprocessing look-up-table (LUT) was measured and linearized. The tests performed were echo level dynamic range (dB), contrast resolution (i.e., gamma of display, number of gray levels/dB) and sensitivity, overall system sensitivity, lateral sensitivity profile, dead zone, spatial resolution and geometric conformity of display. The concept of a computational observer was used to define the lesion signal-to-noise ratio, SNR(L) (or Mahalanobis distance), as a measure for contrast sensitivity. All the measurements were made using digitized images and quantified by objective means, i.e., by image analysis. The whole performance measurement protocol, as well as the quantitative measurements, have been implemented in software. An extensive data-base browser was implemented from which analysis of the images can be started and reports generated. These reports contain all the information about the measurements, such as graphs, images and numbers. The approach of calibrating the gamma by using a linearized LUT was validated by processing simultaneously acquired rf data. The contrast resolution and echo level of the rf data had to be compressed by a factor of two and amplified by a gain factor corresponding to 12 dB. This resulted in contrast curves that were practically identical to those obtained from DICOM image data. The effects of changing the transducer center frequency on the spatial resolution and contrast sensitivity were estimated to illustrate the practical usefulness of the developed approach of quality assurance by measuring objective performance characteristics. The developed methods might be considered as a minimum set of objective quality assurance measures. This set might be used to predict clinical performance of medical ultrasound equipment, taking into account the performance at a unique point in space i.e., the coinciding depths of the elevation and in-plane (azimuth) foci. Furthermore, it should be investigated whether the approach might be used to compare objectively various brands of equipment and to evaluate the performance specifications given by the manufacturer. Last but not least, the developed approach can be used to monitor, in a hospital environment, the medical ultrasound equipment during its life cycle. The software package may be viewed and downloaded at the website http://www.qa4us.eu.

Accuracy of spectral Doppler flow and tissue velocity measurements in ultrasound systems

Blood and tissue velocity are measured and analysed in cardiac, vascular and other applications of diagnostic ultrasound (US). An error in system calibration is a potential risk for misinterpretation of the measurements. To determine the accuracy in velocity calibration, we tested three common commercial US systems using a Doppler string phantom. We tested pulsed and continuous-wave Doppler modes for velocities relevant to both cardiac blood flow and tissue-velocity estimation. The US systems were tested with settings and transducers commonly used in cardiac applications. One system consistently overestimated velocity by about 5%, whereas the other two systems were quite accurate in velocity estimation. These findings emphasize the importance of continuous quality control of US equipment.