1
|
Chen J, Qian L, Ma L, Urakov T, Gu W, Liang L. SymTC: A symbiotic Transformer-CNN net for instance segmentation of lumbar spine MRI. Comput Biol Med 2024; 179:108795. [PMID: 38955128 DOI: 10.1016/j.compbiomed.2024.108795] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2024] [Revised: 06/17/2024] [Accepted: 06/19/2024] [Indexed: 07/04/2024]
Abstract
Intervertebral disc disease, a prevalent ailment, frequently leads to intermittent or persistent low back pain, and diagnosing and assessing of this disease rely on accurate measurement of vertebral bone and intervertebral disc geometries from lumbar MR images. Deep neural network (DNN) models may assist clinicians with more efficient image segmentation of individual instances (discs and vertebrae) of the lumbar spine in an automated way, which is termed as instance image segmentation. In this work, we proposed SymTC, an innovative lumbar spine MR image segmentation model that combines the strengths of Transformer and Convolutional Neural Network (CNN). Specifically, we designed a parallel dual-path architecture to merge CNN layers and Transformer layers, and we integrated a novel position embedding into the self-attention module of Transformer, enhancing the utilization of positional information for more accurate segmentation. To further improve model performance, we introduced a new data synthesis technique to create synthetic yet realistic MR image dataset, named SSMSpine, which is made publicly available. We evaluated our SymTC and the other 16 representative image segmentation models on our private in-house dataset and public SSMSpine dataset, using two metrics, Dice Similarity Coefficient and the 95th percentile Hausdorff Distance. The results indicate that SymTC surpasses the other 16 methods, achieving the highest dice score of 96.169 % for segmenting vertebral bones and intervertebral discs on the SSMSpine dataset. The SymTC code and SSMSpine dataset are publicly available at https://github.com/jiasongchen/SymTC.
Collapse
Affiliation(s)
- Jiasong Chen
- Department of Computer Science, University of Miami, Coral Gables, FL, USA
| | - Linchen Qian
- Department of Computer Science, University of Miami, Coral Gables, FL, USA
| | - Linhai Ma
- Department of Computer Science, University of Miami, Coral Gables, FL, USA
| | - Timur Urakov
- Department of Neurological Surgery, University of Miami Miller School of Medicine, Miami, FL, USA
| | - Weiyong Gu
- Department of Mechanical and Aerospace Engineering, University of Miami, Coral Gables, FL, USA
| | - Liang Liang
- Department of Computer Science, University of Miami, Coral Gables, FL, USA.
| |
Collapse
|
2
|
Theodorou DJ, Theodorou SJ, Gelalis ID, Kakitsubata Y. Lumbar Intervertebral Disc and Discovertebral Segment, Part 1: An Imaging Review of Normal Anatomy. Cureus 2022; 14:e25558. [PMID: 35784982 PMCID: PMC9249043 DOI: 10.7759/cureus.25558] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/31/2022] [Indexed: 12/03/2022] Open
Abstract
The intervertebral disc is designated the most important cartilaginous articulation of the vertebral column that functions to withstand compressive biomechanical forces and confer strength and flexibility to the spine. A thorough study of the complex fine structure and anatomic relationships of the intervertebral disc is essential for the characterization of the integrity of each individual structure in the discovertebral segment. This elaborate work in human cadavers explores the sophisticated internal structure of the normal intervertebral disc and the discovertebral segment, providing detailed data derived from the dissection of specimens through imaging and close anatomic-histologic correlation. Familiarity with the normal appearances and basic functional properties of the lumbar intervertebral disc and discovertebral segment is fundamental for the recognition of aberrations that may have important clinical implications in patients with low back pain. In Part I of this article, the anatomic structure and features of the discovertebral complex in adults will be described.
Collapse
|
3
|
Aavikko A, Lohman M, Ristolainen L, Kautiainen H, Österman K, Schlenzka D, Lund T. ISSLS prize in clinical science 2022: accelerated disc degeneration after pubertal growth spurt differentiates adults with low back pain from their asymptomatic peers. EUROPEAN SPINE JOURNAL : OFFICIAL PUBLICATION OF THE EUROPEAN SPINE SOCIETY, THE EUROPEAN SPINAL DEFORMITY SOCIETY, AND THE EUROPEAN SECTION OF THE CERVICAL SPINE RESEARCH SOCIETY 2022; 31:1080-1087. [PMID: 35333957 DOI: 10.1007/s00586-022-07184-0] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/15/2021] [Revised: 01/20/2022] [Accepted: 03/12/2022] [Indexed: 11/25/2022]
Abstract
PURPOSE In this prospective observational cohort study, the development of lumbar intervertebral discs (LIVD) on magnetic resonance imaging (MRI) was investigated from childhood to adulthood with emphasis on the possible association of disc degeneration (DD) to low back pain (LBP). METHODS In 2021, 89 subjects who were enrolled in 1994 in a longitudinal study with lumbar spine MRI at ages 8, 11 and 18 were invited to participate in a long-term follow-up comprising a clinical examination, selected patient-reported outcome measures and a lumbar spine MRI. We assessed all MRIs (three lowest LIVDs) with the Pfirrmann summary score, and the ratio of signal intensity of nucleus pulposus to signal intensity of cerebrospinal fluid (SINDL). We further analyzed whether disc changes at any age were associated with self-reported LBP at age 34. RESULTS Of the 48 subjects in the follow-up, 35 reported LBP at age 34. The Pfirrmann summary score significantly increased with age (p < 0.001). Subjects reporting LBP at age 34 demonstrated statistically significantly higher summary scores at age 18 and 34 compared to asymptomatic subjects (p = 0.004 at age 18, and p = 0.039 at age 34). SINDL significantly decreased with age (p < 0.001 for all levels separately), but no significant differences between subjects with or without LBP at age 34 were noticed. CONCLUSION Subjects with LBP at age 34 had more widespread or severe DD already at age 18 compared to those without LBP.
Collapse
Affiliation(s)
- Anni Aavikko
- Department of Orthopedics and Traumatology, Helsinki University Hospital, Helsinki, Finland
| | - Martina Lohman
- Department of Radiology, Helsinki University Hospital, Helsinki, Finland
| | - Leena Ristolainen
- Research Institute Orton, Orton Orthopaedic Hospital, Helsinki, Finland
| | | | - Kalevi Österman
- Research Institute Orton, Orton Orthopaedic Hospital, Helsinki, Finland
| | | | - Teija Lund
- Department of Orthopedics and Traumatology, Helsinki University Hospital, Helsinki, Finland.
| |
Collapse
|
4
|
Quantitative assessment of the lumbar intervertebral disc via T2 shows excellent long-term reliability. PLoS One 2021; 16:e0249855. [PMID: 33852631 PMCID: PMC8046347 DOI: 10.1371/journal.pone.0249855] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2020] [Accepted: 03/25/2021] [Indexed: 12/05/2022] Open
Abstract
Methodologies for the quantitative assessment of the spine tissues, in particular the intervertebral disc (IVD), have not been well established in terms of long-term reliability. This is required for designing prospective studies. 1H water T2 in the IVD (“T2”) has attained wider use in assessment of the lumbar intervertebral discs via magnetic resonance imaging. The reliability of IVD T2 measurements are yet to be established. IVD T2 was assessed nine times at regular intervals over 368 days on six anatomical slices centred at the lumbar spine using a spin-echo multi-echo sequence in 12 men. To assess repeatability, intra-class correlation co-efficients (ICCs), standard error of the measurement, minimal detectable difference and co-efficients of variation (CVs) were calculated along with their 95% confidence intervals. Bland-Altman analysis was also performed. ICCs were above 0.93, with the exception of nuclear T2 at L5/S1, where the ICC was 0.88. CVs of the central-slice nucleus sub-region ranged from 4.3% (average of all levels) to 10.1% for L5/S1 and between 2.2% to 3.2% for whole IVD T2 (1.8% for the average of all levels). Averaging between vertebral levels improved reliability. Reliability of measurements was least at L5/S1. ICCs of degenerated IVDs were lower. Test-retest reliability was excellent for whole IVD and good to excellent for IVD subregions. The findings help to establish the long-term repeatability of lumbar IVD T2 for the implementation of prospective studies and determination of significant changes within individuals.
Collapse
|
5
|
Hu X, Feng Z, Shen H, Zhang W, Huang J, Zheng Q, Wang Y. New MR-based measures for the evaluation of age-related lumbar paraspinal muscle degeneration. EUROPEAN SPINE JOURNAL : OFFICIAL PUBLICATION OF THE EUROPEAN SPINE SOCIETY, THE EUROPEAN SPINAL DEFORMITY SOCIETY, AND THE EUROPEAN SECTION OF THE CERVICAL SPINE RESEARCH SOCIETY 2021; 30:2577-2585. [PMID: 33740145 DOI: 10.1007/s00586-021-06811-6] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/30/2020] [Revised: 02/09/2021] [Accepted: 03/07/2021] [Indexed: 01/06/2023]
Abstract
PURPOSE Although signal intensity on T2W axial images is sensitive in detection of fatty infiltration to assess paraspinal muscle degeneration, it is affected by inhomogeneities of magnetic fields and individual variabilities. The purpose of this study was to propose reference adjusted signal measures on T2W axial images and determine their capacities in reflecting age-related lumbar paraspinal muscle degeneration. METHODS Lumbar MR images of 421 population-based subjects (177 men and 244 women, mean age 53.1 years, range 19.8-87.9 years) were studied. A custom software Spine Explore (Tulong 2.0) was used to automatically obtain paraspinal measurements of multifidus, erector spinae and psoas major. FCSA/TCSA was defined as functional cross-sectional area relative to total cross-sectional area of paraspinal muscle. Two new signal measures were canal-adjusted and cerebrospinal fluid (CSF)-adjusted signal, defined as the ratio between mean signal measurements and the mean signal of the canal and CSF. RESULTS The raw signal measurements of the paraspinal muscles were weakly correlated to age (r = 0.28-0.39, P < 0.001). When the signal of canal (r = 0.43-0.59, P < 0.001) or CSF (r = 0.45-0.61, P < 0.001) was used as reference, the correlations substantially increased. Signal measurements of three paraspinal muscles, adjusted or not, were strongly associated with Goutallier score (ρ = 0.60-0.65, P < 0.001) and FCSA/TCSA (r = -0.64 to -0.82, P < 0.001). Greater Goutallier score was associated with greater age (r = 0.38-0.60, P < 0.001), while Lumbar indentation value (LIV) not. CONCLUSION On routine T2W axial MR images the adjusted signal measurements using an internal reference of CSF or canal can better reflect age-related degenerative changes in the paraspinal muscles.
Collapse
Affiliation(s)
- Xiaojian Hu
- Spine Lab, Department of Orthopedic Surgery, School of Medicine, The First Affiliated Hospital, Zhejiang University, 79# Qingchun Road, Hangzhou, China
| | - Zhiyun Feng
- Spine Lab, Department of Orthopedic Surgery, School of Medicine, The First Affiliated Hospital, Zhejiang University, 79# Qingchun Road, Hangzhou, China
| | - Haotian Shen
- Spine Lab, Department of Orthopedic Surgery, School of Medicine, The First Affiliated Hospital, Zhejiang University, 79# Qingchun Road, Hangzhou, China
| | - Wenming Zhang
- Department of Orthopedic Surgery, Jinyun People's Hospital, Lishui, China
| | - Jiawei Huang
- Spine Lab, Department of Orthopedic Surgery, School of Medicine, The First Affiliated Hospital, Zhejiang University, 79# Qingchun Road, Hangzhou, China
| | - Qiangqiang Zheng
- Spine Lab, Department of Orthopedic Surgery, School of Medicine, The First Affiliated Hospital, Zhejiang University, 79# Qingchun Road, Hangzhou, China
| | - Yue Wang
- Spine Lab, Department of Orthopedic Surgery, School of Medicine, The First Affiliated Hospital, Zhejiang University, 79# Qingchun Road, Hangzhou, China.
| |
Collapse
|
6
|
LewandrowskI KU, Muraleedharan N, Eddy SA, Sobti V, Reece BD, Ramírez León JF, Shah S. Feasibility of Deep Learning Algorithms for Reporting in Routine Spine Magnetic Resonance Imaging. Int J Spine Surg 2020; 14:S86-S97. [PMID: 33298549 PMCID: PMC7735442 DOI: 10.14444/7131] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022] Open
Abstract
BACKGROUND Artificial intelligence is gaining traction in automated medical imaging analysis. Development of more accurate magnetic resonance imaging (MRI) predictors of successful clinical outcomes is necessary to better define indications for surgery, improve clinical outcomes with targeted minimally invasive and endoscopic procedures, and realize cost savings by avoiding more invasive spine care. OBJECTIVE To demonstrate the ability for deep learning neural network models to identify features in MRI DICOM datasets that represent varying intensities or severities of common spinal pathologies and injuries and to demonstrate the feasibility of generating automated verbal MRI reports comparable to those produced by reading radiologists. METHODS A 3-dimensional (3D) anatomical model of the lumbar spine was fitted to each of the patient's MRIs by a team of technicians. MRI T1, T2, sagittal, axial, and transverse reconstruction image series were used to train segmentation models by the intersection of the 3D model through these image sequences. Class definitions were extracted from the radiologist report for the central canal: (0) no disc bulge/protrusion/canal stenosis, (1) disc bulge without canal stenosis, (2) disc bulge resulting in canal stenosis, and (3) disc herniation/protrusion/extrusion resulting in canal stenosis. Both the left and right neural foramina were assessed with either (0) neural foraminal stenosis absent, or (1) neural foramina stenosis present. Reporting criteria for the pathologies at each disc level and, when available, the grading of severity were extracted, and a natural language processing model was used to generate a verbal and written report. These data were then used to train a set of very deep convolutional neural network models, optimizing for minimal binary cross-entropy for each classification. RESULTS The initial prediction validation of the implemented deep learning algorithm was done on 20% of the dataset, which was not used for artificial intelligence training. Of the 17,800 total disc locations for which MRI images and radiology reports were available, 14,720 were used to train the model, and 3560 were used to validate against. The convergence of validation accuracy achieved with the deep learning algorithm for the foraminal stenosis detector was 81% (sensitivity = 72.4.4%, specificity = 83.1%) after 25 complete iterations through the entire training dataset (epoch). The accuracy was 86.2% (sensitivity = 91.1%, specificity = 82.5%) for the central stenosis detector and 85.2% (sensitivity = 81.8%, specificity = 87.4%) for the disc herniation detector. CONCLUSIONS Deep learning algorithms may be used for routine reporting in spine MRI. There was a minimal disparity among accuracy, sensitivity, and specificity, indicating that the data were not overfitted to the training set. We concluded that variability in the training data tends to reduce overfitting and overtraining as the deep neural network models learn to focus on the common pathologies. Future studies should demonstrate the accuracy of deep neural network models and the predictive value of favorable clinical outcomes with intervention and surgery. LEVEL OF EVIDENCE 3. CLINICAL RELEVANCE Feasibility, clinical teaching, and evaluation study.
Collapse
Affiliation(s)
- Kai-Uwe LewandrowskI
- Staff Orthopaedic Spine Surgeon Center for Advanced Spine Care of Southern Arizona and Surgical Institute of Tucson, Tucson, Arizona
| | | | | | - Vikram Sobti
- Innovative Radiology, PC, River Forest, Illinois
| | - Brian D Reece
- The Spine and Orthopedic Academic Research Institute, Lewisville, Texas
| | - Jorge Felipe Ramírez León
- Fundación Universitaria Sanitas, Bogotá, Colombia, Research Team, Centro de Columna. Bogotá, Colombia, Centro de Cirugía de Mínima Invasión, CECIMIN-Clínica Reina Sofía, Bogotá, Colombia
| | | |
Collapse
|
7
|
Huang J, Shen H, Wu J, Hu X, Zhu Z, Lv X, Liu Y, Wang Y. Spine Explorer: a deep learning based fully automated program for efficient and reliable quantifications of the vertebrae and discs on sagittal lumbar spine MR images. Spine J 2020; 20:590-599. [PMID: 31759132 DOI: 10.1016/j.spinee.2019.11.010] [Citation(s) in RCA: 50] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/09/2019] [Revised: 11/17/2019] [Accepted: 11/18/2019] [Indexed: 02/03/2023]
Abstract
BACKGROUND CONTEXT Although quantitative measurements improve the assessment of disc degeneration, acquirement of quantitative measurements relies on manual segmentation on lumbar magnetic resonance images (MRIs), which may introduce subjective bias. To date, only a few semiautomatic systems have been developed to quantify important components on MRIs. PURPOSE To develop a deep learning based program (Spine Explorer) for automated segmentation and quantification of the vertebrae and intervertebral discs on lumbar spine MRIs. STUDY DESIGN Cross-sectional study. PATIENT SAMPLE The study was extended on the Hangzhou Lumbar Spine Study, a population-based study of mainland Chinese with focuses on lumbar degenerative changes. From this population-based database, 50 sets lumbar MRIs were randomly selected as training dataset, and another 50 as test dataset. OUTCOME MEASURES Regions of vertebrae and discs were manually segmented on T2W sagittal MRIs to train a convolutional neural network for automated segmentation. Intersection-over-union was calculated to evaluate segmentation performance. Computational definitions were proposed to acquire quantitative morphometric and signal measurements for lumbar vertebrae and discs. MRIs in the test dataset were automatically measured with Spine Explorer and manually with ImageJ. METHODS Intraclass correlation coefficient (ICC) were calculated to examine inter-software agreements. Correlations between disc measurements and Pfirrmann score as well as age were examined to assess measurement validity. RESULTS The trained Spine Explorer automatically segments and measures a lumbar MRI in half a second, with mean Intersection-over-union of 94.7% and 92.6% for the vertebra and disc, respectively. For both vertebra and disc measurements acquired with Spine Explorer and ImageJ, the agreements were excellent (ICC=0.81~1.00). Disc measurements significantly correlated to Pfirrmann score, and greater age was associated with greater anterior disc bulging area (r=0.35~0.44) and fewer signal measurements (r=-0.62~-0.77) as automatically acquired with Spine Explorer. CONCLUSIONS Spine Explorer is an efficient, accurate, and reliable tool to acquire comprehensive quantitative measurements for lumbar vertebra and disc. Implication of such deep learning based program can facilitate clinical studies of the lumbar spine.
Collapse
Affiliation(s)
- Jiawei Huang
- Spine Lab, Department of Orthopedic Surgery, the First Affiliated Hospital, Zhejiang University School of Medicine, The Second Floor of Building 3, 79# Qingchun Road, Hangzhou 310003, China
| | - Haotian Shen
- Spine Lab, Department of Orthopedic Surgery, the First Affiliated Hospital, Zhejiang University School of Medicine, The Second Floor of Building 3, 79# Qingchun Road, Hangzhou 310003, China
| | - Jialong Wu
- Spine Lab, Department of Orthopedic Surgery, the First Affiliated Hospital, Zhejiang University School of Medicine, The Second Floor of Building 3, 79# Qingchun Road, Hangzhou 310003, China
| | - Xiaojian Hu
- Spine Lab, Department of Orthopedic Surgery, the First Affiliated Hospital, Zhejiang University School of Medicine, The Second Floor of Building 3, 79# Qingchun Road, Hangzhou 310003, China
| | - Zhiwei Zhu
- Department of Radiology, Dongyang People's Hospital, Dongyang, China
| | - Xiaoqiang Lv
- Department of Orthopedic Surgery, Dongyang People's Hospital, Dongyang, China
| | - Yong Liu
- Department of Control Science, Institute of Cyber-Systems and Control, Zhejiang University, Hangzhou 310027, China.
| | - Yue Wang
- Spine Lab, Department of Orthopedic Surgery, the First Affiliated Hospital, Zhejiang University School of Medicine, The Second Floor of Building 3, 79# Qingchun Road, Hangzhou 310003, China.
| |
Collapse
|
8
|
Zhang W, Zhu J, Xu X, Fan G. Synthetic MRI of the lumbar spine at 3.0 T: feasibility and image quality comparison with conventional MRI. Acta Radiol 2020; 61:461-470. [PMID: 31522520 DOI: 10.1177/0284185119871670] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/04/2023]
Abstract
Background Synthetic magnetic resonance imaging (MRI), which can generate multiple morphologic MR images as well as quantitative maps from a single sequence, is not widely used in the spine at 3.0 T. Purpose To investigate the feasibility of synthetic MRI of the lumbar spine in clinical practice at 3.0 T. Material and Methods Eighty-four patients with lumbar diseases underwent conventional T1-weighted images, T2-weighted images, short-tau inversion recovery (STIR) images, and synthetic MRI of the lumbar spine at 3.0 T. The quantitative and qualitative image quality and agreement for detection of spinal lesions between conventional and synthetic MRI were compared by two radiologists. Results The signal-to-noise ratios of synthetic MRI showed an inferior image quality in the vertebrae and disc, whereas were higher for spinal canal and fat on the synthetic T1-weighted, T2-weighted, and STIR images. The contrast-to-noise ratios of the synthetic MRI was superior to conventional sequences, except for the vertebrae–disc contrast-to-noise ratio on T1-weighted imaging ( P = 0.005). Image quality assessments showed that synthetic MRI had greater STIR fat suppression ( P < 0.001) and fluid brightness ( P = 0.014), as well as higher degree of artifacts ( P < 0.001) and worse spatial resolution ( P = 0.002). The inter-method agreements for detection of spinal lesions were substantial to perfect (kappa, 0.614–0.925). Conclusion Synthetic MRI is a feasible method for lumbar spine imaging in a clinical setting at 3.0-T MR. It provides morphologic sequences with acceptable image quality, good agreement with conventional MRI for detection of spinal lesions and quantitative image maps with a slightly shorter acquisition time compared with conventional MRI.
Collapse
Affiliation(s)
- Weilan Zhang
- Department of Radiology, First Affiliated Hospital of China Medical University, Shenyang, Liaoning Province, PR China
| | - Jingyi Zhu
- Department of Radiology, First Affiliated Hospital of China Medical University, Shenyang, Liaoning Province, PR China
| | - Xiaohan Xu
- Department of Radiology, First Affiliated Hospital of China Medical University, Shenyang, Liaoning Province, PR China
| | - Guoguang Fan
- Department of Radiology, First Affiliated Hospital of China Medical University, Shenyang, Liaoning Province, PR China
| |
Collapse
|
9
|
The Michel Benoist and Robert Mulholland yearly European Spine Journal review: a survey of the "surgical and research" articles in the European Spine Journal, 2018. EUROPEAN SPINE JOURNAL : OFFICIAL PUBLICATION OF THE EUROPEAN SPINE SOCIETY, THE EUROPEAN SPINAL DEFORMITY SOCIETY, AND THE EUROPEAN SECTION OF THE CERVICAL SPINE RESEARCH SOCIETY 2019; 28:1-9. [PMID: 30604293 DOI: 10.1007/s00586-018-5856-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/08/2018] [Accepted: 12/08/2018] [Indexed: 10/27/2022]
|