Ecology & computer audition: Applications of audio technology to monitor organisms and environment

Among the 17 Sustainable Development Goals (SDGs) proposed within the 2030 Agenda and adopted by all the United Nations member states, the 13th SDG is a call for action to combat climate change. Moreover, SDGs 14 and 15 claim the protection and conservation of life below water and life on land, respectively. In this work, we provide a literature-founded overview of application areas, in which computer audition - a powerful but in this context so far hardly considered technology, combining audio signal processing and machine intelligence - is employed to monitor our ecosystem with the potential to identify ecologically critical processes or states. We distinguish between applications related to organisms, such as species richness analysis and plant health monitoring, and applications related to the environment, such as melting ice monitoring or wildfire detection. This work positions computer audition in relation to alternative approaches by discussing methodological strengths and limitations, as well as ethical aspects. We conclude with an urgent call to action to the research community for a greater involvement of audio intelligence methodology in future ecosystem monitoring approaches.

HEAR4Health: a blueprint for making computer audition a staple of modern healthcare

Recent years have seen a rapid increase in digital medicine research in an attempt to transform traditional healthcare systems to their modern, intelligent, and versatile equivalents that are adequately equipped to tackle contemporary challenges. This has led to a wave of applications that utilise AI technologies; first and foremost in the fields of medical imaging, but also in the use of wearables and other intelligent sensors. In comparison, computer audition can be seen to be lagging behind, at least in terms of commercial interest. Yet, audition has long been a staple assistant for medical practitioners, with the stethoscope being the quintessential sign of doctors around the world. Transforming this traditional technology with the use of AI entails a set of unique challenges. We categorise the advances needed in four key pillars: Hear, corresponding to the cornerstone technologies needed to analyse auditory signals in real-life conditions; Earlier, for the advances needed in computational and data efficiency; Attentively, for accounting to individual differences and handling the longitudinal nature of medical data; and, finally, Responsibly, for ensuring compliance to the ethical standards accorded to the field of medicine. Thus, we provide an overview and perspective of HEAR4Health: the sketch of a modern, ubiquitous sensing system that can bring computer audition on par with other AI technologies in the strive for improved healthcare systems.

NeuroCellCentreDB: Exploring a Novel Dataset for Neuron-like Cell Centre Detection with Deep Neural Networks

The manipulation and stimulation of cell growth is invaluable for neuroscience research such as brain-machine interfaces or applications of neural tissue engineering. For the implementation of such research avenues, in particular the analysis of cells' migration behaviour, and accordingly, the determination of cell positions on microscope images is essential, causing a current need for labour-intensive, manual annotation efforts of the cell positions. In an attempt towards automation of the required annotation efforts, we i) introduce NeuroCellCentreDB, a novel dataset of neuron-like cells on microscope images with annotated cell centres, ii) evaluate a common (bounding box-based) object detector, faster region-based convolutional neural network (FRCNN), for the task at hand, and iii) design and test a fully convolutional neural network, with the specific goal of cell centre detection. We achieve an F1 score of up to 0.766 on the test data with a tolerance radius of 16 pixels. Our code and dataset are publicly available.

Introduction of the Simple One-Step stool Xpert Ultra method to detect TB in children and adults

SETTING: In 2020, the National TB Programme (NTP) of Vietnam conducted an implementation pilot of the Simple One-Step (SOS) stool processing method using Xpert^® MTB/RIF Ultra (Ultra) among children and people living with HIV (PLHIV) with signs and symptoms of TB.DESIGN and OBJECTIVES: Using data from this pilot and collecting information on healthcare workers´ (HCWs) perceptions, we assessed the feasibility, acceptability and potential impact of routine stool testing for TB.RESULTS: HCWs perceived collection of stools from children as least stressful of all sample types, stool processing as acceptable and the SOS stool method as easy to perform. After a 3-month induction period, the proportion of initial non-determinate Ultra stool tests was less than 5%. Combined Ultra testing of a respiratory sample and stool resulted in an increase in the proportion of bacteriologically confirmed TB among PLHIV and children by respectively 4.1% (95% CI 1.6-6.6) and 3.9% (95% CI 1.6-6.2). Among children, Mycobacterium tuberculosis was more often detected in stool (26.1%) than in respiratory samples (23.4%) (P = 0.06), including one child with rifampicin resistance.CONCLUSION: Stool testing can be feasibly implemented both in adult PLHIV and in children in routine settings, providing a non-invasive alternative sample type for the diagnosis of TB for patients who cannot produce sputum.

Fatigue Prediction in Outdoor Running Conditions using Audio Data

Although running is a common leisure activity and a core training regiment for several athletes, between 29% and 79% of runners sustain an overuse injury each year. These injuries are linked to excessive fatigue, which alters how someone runs. In this work, we explore the feasibility of modelling the Borg received perception of exertion (RPE) scale (range: [6]-[19] [20]), a well-validated subjective measure of fatigue, using audio data captured in realistic outdoor environments via smartphones attached to the runners' arms. Using convolutional neural networks (CNNs) on log-Mel spectrograms, we obtain a mean absolute error (MAE) of 2.35 in subject-dependent experiments, demonstrating that audio can be effectively used to model fatigue, while being more easily and non-invasively acquired than by signals from other sensors.

FIN-PRINT a fully-automated multi-stage deep-learning-based framework for the individual recognition of killer whales

Biometric identification techniques such as photo-identification require an array of unique natural markings to identify individuals. From 1975 to present, Bigg's killer whales have been photo-identified along the west coast of North America, resulting in one of the largest and longest-running cetacean photo-identification datasets. However, data maintenance and analysis are extremely time and resource consuming. This study transfers the procedure of killer whale image identification into a fully automated, multi-stage, deep learning framework, entitled FIN-PRINT. It is composed of multiple sequentially ordered sub-components. FIN-PRINT is trained and evaluated on a dataset collected over an 8-year period (2011-2018) in the coastal waters off western North America, including 121,000 human-annotated identification images of Bigg's killer whales. At first, object detection is performed to identify unique killer whale markings, resulting in 94.4% recall, 94.1% precision, and 93.4% mean-average-precision (mAP). Second, all previously identified natural killer whale markings are extracted. The third step introduces a data enhancement mechanism by filtering between valid and invalid markings from previous processing levels, achieving 92.8% recall, 97.5%, precision, and 95.2% accuracy. The fourth and final step involves multi-class individual recognition. When evaluated on the network test set, it achieved an accuracy of 92.5% with 97.2% top-3 unweighted accuracy (TUA) for the 100 most commonly photo-identified killer whales. Additionally, the method achieved an accuracy of 84.5% and a TUA of 92.9% when applied to the entire 2018 image collection of the 100 most common killer whales. The source code of FIN-PRINT can be adapted to other species and will be publicly available.

Acceptability, feasibility, and likelihood of stakeholders implementing the novel BPaL regimen to treat extensively drug-resistant tuberculosis patients

BACKGROUND

BPaL, a 6 month oral regimen composed of bedaquiline, pretomanid, and linezolid for treating extensively drug-resistant tuberculosis (XDR-TB) is a potential alternative for at least 20 months of individualized treatment regimens (ITR). The ITR has low tolerability, treatment adherence, and success rates, and hence to limit patient burden, loss to follow-up and the emergence of resistance it is essential to implement new DR-TB regimens. The objective of this study was to assess the acceptability, feasibility, and likelihood of implementing BPaL in Indonesia, Kyrgyzstan, and Nigeria.

METHODS

We conducted a concurrent mixed-methods study among a cross-section of health care workers, programmatic and laboratory stakeholders between May 2018 and May 2019. We conducted semi-structured interviews and focus group discussions to assess perceptions on acceptability and feasibility of implementing BPaL. We determined the proportions of a recoded 3-point Likert scale (acceptable; neutral; unacceptable), as well as the overall likelihood of implementing BPaL (likely; neutral; unlikely) that participants graded per regimen, pre-defined aspect and country. We analysed the qualitative results using a deductive framework analysis.

RESULTS

In total 188 stakeholders participated in this study: 63 from Kyrgyzstan, 51 from Indonesia, and 74 from Nigeria The majority were health care workers (110). Overall, 88% (146/166) of the stakeholders would likely implement BPaL once available. Overall acceptability for BPaL was high, especially patient friendliness was often rated as acceptable (93%, 124/133). In contrast, patient friendliness of the ITR was rated as acceptable by 45%. Stakeholders appreciated that BPaL would reduce workload and financial burden on the health care system. However, several stakeholders expressed concerns regarding BPaL safety (monitoring), long-term efficacy, and national regulatory requirements regarding introduction of the regimen. Stakeholders stressed the importance of addressing current health systems constraints as well, especially in treatment and safety monitoring systems.

CONCLUSIONS

Acceptability and feasibility of the BPaL regimen is high among TB stakeholders in Indonesia, Kyrgyzstan, and Nigeria. The majority is willing to start using BPaL as the standard of care for eligible patients despite country-specific health system constraints.

Introducing bedaquiline: experiences from the Challenge TB Project

BACKGROUND: Drug-resistant tuberculosis (DR-TB) remains a global public health crisis. In 2013, the World Health Organization recommended the introduction of bedaquiline (BDQ) for eligible DR-TB patients.METHODS: We conducted a retrospective review and analyses of project reports from 2016 to mid-2019 on the processes, activities implemented, available results on enrolment and interim treatment outcomes, across the 23 Challenge TB (CTB) supported countries.RESULTS: Initial introduction of BDQ-containing regimens in the 23 CTB-supported countries took on average 2 years, with subsequent nation-wide scale-up achieved in Ethiopia and Kyrgyzstan within a short time period. Successful implementation required critical interventions including advocacy, revision of policies and guidelines, capacity building of health care workers, and strengthening of laboratory networks. The number of countries providing BDQ increased from 9 to 23; 9398 patients were enrolled on bedaquiline containing regimens; 71% were culture-negative after 6 months of treatment; and the number of countries reporting serious adverse events increased (from 5 to 18). Major challenges included limited in-country coordination with drug regulatory agencies, unrealistic quantification and drug ordering, weak laboratory networks and reporting systems for drug safety.CONCLUSION: BDQ introduction required a systematic and programmatic approach. The initial time investment helped achieve initial introduction and scale-up of coverage, ownership and sustainability by National TB Programmes.

New Avenues in Audio Intelligence: Towards Holistic Real-life Audio Understanding

Computer audition (i.e., intelligent audio) has made great strides in recent years; however, it is still far from achieving holistic hearing abilities, which more appropriately mimic human-like understanding. Within an audio scene, a human listener is quickly able to interpret layers of sound at a single time-point, with each layer varying in characteristics such as location, state, and trait. Currently, integrated machine listening approaches, on the other hand, will mainly recognise only single events. In this context, this contribution aims to provide key insights and approaches, which can be applied in computer audition to achieve the goal of a more holistic intelligent understanding system, as well as identifying challenges in reaching this goal. We firstly summarise the state-of-the-art in traditional signal-processing-based audio pre-processing and feature representation, as well as automated learning such as by deep neural networks. This concerns, in particular, audio interpretation, decomposition, understanding, as well as ontologisation. We then present an agent-based approach for integrating these concepts as a holistic audio understanding system. Based on this, concluding, avenues are given towards reaching the ambitious goal of 'holistic human-parity' machine listening abilities.

Estimation of subnational tuberculosis burden: generation and application of a new tool in Indonesia

SETTING: In many high tuberculosis (TB) burden countries, there is substantial geographical heterogeneity in TB burden. In addition, decisions on TB funding and policy are highly decentralised. Subnational estimates of burden, however, are usually unavailable for planning and target setting.OBJECTIVE and DESIGN: We developed a statistical model termed SUBsET to estimate the distribution of the national TB incidence through a weighted score using selected variables, and applied the model to the 514 districts in Indonesia, which have substantial policy and budgetary autonomy in TB. Estimated incidence was compared to reported facility and domicile-based notifications to estimate the case detection rate (CDR). Local stakeholders led model development and dissemination.RESULTS: The final SUBsET model included district population size, level of urbanisation, socio-economic indicators (living floor space and high school completion), human immunodeficiency virus prevalence and air pollution. We estimated district-level TB incidence to be between 201 and 2,485/100 000/year. The facility-based CDR varied between 0 and 190%, with high variation between neighbouring districts, suggesting strong cross-district health utilisation, which was confirmed by domicile-based CDR estimation. SUBsET results informed district-level TB action plans across Indonesia.CONCLUSION: The SUBsET model could be used to estimate the subnational burden in high-burden countries and inform TB policymaking at the relevant decentralised administrative level.

First joint meeting of three European tuberculosis networks

On 29 May 2013 three European tuberculosis (TB) networks met for the first time to discuss TB prevention, control and care in the World Health Organization (WHO) European Region including the European Union (EU). This meeting, which took place in The Hague, the Netherlands, provided a unique opportunity to discuss progress with the implementation of the Berlin Declaration on TB [1], the European Centre for Disease Prevention and Control (ECDC) Framework Action Plan to fight tuberculosis in the EU [2,3], and the Consolidated Action Plan to prevent and combat multidrug- and extensively drug-resistant tuberculosis (M/XDR-TB) in the WHO European Region [4]. Surveillance focal points, laboratory experts, and National TB Programme Managers (NTPs) exchanged lessons learned and discussed next steps to reach the targets defined in the plans.

WHO guidelines for the programmatic management of drug-resistant tuberculosis: 2011 update

The production of guidelines for the management of drug-resistant tuberculosis (TB) fits the mandate of the World Health Organization (WHO) to support countries in the reinforcement of patient care. WHO commissioned external reviews to summarise evidence on priority questions regarding case-finding, treatment regimens for multidrug-resistant TB (MDR-TB), monitoring the response to MDR-TB treatment, and models of care. A multidisciplinary expert panel used the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to develop recommendations. The recommendations support the wider use of rapid drug susceptibility testing for isoniazid and rifampicin or rifampicin alone using molecular techniques. Monitoring by sputum culture is important for early detection of failure during treatment. Regimens lasting ≥ 20 months and containing pyrazinamide, a fluoroquinolone, a second-line injectable drug, ethionamide (or prothionamide), and either cycloserine or p-aminosalicylic acid are recommended. The guidelines promote the early use of antiretroviral agents for TB patients with HIV on second-line drug regimens. Systems that primarily employ ambulatory models of care are recommended over others based mainly on hospitalisation. Scientific and medical associations should promote the recommendations among practitioners and public health decision makers involved in MDR-TB care. Controlled trials are needed to improve the quality of existing evidence, particularly on the optimal composition and duration of MDR-TB treatment regimens.

Household expenditure and tuberculosis prevalence in VietNam: prediction by a set of household indicators

OBJECTIVE

To study the association between TB and household expenditure in a nationwide TB prevalence survey in Viet Nam using nine household characteristics.

METHOD

To assess the prevalence of TB in Viet Nam, a nationwide stratified cluster sample survey was conducted from 2006 to 2007. Nine household characteristics used in the second Viet Nam Living Standards Survey (VLSS) were scored per household. In the VLSS dataset, we regressed these nine characteristics against household expenditure per capita, and used the coefficients to predict household expenditure level (in quintiles) in our survey and assess its relation with TB prevalence.

RESULTS

The prevalence of bacteriologically confirmed TB was 307 per 100,000 population in persons aged ≥ 15 years (95%CI 249-366). After adjustment for confounders, prevalence was found to be associated with household expenditure level: the rate was 2.5 times higher for those in the lowest household expenditure quintile (95%CI 1.6-3.9) than those in the highest quintile.

CONCLUSION

With a set of nine household characteristics, we were able to predict household expenditure level fairly accurately. There was a significant association between TB prevalence rates and estimated household expenditure level, showing that TB is related to poverty in Viet Nam.

Liquid culture for Mycobacterium tuberculosis: proceed, but with caution

Attempts to improve the diagnosis of tuberculosis (TB) in high-burden countries has resulted in significant funding and initiatives to change the method of diagnosis of TB from light microscopy supplemented with X-ray to a sophisticated diagnostic algorithm based on the latest technological innovations. Such activities are overdue and should be welcomed, but the lack of skills and support available to interpret and use the results represents a danger. The introduction of new diagnostic methods, particularly liquid culture, should be carefully structured according to the local situation, failing which frustration and the disruption of previously underdeveloped but adequately functioning laboratories may result.

EuroSCORE as a predictor for complications and outcome

OBJECTIVE

The purpose of this study was to evaluate the usefulness of EuroSCORE in terms of prediction of the outcome as a result of preoperative information in a cohort of patients.

METHODS

We analyzed the data from 751 patients treated between Jan 1 and Dec 31, 1998. We used contingency tables and applied methods of discriminant analysis for the evaluation.

RESULTS

Compared to the 14.799 patients from whose data the EuroSCORE system had originally been derived [1,2], we had a smaller portion of patients in the low-risk group (24.5% vs. 30.6%), a comparable portion of patients in the medium-risk group (42.2% vs. 40.5%) and a higher proportion of patients in the high-risk group (33.2% vs. 29.0%). This difference in the risk distributions was highly significant (p < 0.001). The application of the EuroSCORE system showed that deaths only occurred in the high-risk group in our hospital. We had 36 deaths amongst the patients, which gives an overall mortality rate of 4.9% and a mortality rate of 1.6% in the medium-risk group and of 12.4% in the high-risk group. The discriminant analysis showed that with the EuroSCORE as single predicting variable - virtually all survivors were classified correctly (giving a specificity of 99.0%). The sensitivity was rather low at only 25%; however, this means that many of the high-risk patients in our clinic survived. The total misclassification rate, however, was 12.7%. Additionally, we found that a higher score value predicts a significantly higher probability for perioperative complications and also results in significantly increased average times for the bypass and total time for surgery.

CONCLUSION

The EuroSCORE is a valuable score system for the prediction of the overall outcome of patients following open heart surgery, and is easy to use. As far as perioperative complications are concerned, however, some specifications seem desirable.

[Image analysis system for detection facial paralysis]

BACKGROUND

Evaluation of facial nerve paresis depends on visual assessment and naturally differs from examiner to examiner. An objective measurement instrument is presented.

PATIENTS AND METHOD

Facial features are automatically localized by a parametric face model in videos of a face during relaxation and exercises. Gray-level information is analyzed by a special steerable filter and used to identify symmetries. The computer system was tested in 19 individuals.

RESULTS

Automatic localization of facial features such as the upper arc of the head and ears was correct in 95%, the eyes in 82%, and the mouth in 73%. Lid paresis was correctly recognized in seven of ten (70%) and oral paresis in 10 of 12 (83%) cases. Unaffected eyelid movements were identified in eight of nine (89%) and healthy oral regions in all seven (100%) cases.

CONCLUSION

The computer system presented is able to automatically localize facial features and to identify facial nerve paresis. It is a considerable step toward automatic and objective grading of facial nerve paresis.

The apical membrane of intestinal brush cells possesses a specialised, but species-specific, composition of glycoconjugates--on-section and in vivo lectin labelling in rats, guinea-pigs and mice

Brush cells are specialised epithelial cells that are assumed to represent chemoreceptors of the digestive tract. They comprise a small population of the epithelial cells lining the intestine, possess a unique ultrastructure and, in many aspects, resemble the receptor cells of taste buds. To characterise glycoconjugates possibly involved in a sensory function, we investigated brush cells in the small intestine of three species using lectin histochemistry in confocal light and thin-section electron microscopy. Brush cells of rats were selectively labelled by the sialic acid-specific lectin Maackia amurensis agglutinin, those of guinea-pigs by the D-galactose-specific lectin Bandeiraea simplicifolia agglutinin, isolectin B4 and those of mice by the L-fucose-specific lectin Ulex europaeus agglutinin lectin I. Lectin binding sites were consistently located in the glycocalyx of the apical membrane and in that of cytoplasmic vesicles. In vivo lectin labelling revealed that the glycoconjugates of the apical membrane are accessible under physiological conditions, that brush cells do not endocytose and that they probably possess a high membrane turnover rate. The results show that specialisations exist in the composition of glycoconjugates forming the glycocalyx of brush cells in all species investigated. The presence of brush cell-specific glycoconjugates would be in accordance with the current hypothesis of a receptive function of brush cells. Differences in the specific glycosylation patterns among rats, guinea-pigs and mice indicate that species-specific adaptations exist.

Brush cells of the mouse intestine possess a specialized glycocalyx as revealed by quantitative lectin histochemistry. Further evidence for a sensory function

Brush cells occur in the epithelium of the small intestine and in various other epithelia of endodermal origin. Ultrastructural and histochemical characteristics suggest that they represent sensory cells. Because the apical membrane of brush cells might be involved in and specialized for (chemo-)receptive functions, we investigated the composition of the glycocalyx and compared it with that of enterocytes. Ultrathin sections of murine small intestine were labeled with a panel of eight lectins. Their binding sites in the brush border and on vesicles of the apical cytoplasm were detected by colloidal gold and quantified using image analysis. The glycocalyx of brush cells contained significantly higher amounts of l-fucose residues than that of enterocytes, as detected by the lectins UEA-I and LTA. In contrast, most of the other lectins bound more avidly to the glycocalyx of enterocytes. The cytoplasmic vesicles closely resembled the apical membrane in their labeling pattern. Quantitation of the brush cells' distribution revealed that the epithelia of the Peyer's patches contained 10-fold higher numbers of brush cells than the small intestinal mucosa distant from lymphoid tissue. We conclude that brush cells possess a glycocalyx with a specialized composition and differ significantly from enterocytes. Because similar peculiarities of the apical membrane have previously been described for sensory cells of the olfactory and gustatory organs, this study provides further evidence in favor of a sensory function of brush cells.

Result of treatment in children with tuberculosis. Evaluation of indications for and results of treatment in a 5 years cohort of children treated for tuberculosis in Turiani Hospital Tanzania

Between January 1983 and January 1988 a total of 146 children started tuberculosis (TB) treatment in Turiani Hospital, Tanzania. During the treatment period 16 children died and another 16 were transferred out. From the remaining 114 children, 84 could be traced and were visited at home. Out of this group 85% were found to be in good clinical condition, whereas 7% had a moderate condition and 1% were seriously ill. Tuberculin sensitivity testing has been carried out in 53 children from the follow-up group. The indications for treatment and the results of the follow-up study are discussed.