Mathematical Content Browsing for Print-Disabled Readers Based on Virtual-World Exploration and Audio-Visual Sensory Substitution

Documents containing mathematical content remain largely inaccessible to blind and visually impaired readers because they are predominantly published as untagged PDFs which do not include the semantic data necessary for effective accessibility. Equations in such documents consist of text interlaced with lines and other graphical elements, and cannot be interpreted using a screen reader. We present a browsing approach for print-disabled readers specifically aimed at such mathematical content. This approach draws on the navigational mechanisms often used to explore the virtual worlds of text adventure games with audio-visual sensory substitution for graphical content. The relative spatial placement of the elements of an equation are represented as a virtual world, so that the reader can navigate between elements. Text elements are announced conventionally using synthesised speech while graphical elements, such as roots and fraction lines, are rendered using a modification of the vOICe algorithm. The virtual world allows the reader to interactively discover the spatial structure of the equation, while the rendition of graphical elements as sound allows the shape and identity of elements that cannot be synthesised as speech to be discovered and recognised. The browsing approach was evaluated by eleven blind and fourteen sighted participants in a user trial that included identifying twelve equations extracted from PDF documents. Overall, equations were identified completely correctly in 78% of cases (74% and 83% respectively for blind and sighted subjects). If partial correctness is considered, the performance is substantially higher. Feedback from the blind subjects indicated that the technique allows spatial information and graphical detail to be discovered. We conclude that the integration of a spatial model represented as a virtual world in conjunction with audio-visual sensory substitution for non-textual elements can be an effective way for blind and visually impaired readers to read currently inaccessible mathematical content in PDF documents.

Automatic Non-Invasive Cough Detection based on Accelerometer and Audio Signals

We present an automatic non-invasive way of detecting cough events based on both accelerometer and audio signals. The acceleration signals are captured by a smartphone firmly attached to the patient’s bed, using its integrated accelerometer. The audio signals are captured simultaneously by the same smartphone using an external microphone. We have compiled a manually-annotated dataset containing such simultaneously-captured acceleration and audio signals for approximately 6000 cough and 68000 non-cough events from 14 adult male patients. Logistic regression (LR), support vector machine (SVM) and multilayer perceptron (MLP) classifiers provide a baseline and are compared with three deep architectures, convolutional neural network (CNN), long short-term memory (LSTM) network, and residual-based architecture (Resnet50) using a leave-one-out cross-validation scheme. We find that it is possible to use either acceleration or audio signals to distinguish between coughing and other activities including sneezing, throat-clearing, and movement on the bed with high accuracy. However, in all cases, the deep neural networks outperform the shallow classifiers by a clear margin and the Resnet50 offers the best performance, achieving an area under the ROC curve (AUC) exceeding 0.98 and 0.99 for acceleration and audio signals respectively. While audio-based classification consistently offers better performance than acceleration-based classification, we observe that the difference is very small for the best systems. Since the acceleration signal requires less processing power, and since the need to record audio is sidestepped and thus privacy is inherently secured, and since the recording device is attached to the bed and not worn, an accelerometer-based highly accurate non-invasive cough detector may represent a more convenient and readily accepted method in long-term cough monitoring.

An African violin – The feasibility of using indigenous wood from southern Africa as tonewood

The wood used to make musical instruments needs to have particular properties. Depending on its function, such as a soundboard for string instruments or the body of a wind instrument, different properties are desirable to obtain the best musical quality. Several different classification schemes exist that correlate physical and mechanical properties of wood to define desirable ranges for tonewoods, and to allow suitable wood species to be chosen. The physical and mechanical properties of various wood species indigenous to southern Africa were characterised and then assessed in terms of their suitability for violin construction using these classification schemes. The results of this analysis show that the most suitable of the wood species assessed are yellowwood and sapele. These were subsequently used by a professional luthier to build an ‘African’ violin. The sound quality of this instrument was determined subjectively through performances to an audience and more objectively via spectral analysis of audio recordings. This analysis shows clear differences in the relative magnitude of the harmonics between the violin made from indigenous wood and an instrument made with conventional wood species. Despite the differences, yellowwood and sapele were found to be suitable tonewoods, resulting in an instrument with a unique sound.

Automatic cough classification for tuberculosis screening in a real-world environment

Objective.The automatic discrimination between the coughing sounds produced by patients with tuberculosis (TB) and those produced by patients with other lung ailments.Approach.We present experiments based on a dataset of 1358 forced cough recordings obtained in a developing-world clinic from 16 patients with confirmed active pulmonary TB and 35 patients suffering from respiratory conditions suggestive of TB but confirmed to be TB negative. Using nested cross-validation, we have trained and evaluated five machine learning classifiers: logistic regression (LR), support vector machines, k-nearest neighbour, multilayer perceptrons and convolutional neural networks.Main Results.Although classification is possible in all cases, the best performance is achieved using LR. In combination with feature selection by sequential forward selection, our best LR system achieves an area under the ROC curve (AUC) of 0.94 using 23 features selected from a set of 78 high-resolution mel-frequency cepstral coefficients. This system achieves a sensitivity of 93% at a specificity of 95% and thus exceeds the 90% sensitivity at 70% specificity specification considered by the World Health Organisation (WHO) as a minimal requirement for a community-based TB triage test.Significance.The automatic classification of cough audio sounds, when applied to symptomatic patients requiring investigation for TB, can meet the WHO triage specifications for the identification of patients who should undergo expensive molecular downstream testing. This makes it a promising and viable means of low cost, easily deployable frontline screening for TB, which can benefit especially developing countries with a heavy TB burden.

COVID-19 cough classification using machine learning and global smartphone recordings

We present a machine learning based COVID-19 cough classifier which can discriminate COVID-19 positive coughs from both COVID-19 negative and healthy coughs recorded on a smartphone. This type of screening is non-contact, easy to apply, and can reduce the workload in testing centres as well as limit transmission by recommending early self-isolation to those who have a cough suggestive of COVID-19. The datasets used in this study include subjects from all six continents and contain both forced and natural coughs, indicating that the approach is widely applicable. The publicly available Coswara dataset contains 92 COVID-19 positive and 1079 healthy subjects, while the second smaller dataset was collected mostly in South Africa and contains 18 COVID-19 positive and 26 COVID-19 negative subjects who have undergone a SARS-CoV laboratory test. Both datasets indicate that COVID-19 positive coughs are 15%–20% shorter than non-COVID coughs. Dataset skew was addressed by applying the synthetic minority oversampling technique (SMOTE). A leave-p-out cross-validation scheme was used to train and evaluate seven machine learning classifiers: logistic regression (LR), k-nearest neighbour (KNN), support vector machine (SVM), multilayer perceptron (MLP), convolutional neural network (CNN), long short-term memory (LSTM) and a residual-based neural network architecture (Resnet50). Our results show that although all classifiers were able to identify COVID-19 coughs, the best performance was exhibited by the Resnet50 classifier, which was best able to discriminate between the COVID-19 positive and the healthy coughs with an area under the ROC curve (AUC) of 0.98. An LSTM classifier was best able to discriminate between the COVID-19 positive and COVID-19 negative coughs, with an AUC of 0.94 after selecting the best 13 features from a sequential forward selection (SFS). Since this type of cough audio classification is cost-effective and easy to deploy, it is potentially a useful and viable means of non-contact COVID-19 screening.

Wild-type and mutant (G2019S) leucine-rich repeat kinase 2 (LRRK2) associate with subunits of the translocase of outer mitochondrial membrane (TOM) complex

Leucine-rich repeat kinase 2 (LRRK2) is important in various cellular processes including mitochondrial homeostasis and mutations in this gene lead to Parkinson's disease (PD). However, the full spectrum of LRRK2's functions remain to be elucidated. The translocase of outer mitochondrial membrane (TOM) complex is essential for the import of almost all nuclear-encoded mitochondrial proteins and is fundamental for cellular survival. Using co-immunoprecipitation, super-resolution structured illumination microscopy (SR-SIM), and 3D virtual reality (VR) assisted co-localization analysis techniques we show that wild-type and mutant (G2019S) LRRK2 associate and co-localize with subunits of the TOM complex, either under basal (dimethyl sulfoxide, DMSO) or stress-induced (carbonyl cyanide m-chlorophenyl hydrazine, CCCP) conditions. Interestingly, LRRK2 interacted with TOM40 under both DMSO and CCCP conditions, and when the PD causing mutation, G2019S was introduced, the association was not altered. Moreover, overexpression of G2019S LRRK2 resulted in the formation of large, perinuclear aggregates that co-localized with the TOM complex. Taken together, this is the first study to show that both WT and mutant LRRK2 associate with the TOM complex subunits. These findings provide additional evidence for LRRK2's role in mitochondrial function which has important implications for its role in PD pathogenesis.

Clustering Acoustic Segments Using Multi-Stage Agglomerative Hierarchical Clustering

Agglomerative hierarchical clustering becomes infeasible when applied to large datasets due to its O(N2) storage requirements. We present a multi-stage agglomerative hierarchical clustering (MAHC) approach aimed at large datasets of speech segments. The algorithm is based on an iterative divide-and-conquer strategy. The data is first split into independent subsets, each of which is clustered separately. Thus reduces the storage required for sequential implementations, and allows concurrent computation on parallel computing hardware. The resultant clusters are merged and subsequently re-divided into subsets, which are passed to the following iteration. We show that MAHC can match and even surpass the performance of the exact implementation when applied to datasets of speech segments.