A Quick and Easy Way to Estimate Entropy and Mutual Information for Neuroscience

First activity and interactions in thalamus and cortex using raw single-trial EEG and MEG elicited by somatosensory stimulation

Introduction

One of the primary motivations for studying the human brain is to comprehend how external sensory input is processed and ultimately perceived by the brain. A good understanding of these processes can promote the identification of biomarkers for the diagnosis of various neurological disorders; it can also provide ways of evaluating therapeutic techniques. In this work, we seek the minimal requirements for identifying key stages of activity in the brain elicited by median nerve stimulation.

Methods

We have used a priori knowledge and applied a simple, linear, spatial filter on the electroencephalography and magnetoencephalography signals to identify the early responses in the thalamus and cortex evoked by short electrical stimulation of the median nerve at the wrist. The spatial filter is defined first from the average EEG and MEG signals and then refined using consistency selection rules across ST. The refined spatial filter is then applied to extract the timecourses of each ST in each targeted generator. These ST timecourses are studied through clustering to quantify the ST variability. The nature of ST connectivity between thalamic and cortical generators is then studied within each identified cluster using linear and non-linear algorithms with time delays to extract linked and directional activities. A novel combination of linear and non-linear methods provides in addition discrimination of influences as excitatory or inhibitory.

Results

Our method identifies two key aspects of the evoked response. Firstly, the early onset of activity in the thalamus and the somatosensory cortex, known as the P14 and P20 in EEG and the second M20 for MEG. Secondly, good estimates are obtained for the early timecourse of activity from these two areas. The results confirm the existence of variability in ST brain activations and reveal distinct and novel patterns of connectivity in different clusters.

Discussion

It has been demonstrated that we can extract new insights into stimulus processing without the use of computationally costly source reconstruction techniques which require assumptions and detailed modeling of the brain. Our methodology, thanks to its simplicity and minimal computational requirements, has the potential for real-time applications such as in neurofeedback systems and brain-computer interfaces.

Kernel-weighted contribution: a method of visual attribution for 3D deep learning segmentation in medical imaging

Purpose

Explaining deep learning model decisions, especially those for medical image segmentation, is a critical step toward the understanding and validation that will enable these powerful tools to see more widespread adoption in healthcare. We introduce kernel-weighted contribution, a visual explanation method for three-dimensional medical image segmentation models that produces accurate and interpretable explanations. Unlike previous attribution methods, kernel-weighted contribution is explicitly designed for medical image segmentation models and assesses feature importance using the relative contribution of each considered activation map to the predicted segmentation.

Approach

We evaluate our method on a synthetic dataset that provides complete knowledge over input features and a comprehensive explanation quality metric using this ground truth. Our method and three other prevalent attribution methods were applied to five different model layer combinations to explain segmentation predictions for 100 test samples and compared using this metric.

Results

Kernel-weighted contribution produced superior explanations of obtained image segmentations when applied to both encoder and decoder sections of a trained model as compared to other layer combinations (p < 0.0005 ). In between-method comparisons, kernel-weighted contribution produced superior explanations compared with other methods using the same model layers in four of five experiments (p < 0.0005 ) and showed equivalently superior performance to GradCAM++ when only using non-transpose convolution layers of the model decoder (p = 0.008 ).

Conclusion

The reported method produced explanations of superior quality uniquely suited to fully utilize the specific architectural considerations present in image and especially medical image segmentation models. Both the synthetic dataset and implementation of our method are available to the research community.

Data Compression Versus Signal Fidelity Tradeoff in Wired-OR Analog-to-Digital Compressive Arrays for Neural Recording

Future high-density and high channel count neural interfaces that enable simultaneous recording of tens of thousands of neurons will provide a gateway to study, restore and augment neural functions. However, building such technology within the bit-rate limit and power budget of a fully implantable device is challenging. The wired-OR compressive readout architecture addresses the data deluge challenge of a high channel count neural interface using lossy compression at the analog-to-digital interface. In this article, we assess the suitability of wired-OR for several steps that are important for neuroengineering, including spike detection, spike assignment and waveform estimation. For various wiring configurations of wired-OR and assumptions about the quality of the underlying signal, we characterize the trade-off between compression ratio and task-specific signal fidelity metrics. Using data from 18 large-scale microelectrode array recordings in macaque retina ex vivo, we find that for an event SNR of 7-10, wired-OR correctly detects and assigns at least 80% of the spikes with at least 50× compression. The wired-OR approach also robustly encodes action potential waveform information, enabling downstream processing such as cell-type classification. Finally, we show that by applying an LZ77-based lossless compressor (gzip) to the output of the wired-OR architecture, 1000× compression can be achieved over the baseline recordings.

Discovering pathway biomarkers of hepatocellular carcinoma occurrence and development by dynamic network entropy analysis

OBJECTIVE

Gene expression profiling techniques measure the transcription of thousands of genes in a parallel manner. With more and more hepatocellular carcinoma (HCC) transcriptomic data becoming available, the high-throughput data provides an unprecedented opportunity to discover HCC diagnostic biomarkers. In this work, we propose a bioinformatics method based on dynamic network entropy analysis, called DNEA, to identify potential pathway biomarkers for HCC occurrence and development by integrating transcriptome and interactome.

METHODS

We firstly collect the pathways documented in different knowledge-bases and then impose the genome-wide human transcriptomic data of multistage cancerous tissues during the development and progression of HCC. After linking the gene sets of pathways into individual connected networks, we map the corresponding gene expression information onto these pathways. The dynamic network entropy of individual pathways is calculated to evaluate its activities and dysfunctionalities during the disease occurrence and development. We use the overall significant difference in the entropic dynamics during the time course to prioritize distinctive pathways during disease progression. Then machine learning classification methods are employed to screen out pathway biomarkers with the classification ability to distinguish different-stage samples of HCC progression.

RESULTS

Pathway biomarkers discovered based on DNEA demonstrate good classification performance in measuring HCC progression. The classification accuracy is as follows: DNA replication pathway (mean AUC= 0.82, 20 genes) from KEGG, FMLP pathway (mean AUC=0.84, 14 genes) from BioCarta, and downstream signaling of activated FGFR pathway (mean AUC =0.80, 15 genes) from Reactome. At the same time, previous studies have shown that these genes and pathways screened are closely related to the occurrence and development of HCC in terms of oncogenesis dysfunctions.

CONCLUSIONS

Our method for cancer biomarker discovery based on dynamic network entropy analysis is effective and efficient in identifying pathway biomarkers related to the progression of complex diseases.

Complexity-based Encoded Information Quantification in Neurophysiological Recordings

Brain activity differs vastly between sleep, cognitive tasks, and action. Information theory is an appropriate concept to analytically quantify these brain states. Based on neurophysiological recordings, this concept can handle complex data sets, is free of any requirements about the data structure, and can infer the present underlying brain mechanisms. Specifically, by utilizing algorithmic information theory, it is possible to estimate the absolute information contained in brain responses. While current approaches that apply this theory to neurophysiological recordings can discriminate between different brain states, they are limited in directly quantifying the degree of similarity or encoded information between brain responses. Here, we propose a method grounded in algorithmic information theory that affords direct statements about responses' similarity by estimating the encoded information through a compression-based scheme. We validated this method by applying it to both synthetic and real neurophysiological data and compared its efficiency to the mutual information measure. This proposed procedure is especially suited for task paradigms contrasting different event types because it can precisely quantify the similarity of neuronal responses.

Understanding Design Features of Music and Language: The Choric/Dialogic Distinction

Music and spoken language share certain characteristics: both consist of sequences of acoustic elements that are combinatorically combined, and these elements partition the same continuous acoustic dimensions (frequency, formant space and duration). However, the resulting categories differ sharply: scale tones and note durations of small integer ratios appear in music, while speech uses phonemes, lexical tone, and non-isochronous durations. Why did music and language diverge into the two systems we have today, differing in these specific features? We propose a framework based on information theory and a reverse-engineering perspective, suggesting that design features of music and language are a response to their differential deployment along three different continuous dimensions. These include the familiar propositional-aesthetic ('goal') and repetitive-novel ('novelty') dimensions, and a dialogic-choric ('interactivity') dimension that is our focus here. Specifically, we hypothesize that music exhibits specializations enhancing coherent production by several individuals concurrently-the 'choric' context. In contrast, language is specialized for exchange in tightly coordinated turn-taking-'dialogic' contexts. We examine the evidence for our framework, both from humans and non-human animals, and conclude that many proposed design features of music and language follow naturally from their use in distinct dialogic and choric communicative contexts. Furthermore, the hybrid nature of intermediate systems like poetry, chant, or solo lament follows from their deployment in the less typical interactive context.

Entropy, Economics, and Criticality

Information theory is a well-established method for the study of many phenomena and more than 70 years after Claude Shannon first described it in A Mathematical Theory of Communication it has been extended well beyond Shannon’s initial vision. It is now an interdisciplinary tool that is used from ‘causal’ information flow to inferring complex computational processes and it is common to see it play an important role in fields as diverse as neuroscience, artificial intelligence, quantum mechanics, and astrophysics. In this article, I provide a selective review of a specific aspect of information theory that has received less attention than many of the others: as a tool for understanding, modelling, and detecting non-linear phenomena in finance and economics. Although some progress has been made in this area, it is still an under-developed area that I argue has considerable scope for further development.