Enhancement of handwritten text recognition using AI-based hybrid approach

Handwritten text recognition (HTR) within computer vision and image processing stands as a prominent and challenging research domain, holding significant implications for diverse applications. Among these, it finds usefulness in reading bank checks, prescriptions, and deciphering characters on various forms. Optical character recognition (OCR) technology, specifically tailored for handwritten documents, plays a pivotal role in translating characters from a range of file formats, encompassing both word and image documents. Challenges in HTR encompass intricate layout designs, varied handwriting styles, limited datasets, and less accuracy achieved. Recent advancements in Deep Learning and Machine Learning algorithms, coupled with the vast repositories of unprocessed data, have propelled researchers to achieve remarkable progress in HTR. This paper aims to address the challenges in handwritten text recognition by proposing a hybrid approach. The primary objective is to enhance the accuracy of recognizing handwritten text from images. Through the integration of Convolutional Neural Networks (CNN) and Bidirectional Long Short-Term Memory (BiLSTM) with a Connectionist Temporal Classification (CTC) decoder, the results indicate substantial improvement. The proposed hybrid model achieved an impressive 98.50% and 98.80% accuracy on the IAM and RIMES datasets, respectively. This underscores the potential and efficacy of the consecutive use of these advanced neural network architectures in enhancing handwritten text recognition accuracy. •The proposed method introduces a hybrid approach for handwritten text recognition, employing CNN and BiLSTM with CTC decoder.•Results showcase a remarkable accuracy improvement of 98.50% and 98.80% on IAM and RIMES datasets, emphasizing the potential of this model for enhanced accuracy in recognizing handwritten text from images.

Developing a multivariate time series forecasting framework based on stacked autoencoders and multi-phase feature

Time series forecasting across different domains has received massive attention as it eases intelligent decision-making activities. Recurrent neural networks and various deep learning algorithms have been applied to modeling and forecasting multivariate time series data. Due to intricate non-linear patterns and significant variations in the randomness of characteristics across various categories of real-world time series data, achieving effectiveness and robustness simultaneously poses a considerable challenge for specific deep-learning models. We have proposed a novel prediction framework with a multi-phase feature selection technique, a long short-term memory-based autoencoder, and a temporal convolution-based autoencoder to fill this gap. The multi-phase feature selection is applied to retrieve the optimal feature selection and optimal lag window length for different features. Moreover, the customized stacked autoencoder strategy is employed in the model. The first autoencoder is used to resolve the random weight initialization problem. Additionally, the second autoencoder models the temporal relation between non-linear correlated features with convolution networks and recurrent neural networks. Finally, the model's ability to generalize, predict accurately, and perform effectively is validated through experimentation with three distinct real-world time series datasets. In this study, we conducted experiments on three real-world datasets: Energy Appliances, Beijing PM2.5 Concentration, and Solar Radiation. The Energy Appliances dataset consists of 29 attributes with a training size of 15,464 instances and a testing size of 4239 instances. For the Beijing PM2.5 Concentration dataset, there are 18 attributes, with 34,952 instances in the training set and 8760 instances in the testing set. The Solar Radiation dataset comprises 11 attributes, with 22,857 instances in the training set and 9797 instances in the testing set. The experimental setup involved evaluating the performance of forecasting models using two distinct error measures: root mean square error and mean absolute error. To ensure robust evaluation, the errors were calculated at the identical scale of the data. The results of the experiments demonstrate the superiority of the proposed model compared to existing models, as evidenced by significant advantages in various metrics such as mean squared error and mean absolute error. For PM2.5 air quality data, the proposed model's mean absolute error is 7.51 over 12.45, about ∼40% improvement. Similarly, the mean square error for the dataset is improved from 23.75 to 11.62, which is ∼51%of improvement. For the solar radiation dataset, the proposed model resulted in ∼34.7% improvement in means squared error and ∼75% in mean absolute error. The recommended framework demonstrates outstanding capabilities in generalization and outperforms datasets spanning multiple indigenous domains.

Machine learning insights into PM2.5 changes during COVID-19 lockdown: LSTM and RF analysis in Mashhad

This study seeks to investigate the impact of COVID-19 lockdown measures on air quality in the city of Mashhad employing two strategies. We initiated our research using basic statistical methods such as paired sample t-tests to compare hourly PM2.5 data in two scenarios: before and during quarantine, and pre- and post-lockdown. This initial analysis provided a broad understanding of potential changes in air quality. Notably, a low reduction of 2.40% in PM2.5 was recorded when compared to air quality prior to the lockdown period. This finding highlights the wide range of factors that impact the levels of particulate matter in urban settings, with the transportation sector often being widely recognized as one of the principal causes of this issue. Nevertheless, throughout the period after the quarantine, a remarkable decrease in air quality was observed characterized by distinct seasonal patterns, in contrast to previous years. This finding demonstrates a significant correlation between changes in human mobility patterns and their influence on the air quality of urban areas. It also emphasizes the need to use air pollution modeling as a fundamental tool to evaluate and understand these linkages to support long-term plans for reducing air pollution. To obtain a more quantitative understanding, we then employed cutting-edge machine learning methods, such as random forest and long short-term memory algorithms, to accurately determine the effect of the lockdown on PM2.5 levels. Our models' results demonstrated remarkable efficacy in assessing the pollutant concentration in Mashhad during lockdown measures. The test set yielded an R-squared value of 0.82 for the long short-term memory network model, whereas the random forest model showed a calculated cross-validation R-squared of 0.78. The required computational cost for training the LSTM and the RF models across all data was 25 min and 3 s, respectively. In summary, through the integration of statistical methods and machine learning, this research attempts to provide a comprehensive understanding of the impact of human interventions on air quality dynamics.

A short- and medium-term forecasting model for roof PV systems with data pre-processing

This study worked with Chunghwa Telecom to collect data from 17 rooftop solar photovoltaic plants installed on top of office buildings, warehouses, and computer rooms in northern, central and southern Taiwan from January 2021 to June 2023. A data pre-processing method combining linear regression and K Nearest Neighbor (k-NN) was proposed to estimate missing values for weather and power generation data. Outliers were processed using historical data and parameters highly correlated with power generation volumes were used to train an artificial intelligence (AI) model. To verify the reliability of this data pre-processing method, this study developed multilayer perceptron (MLP) and long short-term memory (LSTM) models to make short-term and medium-term power generation forecasts for the 17 solar photovoltaic plants. Study results showed that the proposed data pre-processing method reduced normalized root mean square error (nRMSE) for short- and medium-term forecasts in the MLP model by 17.47% and 11.06%, respectively, and also reduced the nRMSE for short- and medium-term forecasts in the LSTM model by 20.20% and 8.03%, respectively.

Integrating an LSTM framework for predicting ankle joint biomechanics during gait using inertial sensors

The ankle joint plays a crucial role in gait, facilitating the articulation of the lower limb, maintaining foot-ground contact, balancing the body, and transmitting the center of gravity. This study aimed to implement long short-term memory (LSTM) networks for predicting ankle joint angles, torques, and contact forces using inertial measurement unit (IMU) sensors. Twenty-five healthy participants were recruited. Two IMU sensors were attached to the foot dorsum and the vertical axis of the distal anteromedial tibia in the right lower limb to record acceleration and angular velocity during running. We proposed a LSTM-MLP (multilayer perceptron) model for training time-series data from IMU sensors and predicting ankle joint biomechanics. The model underwent validation and testing using a custom nested k-fold cross-validation process. The average values of the coefficient of determination (R2), mean absolute error (MAE), and mean squared error (MSE) for ankle dorsiflexion joint and moment, subtalar inversion joint and moment, and ankle joint contact forces were 0.89 ± 0.04, 0.75 ± 1.04, and 2.96 ± 4.96 for walking, and 0.87 ± 0.07, 0.88 ± 1.26, and 4.1 ± 7.17 for running, respectively. This study demonstrates that IMU sensors, combined with LSTM neural networks, are invaluable tools for evaluating ankle joint biomechanics in lower limb pathological diagnosis and rehabilitation, offering a cost-effective and versatile alternative to traditional experimental settings.

A self-attention model for cross-subject seizure detection

Epilepsy is a neurological disorder characterized by recurring seizures, detected by electroencephalography (EEG). EEG signals can be detected by manual time-consuming analysis and recently by automatic detection. The latter poses a significant challenge due to the high dimensional and non-stationary nature of EEG signals. Recently, deep learning (DL) techniques have emerged as valuable tools for seizure detection. In this study, a novel data-driven model based on DL, incorporating a self-attention mechanism (SAT), is proposed. One notable advantage of the proposed method is its simplicity in application, as the raw signal data is directly fed into the suggested network without requiring expertise in signal processing. The model leverages a one-dimensional convolutional neural network (CNN) to extract relevant features from EEG signals. These features are then passed through a long short-term memory (LSTM) module to benefit from its memory capabilities, along with a SAT mechanism. The key contribution of this paper lies in the addition of the SAT layer to the LSTM encoder, enabling enhanced exploration of the latent mapping during the encoding step. Cross-subject experiments revealed good performance of this approach with F1-score of 97.8% and 92.7% for binary and five-class epileptic seizure recognition tasks, respectively, on the public UCI dataset, and 97.9% on the CHB-MIT database, surpassing state-of-the-art DL performance. Besides, the proposed method exhibits robustness to inter-subject variability.

Improved LSTM-based deep learning model for COVID-19 prediction using optimized approach

Individuals in any country are badly impacted both economically and physically whenever an epidemic of infectious illnesses breaks out. A novel coronavirus strain was responsible for the outbreak of the coronavirus sickness in 2019. Corona Virus Disease 2019 (COVID-19) is the name that the World Health Organization (WHO) officially gave to the pneumonia that was caused by the novel coronavirus on February 11, 2020. The use of models that are informed by machine learning is currently a major focus of study in the field of improved forecasting. By displaying annual trends, forecasting models can be of use in performing impact assessments of potential outcomes. In this paper, proposed forecast models consisting of time series models such as long short-term memory (LSTM), bidirectional long short-term memory (Bi-LSTM), generalized regression unit (GRU), and dense-LSTM have been evaluated for time series prediction of confirmed cases, deaths, and recoveries in 12 major countries that have been affected by COVID-19. Tensorflow1.0 was used for programming. Indices known as mean absolute error (MAE), root means square error (RMSE), Median Absolute Error (MEDAE) and r2 score are utilized in the process of evaluating the performance of models. We presented various ways to time-series forecasting by making use of LSTM models (LSTM, BiLSTM), and we compared these proposed methods to other machine learning models to evaluate the performance of the models. Our study suggests that LSTM based models are among the most advanced models to forecast time series data.

A comparative study of data-driven models for runoff, sediment, and nitrate forecasting

Effective prediction of qualitative and quantitative indicators for runoff is quite essential in water resources planning and management. However, although several data-driven and model-driven forecasting approaches have been employed in the literature for streamflow forecasting, to our knowledge, the literature lacks a comprehensive comparison of well-known data-driven and model-driven forecasting techniques for runoff evaluation in terms of quality and quantity. This study filled this knowledge gap by comparing the accuracy of runoff, sediment, and nitrate forecasting using four robust data-driven techniques: artificial neural network (ANN), long short-term memory (LSTM), wavelet artificial neural network (WANN), and wavelet long short-term memory (WLSTM) models. These comparisons were performed in two main tiers: (1) Comparing the machine learning algorithms' results with the model-driven approach; In order to simulate the runoff, sediment, and nitrate loads, the Soil and Water Assessment Tool (SWAT) model was employed, and (2) Comparing the machine learning algorithms with each other; The wavelet function was utilized in the ANN and LSTM algorithms. These comparisons were assessed based on the substantial statistical indices of coefficient of determination (R-Squared), Nash-Sutcliff efficiency coefficient (NSE), mean absolute error (MAE), and root mean square error (RMSE). Finally, to prove the applicability and efficiency of the proposed novel framework, it was successfully applied to Eagle Creek Watershed (ECW), Indiana, U.S. Results demonstrated that the data-driven algorithms significantly outperformed the model-driven models for both the calibration/training and validation/testing phases. Furthermore, it was found that the coupled ANN and LSTM models with wavelet function led to more accurate results than those without this function.

Long short-term memory with activation on gradient

As the number of long short-term memory (LSTM) layers increases, vanishing/exploding gradient problems exacerbate and have a negative impact on the performance of the LSTM. In addition, the ill-conditioned problem occurs in the training process of LSTM and adversely affects its convergence. In this work, a simple and effective method of the gradient activation is applied to the LSTM, while empirical criteria for choosing gradient activation hyperparameters are found. Activating the gradient refers to modifying the gradient with a specific function named the gradient activation function. Moreover, different activation functions and different gradient operations are compared to prove that the gradient activation is effective on LSTM. Furthermore, comparative experiments are conducted, and their results show that the gradient activation alleviates the above problems and accelerates the convergence of the LSTM. The source code is publicly available at https://github.com/LongJin-lab/ACT-In-NLP.

Feature level fine grained sentiment analysis using boosted long short-term memory with improvised local search whale optimization

Background

In the modern era, Internet-based e-commerce world, consumers express their thoughts on the product or service through ranking and reviews. Sentiment analysis uncovers contextual inferences in user sentiment, assisting the commercial industry and end users in understanding the perception of the product or service. Variations in textual arrangement, complex logic, and sequence length are some of the challenges to accurately forecast the sentiment score of user reviews. Therefore, a novel improvised local search whale optimization improved long short-term memory (LSTM) for feature-level sentiment analysis of online product reviews is proposed in this study.

Methods

The proposed feature-level sentiment analysis method includes 'data collection', 'pre-processing', 'feature extraction', 'feature selection', and finally 'sentiment classification'. First, the product reviews given from different customers are acquired, and then the retrieved data is pre-processed. These pre-processed data go through a feature extraction procedure using a modified inverse class frequency algorithm (LFMI) based on log term frequency. Then the feature is selected via levy flight-based mayfly optimization algorithm (LFMO). At last, the selected data is transformed to the improvised local search whale optimization boosted long short-term memory (ILW-LSTM) model, which categorizes the sentiment of the customer reviews as 'positive', 'negative', 'very positive', 'very negative', and 'neutral'. The 'Prompt Cloud dataset' is used for the performance study of the suggested classifiers. Our suggested ILW-LSTM model is put to the test using standard performance evaluation. The primary metrics used to assess our suggested model are 'accuracy', 'recall', 'precision', and 'F1-score'.

Results and Conclusion

The proposed ILW-LSTM method provides an accuracy of 97%. In comparison to other leading algorithms, the outcome reveals that the ILW-LSTM model outperformed well in feature-level sentiment classification.

Facial expression recognition in videos using hybrid CNN & ConvLSTM

The three-dimensional convolutional neural network (3D-CNN) and long short-term memory (LSTM) have consistently outperformed many approaches in video-based facial expression recognition (VFER). The image is unrolled to a one-dimensional vector by the vanilla version of the fully-connected LSTM (FC-LSTM), which leads to the loss of crucial spatial information. Convolutional LSTM (ConvLSTM) overcomes this limitation by performing LSTM operations in convolutions without unrolling, thus retaining useful spatial information. Motivated by this, in this paper, we propose a neural network architecture that consists of a blend of 3D-CNN and ConvLSTM for VFER. The proposed hybrid architecture captures spatiotemporal information from the video sequences of emotions and attains competitive accuracy on three FER datasets open to the public, namely the SAVEE, CK + , and AFEW. The experimental results demonstrate excellent performance without external emotional data with the added advantage of having a simple model with fewer parameters. Moreover, unlike the state-of-the-art deep learning models, our designed FER pipeline improves execution speed by many factors while achieving competitive recognition accuracy. Hence, the proposed FER pipeline is an appropriate candidate for recognizing facial expressions on resource-limited embedded platforms for real-time applications.

Spectral noise and data reduction using a long short-term memory network for nonlinear ultrasonic modulation-based fatigue crack detection

This paper presents a spectral noise and data reduction technique based on long short-term memory (LSTM) network for nonlinear ultrasonic modulation-based fatigue crack detection. The amplitudes of the nonlinear modulation components created by a micro fatigue crack are often very small and masked by noise. In addition, the collection of large amounts of data is often undesirable owing to the limited power, data storage, and data transmission bandwidth of monitoring systems. To tackle the issues, an LSTM network was applied to ultrasonic signals to reduce the noise level and the amount of data. The proposed technique offers the following benefits: (1) spectral noise reduction using the LSTM network for ultrasonic signals and (2) data reduction without compromising the spectral density amplitude of the existing nonlinear modulation components. Finally, the performance evaluation was conducted using the data obtained from complex geometry and real structure under external noises, indicating that the proposed method can be applied to various structures.

Comparative analysis of water quality prediction performance based on LSTM in the Haihe River Basin, China

As the most water shortage and water polluted area in China, the water quality prediction is of utmost needed and important in Haihe River Basin for its water resource management. The long short-term memory (LSTM) has been a widely used tool for water quality forecast in recent years. The performance and adaptability of LSTM for water quality prediction of different indicators needs to be discussed before it adopted in a specific basin. However, literature contains very few studies on the comparative analysis of the various prediction accuracy of different water quality indicators and the causes, especially in Haihe River Basin. In this study, LSTM was employed to predict biochemical oxygen demand (BOD), permanganate index (COD_Mn), dissolved oxygen (DO), ammonia nitrogen (NH₃-N), total phosphorus (TP), hydrogen ion concentration (pH), and chemical oxygen demand digested by potassium dichromate (COD_Cr). According to results under 24 different input conditions, it is demonstrated that LSTMs present better predicting on BOD, COD_Mn, COD_Cr, and TP (median Nash-Sutcliffe efficiency reaching 0.766, 0.835, 0.837, and 0.711, respectively) than NH₃-N, DO, and pH (median Nash-Sutcliffe efficiency of 0.638, 0.625, and 0.229, respectively). Besides, the performance of LSTM to predict water quality is linearly related to the maximum value of temporal autocorrelation and cross-correlation coefficients of water quality indicators calculated by maximal information coefficient with the coefficients of determination of 0.79 to approximately 0.80. This study would provide new knowledge and support for the practical application and improvement of the LSTM in water quality prediction.

Reconstruction of missing spring discharge by using deep learning models with ensemble empirical mode decomposition of precipitation

A continuous and complete spring discharge record is critical in understanding the hydrodynamic behavior of karst aquifers and the variability of freshwater resources. However, due to equipment errors, failure of observation and other reasons, missing data is a common problem for spring discharge monitoring and further hydrological investigations and data analysis. In this study, a novel approach that integrates deep learning algorithms and ensemble empirical mode decomposition (EEMD) is proposed to reconstruct the missing spring discharge data with a given local precipitation record. Using EEMD, the local precipitation data is decomposed into several intrinsic mode functions (IMFs) from high to low frequencies and a residual function, which are served as the input of convolutional neural network (CNN), long short-term memory (LSTM), and hybrid CNN-LSTM models to reconstruct the missing discharge data. Evaluation metrics, including root mean squared error (RMSE), mean absolute error (MAE), and Nash-Sutcliffe efficiency coefficient (NSE), are calculated to evaluate the reconstruction performance. The monthly spring discharge and precipitation data from March 1978 to October 2021 collected at Barton Springs in Texas are used for the validation and evaluation of newly proposed deep learning models. The results indicate that deep learning models coupled with EEMD overperform the models without EEMD and significantly improve the reconstruction results. The LSTM-EEMD model obtains the best reconstruction results among three deep learning algorithms. For models with monthly data, the missing rate affects the reconstruction performance because of the number of data samples: the best reconstruction results are achieved when the missing rate was low. If the missing rate was 50%, the reconstruction results become notably poorer. However, when the daily precipitation and discharge data are used, the models can obtain satisfactory reconstruction results with missing rate ranged from 10 to 50%.

Evaluation of e-learners' concentration using recurrent neural networks

Recently, interest in e-learning has increased rapidly owing to the lockdowns imposed by COVID-19. A major disadvantage of e-learning is the difficulty in maintaining concentration because of the limited interaction between teachers and students. The objective of this paper is to develop a methodology to predict e-learners' concentration by applying recurrent neural network models to eye gaze and facial landmark data extracted from e-learners' video data. One hundred eighty-four video data of ninety-two e-learners were obtained, and their frame data were extracted using the OpenFace 2.0 toolkit. Recurrent neural networks, long short-term memory, and gated recurrent units were utilized to predict the concentration of e-learners. A set of comparative experiments was conducted. As a result, gated recurrent units exhibited the best performance. The main contribution of this paper is to present a methodology to predict e-learners' concentration in a natural e-learning environment.

PM2.5 Forecast in Korea using the Long Short-Term Memory (LSTM) Model

The National Institute of Environmental Research, under the Ministry of Environment of Korea, provides two-day forecasts, through AirKorea, of the concentration of particulate matter with diameters of ≤ 2.5 μm (PM2.5) in terms of four grades (low, moderate, high, and very high) over 19 districts nationwide. Particulate grades are subjectively designated by human forecasters based on forecast results from the Community Multiscale Air Quality (CMAQ) and artificial intelligence (AI) models in conjunction with weather patterns. This study evaluates forecasts from the long short-term memory (LSTM) algorithm relative to those from CMAQ-solely and AirKorea using observations from 2019. The skills of the one-day PM2.5 forecasts over the 19 districts were 39-70% for CMAQ, 72-79% for LSTM, and 73-80% for AirKorea; the AI forecasts showed comparable skills to the human forecasters at AirKorea. The one-day forecast skill levels of high and very high PM2.5 pollution grades are 31-98%, 31-74%, and 39-81% for the CMAQ-solely, the LSTM, and the AirKorea forecasts, respectively. Despite good skills for forecasting the high and very high events, CMAQ-solely forecasts also generate substantially higher false alarm rates (up to 86%) than the LSTM and AirKorea forecasts (up to 58%). Hence, applying only the LSTM model to the CMAQ forecasts can yield reasonable forecast skill levels comparable to the operational AirKorea forecasts that elaborately combine the CMAQ model, AI models, and human forecasters. The present results suggest that applications of appropriate AI models can greatly enhance PM2.5 forecast skills for Korea in a more objective way.

Supplementary Information

The online version contains supplementary material available at 10.1007/s13143-022-00293-2.

Deepaware: A hybrid deep learning and context-aware heuristics-based model for atrial fibrillation detection

BACKGROUND

State-of-the-art automatic atrial fibrillation (AF) detection models trained on RR-interval (RRI) features generally produce high performance on standard benchmark electrocardiogram (ECG) AF datasets. These models, however, result in a significantly high false positive rates (FPRs) when applied on ECG data collected under free-living ambulatory conditions and in the presence of non-AF arrhythmias.

METHOD

This paper proposes DeepAware, a novel hybrid model combining deep learning (DL) and context-aware heuristics (CAH), which reduces the FPR effectively and improves the AF detection performance on participant-operated ambulatory ECG from free-living conditions. It exploits the RRI and P-wave features, as well as the contextual features from the ambulatory ECG.

RESULTS

DeepAware is shown to be very generalizable and superior to the state-of-the-art models when applied on unseen benchmark ECG AF datasets. Most importantly, the model is able to detect AF efficiently when applied on participant-operated ambulatory ECG recordings from free-living conditions and has achieved a sensitivity (Se), specificity (Sp), and accuracy (Acc) of 97.94%, 98.39%, 98.06%, respectively. Results also demonstrate the effect of atrial activity analysis (via P-waves detection) and CAH in reducing the FPR over the RRI features-based AF detection model.

CONCLUSIONS

The proposed DeepAware model can substantially reduce the physician's workload of manually reviewing the false positives (FPs) and facilitate long-term ambulatory monitoring for early detection of AF.

Multi-step-ahead water level forecasting for operating sluice gates in Hai Duong, Vietnam

Recently, machine learning (ML) is being applied to various fields, including hydrology and hydraulics. The numerical models based on ML algorithms have been widely used for forecasting water levels or flowrate in different timescales. Especially in estuary areas where the hydrodynamic regime becomes complicated, the water level forecast information in this area plays an essential role in the operation of tidal sluices. This study proposes an efficient approach using an ML model, long short-term memory (LSTM), to predict short-term water levels in tidal sluice gates from 6 to 48 hours ahead. The An Tho culvert located in the Bac Hung Hai irrigation system, the most extensive irrigation system in Vietnam, was selected as a case study station. The high accuracy of predictive results reveals LSTM models' effectiveness in different forecasting scenarios. In the first scenario using just water level data at the prediction station, the Kling-Gupta efficiency (KGE) coefficient ranges from nearly 0.89 to 0.96. Meanwhile, in the second scenario, the combination of observed data of three gauge stations exhibited better performance with KGE coefficients ranging from just under 0.93 to 0.98 for eight forecasted cases. The findings of this study highlight the performance of LSTM models in providing high-accuracy short-period water level forecasts for areas near estuaries. These obtained results can play a vital role in the management and operation of tidal sluices in the Bac Hung Hai irrigation system, as well as a reference for the operation of other irrigation systems around the world.

The impact of public health emergencies on hotel demand - Estimation from a new foresight perspective on the COVID-19

This paper proposes a new foresight approach to estimate the impact of public health emergencies on hotel demand. The forecasting-based influence evaluation consists of four modules: decomposing hotel demand before an emergency, matching each decomposed component to a forecasting model, combining the predictions as the expected demand after the emergency, and estimating the impact by comparing actual demand against that predicted. The method is applied to analyze the impact of COVID-19 on Macao's hotel industry. The empirical results show that: 1) the new approach accurately estimates COVID-19's impact on hotel demand; 2) the seasonal and industry development components contribute significantly to the estimate of expected demand; 3) COVID-19's impact is heterogeneous across hotel services.

The impact of public health emergencies on hotel demand - Estimation from a new foresight perspective on the COVID-19

This paper proposes a new foresight approach to estimate the impact of public health emergencies on hotel demand. The forecasting-based influence evaluation consists of four modules: decomposing hotel demand before an emergency, matching each decomposed component to a forecasting model, combining the predictions as the expected demand after the emergency, and estimating the impact by comparing actual demand against that predicted. The method is applied to analyze the impact of COVID-19 on Macao's hotel industry. The empirical results show that: 1) the new approach accurately estimates COVID-19's impact on hotel demand; 2) the seasonal and industry development components contribute significantly to the estimate of expected demand; 3) COVID-19's impact is heterogeneous across hotel services.

A framework for motor imagery with LSTM neural network

BACKGROUND AND OBJECTIVE

How to learn robust representations from brain activities and to improve algorithm performance are the most significant issues for brain-computer interface systems.

METHODS

This study introduces a long short-term memory recurrent neural network to decode the multichannel electroencephalogram or electrocorticogram for implementing an effective motor imagery-based brain-computer interface system. The unique information processing mechanism of the long short-term memory network characterizes spatio-temporal dynamics in time sequences. This study evaluates the proposed method using publically available electroencephalogram/electrocorticogram datasets.

RESULTS

The decoded features coupled with a gradient boosting classifier could obtain high recognition accuracies of 99% for electroencephalogram and 100% for electrocorticogram, respectively.

CONCLUSIONS

The results demonstrated that the proposed model can estimate robust spatial-temporal features and obtain significant performance improvement for motor imagery-based brain-computer interface systems. Further, the proposed method is of low computational complexity.

Time series forecasting for uni- variant data using hybrid GA-OLSTM model and performance evaluations

Time series forecasting of uni-variant rainfall data is done using a hybrid genetic algorithm integrated with optimized long-short term memory (GA-OLSTM) model. The parameters included for the valuation of the efficiency of the considered model, were mean square error (MSE), root mean square error (RMSE), cosine similarity (CS) and correlation coefficient (r). With various epochs like 5, 10, 15 and 20, the optimal window size and the number of units were observed using the GA search algorithm which was found to be (49, 9), (12, 8), (40, 8), and (36, 2) respectively. The computed MSE, RMSE, CS and r for 10 epochs were found to be 0.006, 0.078, 0.910 and 0.858 respectively for the LSTM model, whereas the same parameters were computed using the Hybrid GA-OLSTM model was 0.004, 0.063, 0.947 and 0.917 respectively. The experimental results expressed that the Hybrid GA-OLSTM model gave significantly better results comparing the LSTM model for 10 epochs has been discussed in this research article.

Classification of Ear Imagery Database using Bayesian Optimization based on CNN-LSTM Architecture

The external and middle ear conditions are diagnosed using a digital otoscope. The clinical diagnosis of ear conditions is suffered from restricted accuracy due to the increased dependency on otolaryngologist expertise, patient complaint, blurring of the otoscopic images, and complexity of lesions definition. There is a high requirement for improved diagnosis algorithms based on otoscopic image processing. This paper presented an ear diagnosis approach based on a convolutional neural network (CNN) as feature extraction and long short-term memory (LSTM) as a classifier algorithm. However, the suggested LSTM model accuracy may be decreased by the omission of a hyperparameter tuning process. Therefore, Bayesian optimization is used for selecting the hyperparameters to improve the results of the LSTM network to obtain a good classification. This study is based on an ear imagery database that consists of four categories: normal, myringosclerosis, earwax plug, and chronic otitis media (COM). This study used 880 otoscopic images divided into 792 training images and 88 testing images to evaluate the approach performance. In this paper, the evaluation metrics of ear condition classification are based on a percentage of accuracy, sensitivity, specificity, and positive predictive value (PPV). The findings yielded a classification accuracy of 100%, a sensitivity of 100%, a specificity of 100%, and a PPV of 100% for the testing database. Finally, the proposed approach shows how to find the best hyperparameters concerning the Bayesian optimization for reliable diagnosis of ear conditions under the consideration of LSTM architecture. This approach demonstrates that CNN-LSTM has higher performance and lower training time than CNN, which has not been used in previous studies for classifying ear diseases. Consequently, the usefulness and reliability of the proposed approach will create an automatic tool for improving the classification and prediction of various ear pathologies.

EEG-based emotion charting for Parkinson's disease patients using Convolutional Recurrent Neural Networks and cross dataset learning

Electroencephalogram (EEG) based emotion classification reflects the actual and intrinsic emotional state, resulting in more reliable, natural, and meaningful human-computer interaction with applications in entertainment consumption behavior, interactive brain-computer interface, and monitoring of psychological health of patients in the domain of e-healthcare. Challenges of EEG-based emotion recognition in real-world applications are variations among experimental settings and cognitive health conditions. Parkinson's Disease (PD) is the second most common neurodegenerative disorder, resulting in impaired recognition and expression of emotions. The deficit of emotional expression poses challenges for the healthcare services provided to PD patients. This study proposes 1D-CRNN-ELM architecture, which combines one-dimensional Convolutional Recurrent Neural Network (1D-CRNN) with an Extreme Learning Machine (ELM), robust for the emotion detection of PD patients, also available for cross dataset learning with various emotions and experimental settings. In the proposed framework, after EEG preprocessing, the trained CRNN can use as a feature extractor with ELM as the classifier, and again this trained CRNN can be used for learning of new emotions set with fine-tuning of other datasets. This paper also applied cross dataset learning of emotions by training with PD patients datasets and fine-tuning with publicly available datasets of AMIGOS and SEED-IV, and vice versa. Random splitting of train and test data with 80 - 20 ratio resulted in an accuracy of 97.75% for AMIGOS, 83.20% for PD, and 86.00% for HC with six basic emotion classes. Fine-tuning of trained architecture with four emotions of the SEED-IV dataset results in 92.5% accuracy. To validate the generalization of our results, leave one subject (patient) out cross-validation is also incorporated with mean accuracies of 95.84% for AMIGOS, 75.09% for PD, 77.85% for HC, and 84.97% for SEED-IV is achieved. Only a 1 - sec segment of EEG signal from 14 channels is enough to detect emotions with this performance. The proposed method outperforms state-of-the-art studies to classify EEG-based emotions with publicly available datasets, provide cross dataset learning, and validate the robustness of the deep learning framework for real-world application of psychological healthcare monitoring of Parkinson's disease patients.

Analysis of agricultural exports based on deep learning and text mining

Agricultural exports are an important source of economic profit for many countries. Accurate predictions of a country's agricultural exports month on month are key to understanding a country's domestic use and export figures and facilitate advance planning of export, import, and domestic use figures and the resulting necessary adjustments of production and marketing. This study proposes a novel method for predicting the rise and fall of agricultural exports, called agricultural exports time series-long short-term memory (AETS-LSTM). The method applies Jieba word segmentation and Word2Vec to train word vectors and uses TF-IDF and word cloud to learn news-related keywords and finally obtain keyword vectors. This research explores whether the purchasing managers' index (PMI) of each industry can effectively use the AETS-LSTM model to predict the rise and fall of agricultural exports. Research results show that the inclusion of keyword vectors in the PMI values of the finance and insurance industries has a relative impact on the prediction of the rise and fall of agricultural exports, which can improve the prediction accuracy for the rise and fall of agricultural exports by 82.61%. The proposed method achieves improved prediction ability for the chemical/biological/medical, transportation equipment, wholesale, finance and insurance, food and textiles, basic materials, education/professional, science/technical, information/communications/broadcasting, transportation and storage, retail, and electrical and machinery equipment categories, while its performance for the electrical and optical categories shows improved prediction by combining keyword vectors, and its accuracy for the accommodation and food service, and construction and real estate industries remained unchanged. Therefore, the proposed method offers improved prediction capacity for agricultural exports month on month, allowing agribusiness operators and policy makers to evaluate and adjust domestic and foreign production and sales.

Integrating mechanistic and deep learning models for accurately predicting the enrichment of polyhydroxyalkanoates accumulating bacteria in mixed microbial cultures

The enrichment of polyhydroxyalkanoates (PHA) accumulating bacteria (PAB) in mixed microbial cultures (MMC) is extremely difficult to be predicted and optimized. Here we demonstrate that mechanistic and deep learning models can be integrated innovatively to accurately predict the dynamic enrichment of PAB. Well-calibrated activated sludge models (ASM) of the PAB enrichment process provide time-dependent data under different operating conditions. Recurrent neural network (RNN) models are trained and tested based on the time-dependent dataset generated by ASM. The accurate prediction performance is achieved (R² > 0.991) for three different PAB enrichment datasets by the optimized RNN model. The optimized RNN model can also predict the equilibrium concentration of PAB (R² = 0.944) and corresponding time, which represents the end of the PAB enrichment process. This study demonstrates the strength of integrating mechanistic and deep learning models to predict long-term variations of specific microbes, helping to optimize their selection process for PHA production.

An integrated deep learning model for motor intention recognition of multi-class EEG Signals in upper limb amputees

BACKGROUND AND OBJECTIVE

Recognition of motor intention based on electroencephalogram (EEG) signals has attracted considerable research interest in the field of pattern recognition due to its notable application of non-muscular communication and control for those with severe motor disabilities. In analysis of EEG data, achieving a higher classification performance is dependent on the appropriate representation of EEG features which is mostly characterized by one unique frequency before applying a learning model. Neglecting other frequencies of EEG signals could deteriorate the recognition performance of the model because each frequency has its unique advantages. Motivated by this idea, we propose to obtain distinguishable features with different frequencies by introducing an integrated deep learning model to accurately classify multiple classes of upper limb movement intentions.

METHODS

The proposed model is a combination of long short-term memory (LSTM) and stacked autoencoder (SAE). To validate the method, four high-level amputees were recruited to perform five motor intention tasks. The acquired EEG signals were first preprocessed before exploring the consequence of input representation on the performance of LSTM-SAE by feeding four frequency bands related to the tasks into the model. The learning model was further improved by t-distributed stochastic neighbor embedding (t-SNE) to eliminate feature redundancy, and to enhance the motor intention recognition.

RESULTS

The experimental results of the classification performance showed that the proposed model achieves an average performance of 99.01% for accuracy, 99.10% for precision, 99.09% for recall, 99.09% for f1_score, 99.77% for specificity, and 99.0% for Cohen's kappa, across multi-subject and multi-class scenarios. Further evaluation with 2-dimensional t-SNE revealed that the signal decomposition has a distinct multi-class separability in the feature space.

CONCLUSION

This study demonstrated the predominance of the proposed model in its ability to accurately classify upper limb movements from multiple classes of EEG signals, and its potential application in the development of a more intuitive and naturalistic prosthetic control.

COVID-19 cough classification using machine learning and global smartphone recordings

We present a machine learning based COVID-19 cough classifier which can discriminate COVID-19 positive coughs from both COVID-19 negative and healthy coughs recorded on a smartphone. This type of screening is non-contact, easy to apply, and can reduce the workload in testing centres as well as limit transmission by recommending early self-isolation to those who have a cough suggestive of COVID-19. The datasets used in this study include subjects from all six continents and contain both forced and natural coughs, indicating that the approach is widely applicable. The publicly available Coswara dataset contains 92 COVID-19 positive and 1079 healthy subjects, while the second smaller dataset was collected mostly in South Africa and contains 18 COVID-19 positive and 26 COVID-19 negative subjects who have undergone a SARS-CoV laboratory test. Both datasets indicate that COVID-19 positive coughs are 15%–20% shorter than non-COVID coughs. Dataset skew was addressed by applying the synthetic minority oversampling technique (SMOTE). A leave-p-out cross-validation scheme was used to train and evaluate seven machine learning classifiers: logistic regression (LR), k-nearest neighbour (KNN), support vector machine (SVM), multilayer perceptron (MLP), convolutional neural network (CNN), long short-term memory (LSTM) and a residual-based neural network architecture (Resnet50). Our results show that although all classifiers were able to identify COVID-19 coughs, the best performance was exhibited by the Resnet50 classifier, which was best able to discriminate between the COVID-19 positive and the healthy coughs with an area under the ROC curve (AUC) of 0.98. An LSTM classifier was best able to discriminate between the COVID-19 positive and COVID-19 negative coughs, with an AUC of 0.94 after selecting the best 13 features from a sequential forward selection (SFS). Since this type of cough audio classification is cost-effective and easy to deploy, it is potentially a useful and viable means of non-contact COVID-19 screening.

Forecasting the spread of COVID-19 using LSTM network

BACKGROUND

The novel coronavirus (COVID-19) is caused by severe acute respiratory syndrome coronavirus 2, and within a few months, it has become a global pandemic. This forced many affected countries to take stringent measures such as complete lockdown, shutting down businesses and trade, as well as travel restrictions, which has had a tremendous economic impact. Therefore, having knowledge and foresight about how a country might be able to contain the spread of COVID-19 will be of paramount importance to the government, policy makers, business partners and entrepreneurs. To help social and administrative decision making, a model that will be able to forecast when a country might be able to contain the spread of COVID-19 is needed.

RESULTS

The results obtained using our long short-term memory (LSTM) network-based model are promising as we validate our prediction model using New Zealand's data since they have been able to contain the spread of COVID-19 and bring the daily new cases tally to zero. Our proposed forecasting model was able to correctly predict the dates within which New Zealand was able to contain the spread of COVID-19. Similarly, the proposed model has been used to forecast the dates when other countries would be able to contain the spread of COVID-19.

CONCLUSION

The forecasted dates are only a prediction based on the existing situation. However, these forecasted dates can be used to guide actions and make informed decisions that will be practically beneficial in influencing the real future. The current forecasting trend shows that more stringent actions/restrictions need to be implemented for most of the countries as the forecasting model shows they will take over three months before they can possibly contain the spread of COVID-19.

Concrete compressive strength prediction modeling utilizing deep learning long short-term memory algorithm for a sustainable environment

One of the most critical parameters in concrete design is compressive strength. As the compressive strength of concrete is correctly measured, time and cost can be decreased. Concrete strength is relatively resilient to impacts on the environment. The production of concrete compressive strength is greatly influenced by severe weather conditions and increases in humidity rates. In this research, a model has been developed to predict concrete compressive strength utilizing a detailed dataset obtained from previously published studies based on a deep learning method, namely, long short-term memory (LSTM), and a conventional machine learning (ML) algorithm, namely, support vector machine (SVM). The input variables of the model include cement, blast furnace slag, fly ash, water, superplasticizer, coarse aggregate, fine aggregate, and age of specimens. To demonstrate the efficiency of the proposed models, three statistical indices, namely, the coefficient of determination (R²), mean absolute error (MAE), and root mean square error (RMSE), were used. Findings shows that LSTM outperformed SVM with R²=0.98, R²= 0.78, MAE=1.861, MAE=6.152, and RMSE=2.36, RMSE=7.93, respectively. The results of this study suggest that high-performance concrete (HPC) compressive strength can be reliably measured using the proposed LSTM model.

A novel LSTM-CNN-grid search-based deep neural network for sentiment analysis

As the number of users getting acquainted with the Internet is escalating rapidly, there is more user-generated content on the web. Comprehending hidden opinions, sentiments, and emotions in emails, tweets, reviews, and comments is a challenge and equally crucial for social media monitoring, brand monitoring, customer services, and market research. Sentiment analysis determines the emotional tone behind a series of words may essentially be used to understand the attitude, opinions, and emotions of users. We propose a novel long short-term memory (LSTM)-convolutional neural networks (CNN)-grid search-based deep neural network model for sentiment analysis. The study considers baseline algorithms like convolutional neural networks, K-nearest neighbor, LSTM, neural networks, LSTM-CNN, and CNN-LSTM which have been evaluated using accuracy, precision, sensitivity, specificity, and F-1 score, on multiple datasets. Our results show that the proposed model based on hyperparameter optimization outperforms other baseline models with an overall accuracy greater than 96%.

Knowledge discovery from emergency ambulance dispatch during COVID-19: A case study of Nagoya City, Japan

Accurate forecasting of medical service requirements is an important big data problem that is crucial for resource management in critical times such as natural disasters and pandemics. With the global spread of coronavirus disease 2019 (COVID-19), several concerns have been raised regarding the ability of medical systems to handle sudden changes in the daily routines of healthcare providers. One significant problem is the management of ambulance dispatch and control during a pandemic. To help address this problem, we first analyze ambulance dispatch data records from April 2014 to August 2020 for Nagoya City, Japan. Significant changes were observed in the data during the pandemic, including the state of emergency (SoE) declared across Japan. In this study, we propose a deep learning framework based on recurrent neural networks to estimate the number of emergency ambulance dispatches (EADs) during a SoE. The fusion of data includes environmental factors, the localization data of mobile phone users, and the past history of EADs, thereby providing a general framework for knowledge discovery and better resource management. The results indicate that the proposed blend of training data can be used efficiently in a real-world estimation of EAD requirements during periods of high uncertainties such as pandemics.

Prediction of COVID-19 Trend in India and Its Four Worst-Affected States Using Modified SEIRD and LSTM Models

Since the beginning of COVID-19 (corona virus disease 2019), the Indian government implemented several policies and restrictions to curtail its spread. The timely decisions taken by the government helped in decelerating the spread of COVID-19 to a large extent. Despite these decisions, the pandemic continues to spread. Future predictions about the spread can be helpful for future policy-making, i.e., to plan and control the COVID-19 spread. Further, it is observed throughout the world that asymptomatic corona cases play a major role in the spread of the disease. This motivated us to include such cases for accurate trend prediction. India was chosen for the study as the population and population density is very high for India, resulting in the spread of the disease at high speed. In this paper, the modified SEIRD (susceptible–exposed–infected–recovered–deceased) model is proposed for predicting the trend and peak of COVID-19 in India and its four worst-affected states. The modified SEIRD model is based on the SEIRD model, which also uses an asymptomatic exposed population that is asymptomatic but infectious for the predictions. Further, a deep learning-based long short-term memory (LSTM) model is also used for trend prediction in this paper. Predictions of LSTM are compared with the predictions obtained from the proposed modified SEIRD model for the next 30 days. The epidemiological data up to 6th September 2020 have been used for carrying out predictions in this paper. Different lockdowns imposed by the Indian government have also been used in modeling and analyzing the proposed modified SEIRD model.

Modelling the impact of weather parameters on the microbial quality of water in distribution systems

In this study, a framework for integrating weather variables and seasons into the modelling and prediction of the microbial quality in drinking water distribution networks is presented. Statistical analysis and Bayesian network (BN) modelling were used to evaluate relationships among water quality parameters in distribution pipes and their dependencies on weather parameters. Two robust predictive models for Total Bacteria in the network were built based on a deep learning approach (Long Short-Term Memory (LSTM)). The first model included water quality parameters alone as inputs while the second model included weather parameters. The seven-year dataset used in this study constituted water quality parameters measured at seven location in the water distribution network for the city of Ålesund in Norway, and weather data for the same period. Results of the initial statistical analysis and the BN models showed that, air temperature, the summer season, precipitation, as well as water quality parameters namely, residual chlorine, water temperature, alkalinity and electrical conductivity have strong relations with the counts of Total Bacteria in the distribution networks studied. It was found that the integration of the weather parameters in the Total Bacteria prediction models significantly improved the quality of the predictions. Compared to the LSTM 1, LSTM 2 achieved MAE and MSE values as high as to 6.8 and 4.9 times respectively when the model was tested on the seven locations. In addition, the R² values were marginally higher in LSTM 2 (0.92-0.95) than in LSTM (0.81-0.86). The prediction results demonstrate the relevance of integrating weather parameters such as air temperature seasons in predicting bacteria levels in water distribution systems. This suggests that changes in the microbial quality of water in distribution systems and potentially drinking water sources could be reliably assessed by integrating online sensors of water quality and weather parameters with efficient models such as the LSTM. Applying this efficient modelling approach in the management of water supply systems could offer immense support in addressing current challenges in assessing the microbial quality of water and minimizing associated health risks.

OPTICAL+: a frequency-based deep learning scheme for recognizing brain wave signals

A human-computer interaction (HCI) system can be used to detect different categories of the brain wave signals that can be beneficial for neurorehabilitation, seizure detection and sleep stage classification. Research on developing HCI systems using brain wave signals has progressed a lot over the years. However, real-time implementation, computational complexity and accuracy are still a concern. In this work, we address the problem of selecting the appropriate filtering frequency band while also achieving a good system performance by proposing a frequency-based approach using long short-term memory network (LSTM) for recognizing different brain wave signals. Adaptive filtering using genetic algorithm is incorporated for a hybrid system utilizing common spatial pattern and LSTM network. The proposed method (OPTICAL+) achieved an overall average classification error rate of 30.41% and a kappa coefficient value of 0.398, outperforming the state-of-the-art methods. The proposed OPTICAL+ predictor can be used to develop improved HCI systems that will aid in neurorehabilitation and may also be beneficial for sleep stage classification and seizure detection.

A Remaining Useful Life Prognosis of Turbofan Engine Using Temporal and Spatial Feature Fusion

The prognosis of the remaining useful life (RUL) of turbofan engine provides an important basis for predictive maintenance and remanufacturing, and plays a major role in reducing failure rate and maintenance costs. The main problem of traditional methods based on the single neural network of shallow machine learning is the RUL prognosis based on single feature extraction, and the prediction accuracy is generally not high, a method for predicting RUL based on the combination of one-dimensional convolutional neural networks with full convolutional layer (1-FCLCNN) and long short-term memory (LSTM) is proposed. In this method, LSTM and 1- FCLCNN are adopted to extract temporal and spatial features of FD001 andFD003 datasets generated by turbofan engine respectively. The fusion of these two kinds of features is for the input of the next convolutional neural networks (CNN) to obtain the target RUL. Compared with the currently popular RUL prediction models, the results show that the model proposed has higher prediction accuracy than other models in RUL prediction. The final evaluation index also shows the effectiveness and superiority of the model.

Going by the numbers : Learning and modeling COVID-19 disease dynamics

The COrona VIrus Disease (COVID-19) pandemic caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV2) has resulted in a challenging number of infections and deaths worldwide. In order to combat the pandemic, several countries worldwide enforced mitigation measures in the forms of lockdowns, social distancing, and disinfection measures. In an effort to understand the dynamics of this disease, we propose a Long Short-Term Memory (LSTM) based model. We train our model on more than four months of cumulative COVID-19 cases and deaths. Our model can be adjusted based on the parameters in order to provide predictions as needed. We provide results at both the country and county levels. We also perform a quantitative comparison of mitigation measures in various counties in the United States based on the rate of difference of a short and long window parameter of the proposed LSTM model. The analyses provided by our model can provide valuable insights based on the trends in the rate of infections and deaths. This can also be of help for countries and counties deciding on mitigation and reopening strategies. We believe that the results obtained from the proposed method will contribute to societal benefits for a current global concern.

LSTM based prediction of malaria abundances using big data

Malaria prevails in subtropical countries where health monitoring facilities are minimal. Time series prediction models are required to forecast malaria and minimize the effect of this disease on the population. This study proposes a novel scalable framework to predict the instances of malaria in selected geographical locations. Satellite data and clinical data, along with a long short-term memory (LSTM) classifier, were used to predict malaria abundances in the state of Telangana, India. The proposed model provided a 12 months seasonal pattern for selected regions in the state. Each region had different responses based on environmental factors. Analysis indicated that both environmental and clinical variables play an important role in malaria transmission. In conclusion, the Apache Spark-based LSTM presents an effective strategy to identify locations of endemic malaria.

Estimation of Arterial Blood Pressure Based on Artificial Intelligence Using Single Earlobe Photoplethysmography during Cardiopulmonary Resuscitation

This study investigates the feasibility of estimation of blood pressure (BP) using a single earlobe photoplethysmography (Ear PPG) during cardiopulmonary resuscitation (CPR). We have designed a system that carries out Ear PPG for estimation of BP. In particular, the BP signals are estimated according to a long short-term memory (LSTM) model using an Ear PPG. To investigate the proposed method, two statistical analyses were conducted for comparison between BP measured by the micromanometer-based gold standard method (BP_MEAS) and the Ear PPG-based proposed method (BP_EST) for swine cardiac model. First, Pearson's correlation analysis showed high positive correlations (r = 0.92, p < 0.01) between BP_MEAS and BP_EST. Second, the paired-samples t-test on the BP parameters (systolic and diastolic blood pressure) of the two methods indicated no significant differences (p > 0.05). Therefore, the proposed method has the potential for estimation of BP for CPR biofeedback based on LSTM using a single Ear PPG.

Multi-Class Classification of Sleep Apnea/Hypopnea Events Based on Long Short-Term Memory Using a Photoplethysmography Signal

In this study, we proposed a new method for multi-class classification of sleep apnea/hypopnea events based on a long short-term memory (LSTM) using photoplethysmography (PPG) signals. The three-layer LSTM model was used with batch-normalization and dropout to classify the multi-class events including normal, apnea, and hypopnea. The PPG signals, which were measured by the nocturnal polysomnography with 7 h from 82 patients suffered from sleep apnea, were used to model training and evaluation. The performance of the proposed method was evaluated on the training set from 63 patients and test set from 13 patients. The results of the LSTM model showed the following high performances: the positive predictive value of 94.16% for normal, 81.38% for apnea, and 97.92% for hypopnea; sensitivity of 86.03% for normal, 91.24% for apnea, and 99.38% for hypopnea events. The proposed method had especially higher performance of hypopnea classification which had been a drawback of previous studies. Furthermore, it can be applied to a system that can classify sleep apnea/hypopnea and normal events automatically without expert's intervention at home.

Local climate zone-based urban land cover classification from multi-seasonal Sentinel-2 images with a recurrent residual network

The local climate zone (LCZ) scheme was originally proposed to provide an interdisciplinary taxonomy for urban heat island (UHI) studies. In recent years, the scheme has also become a starting point for the development of higher-level products, as the LCZ classes can help provide a generalized understanding of urban structures and land uses. LCZ mapping can therefore theoretically aid in fostering a better understanding of spatio-temporal dynamics of cities on a global scale. However, reliable LCZ maps are not yet available globally. As a first step toward automatic LCZ mapping, this work focuses on LCZ-derived land cover classification, using multi-seasonal Sentinel-2 images. We propose a recurrent residual network (Re-ResNet) architecture that is capable of learning a joint spectral-spatial-temporal feature representation within a unitized framework. To this end, a residual convolutional neural network (ResNet) and a recurrent neural network (RNN) are combined into one end-to-end architecture. The ResNet is able to learn rich spectral-spatial feature representations from single-seasonal imagery, while the RNN can effectively analyze temporal dependencies of multi-seasonal imagery. Cross validations were carried out on a diverse dataset covering seven distinct European cities, and a quantitative analysis of the experimental results revealed that the combined use of the multi-temporal information and Re-ResNet results in an improvement of approximately 7 percent points in overall accuracy. The proposed framework has the potential to produce consistent-quality urban land cover and LCZ maps on a large scale, to support scientific progress in fields such as urban geography and urban climatology.

Deep facial analysis: A new phase I epilepsy evaluation using computer vision

Semiology observation and characterization play a major role in the presurgical evaluation of epilepsy. However, the interpretation of patient movements has subjective and intrinsic challenges. In this paper, we develop approaches to attempt to automatically extract and classify semiological patterns from facial expressions. We address limitations of existing computer-based analytical approaches of epilepsy monitoring, where facial movements have largely been ignored. This is an area that has seen limited advances in the literature. Inspired by recent advances in deep learning, we propose two deep learning models, landmark-based and region-based, to quantitatively identify changes in facial semiology in patients with mesial temporal lobe epilepsy (MTLE) from spontaneous expressions during phase I monitoring. A dataset has been collected from the Mater Advanced Epilepsy Unit (Brisbane, Australia) and is used to evaluate our proposed approach. Our experiments show that a landmark-based approach achieves promising results in analyzing facial semiology, where movements can be effectively marked and tracked when there is a frontal face on visualization. However, the region-based counterpart with spatiotemporal features achieves more accurate results when confronted with extreme head positions. A multifold cross-validation of the region-based approach exhibited an average test accuracy of 95.19% and an average AUC of 0.98 of the ROC curve. Conversely, a leave-one-subject-out cross-validation scheme for the same approach reveals a reduction in accuracy for the model as it is affected by data limitations and achieves an average test accuracy of 50.85%. Overall, the proposed deep learning models have shown promise in quantifying ictal facial movements in patients with MTLE. In turn, this may serve to enhance the automated presurgical epilepsy evaluation by allowing for standardization, mitigating bias, and assessing key features. The computer-aided diagnosis may help to support clinical decision-making and prevent erroneous localization and surgery.