Analyzing user-generated online content for drug discovery: development and use of MedCrawler

TCMNER and PubMed: A Novel Chinese Character-Level-Based Model and a Dataset for TCM Named Entity Recognition

Intelligent traditional Chinese medicine (TCM) has become a popular research field by means of prospering of deep learning technology. Important achievements have been made in such representative tasks as automatic diagnosis of TCM syndromes and diseases and generation of TCM herbal prescriptions. However, one unavoidable issue that still hinders its progress is the lack of labeled samples, i.e., the TCM medical records. As an efficient tool, the named entity recognition (NER) models trained on various TCM resources can effectively alleviate this problem and continuously increase the labeled TCM samples. In this work, on the basis of in-depth analysis, we argue that the performance of the TCM named entity recognition model can be better by using the character-level representation and tagging and propose a novel word-character integrated self-attention module. With the help of TCM doctors and experts, we define 5 classes of TCM named entities and construct a comprehensive NER dataset containing the standard content of the publications and the clinical medical records. The experimental results on this dataset demonstrate the effectiveness of the proposed module.

IHWC: intelligent hidden web crawler for harvesting data in urban domains

Due to the massive size of the hidden web, searching, retrieving and mining rich and high-quality data can be a daunting task. Moreover, with the presence of forms, data cannot be accessed easily. Forms are dynamic, heterogeneous and spread over trillions of web pages. Significant efforts have addressed the problem of tapping into the hidden web to integrate and mine rich data. Effective techniques, as well as application in special cases, are required to be explored to achieve an effective harvest rate. One such special area is atmospheric science, where hidden web crawling is least implemented, and crawler is required to crawl through the huge web to narrow down the search to specific data. In this study, an intelligent hidden web crawler for harvesting data in urban domains (IHWC) is implemented to address the relative problems such as classification of domains, prevention of exhaustive searching, and prioritizing the URLs. The crawler also performs well in curating pollution-related data. The crawler targets the relevant web pages and discards the irrelevant by implementing rejection rules. To achieve more accurate results for a focused crawl, ICHW crawls the websites on priority for a given topic. The crawler has fulfilled the dual objective of developing an effective hidden web crawler that can focus on diverse domains and to check its integration in searching pollution data in smart cities. One of the objectives of smart cities is to reduce pollution. Resultant crawled data can be used for finding the reason for pollution. The crawler can help the user to search the level of pollution in a specific area. The harvest rate of the crawler is compared with pioneer existing work. With an increase in the size of a dataset, the presented crawler can add significant value to emission accuracy. Our results are demonstrating the accuracy and harvest rate of the proposed framework, and it efficiently collect hidden web interfaces from large-scale sites and achieve higher rates than other crawlers.

Data Mining and Systematic Pharmacology to Reveal the Mechanisms of Traditional Chinese Medicine in Recurrent Respiratory Tract Infections' Treatment

Traditional Chinese medicine (TCM) was widely used in the treatment of recurrent respiratory tract infections (RRTIs) in East Asia, but its mechanism was not clear because of its complex prescription rules. This research prospectively collected 100 prescriptions of RRTI children treated with TCM. The characteristics of TCM in prescriptions were described and analyzed, and the rules of prescriptions were analyzed by hierarchical clustering and association rules. The results showed that the principle of RRTI was to pay equal attention to cold and mild, and six new meaningful prescriptions were obtained. Among them, the new prescription composed of Astragali Radix (Huangqi), Atractylodis Macrocephalae Rhizoma (Baizhu), Saposhnikoviae Radix (Fangfeng), Angelicae Sinensis Radix (Danggui), and Paeoniae Radix Rubra (Chishao) was an important method to treat RRTI. In order to explore the mechanism of the new prescription, the research obtained the action target of each herb of the core prescription on Integrative Pharmacology-based Research Platform of Traditional Chinese Medicine, TCMIP v2.0. The target genes were enriched by Metascape, and 93 Kyoto Encyclopedia of Genes and Genomes (KEGG) pathways were obtained. According to the classification and statistics of KEGG type, it was found that the new prescription mainly intervened in the metabolic pathway dominated by amino acid metabolism. In addition, there were also many interventions in the nervous system-, endocrine system-, and digestive system-related pathways. This study summarized the prescription rule of TCM in the treatment of RRTI, analyzed the mechanism of supplementing deficiency, and provided a new idea for the treatment of RRTI.

Webcrawling and machine learning as a new approach for the spatial distribution of atmospheric emissions

In this study we apply two methods for data collection that are relatively new in the field of atmospheric science. The two developed methods are designed to collect essential geo-localized information to be used as input data for a high resolution emission inventory for residential wood combustion (RWC). The first method is a webcrawler that extracts openly online available real estate data in a systematic way, and thereafter structures them for analysis. The webcrawler reads online Norwegian real estate advertisements and it collects the geo-position of the dwellings. Dwellings are classified according to the type (e.g., apartment, detached house) they belong to and the heating systems they are equipped with. The second method is a model trained for image recognition and classification based on machine learning techniques. The images from the real estate advertisements are collected and processed to identify wood burning installations, which are automatically classified according to the three classes used in official statistics, i.e., open fireplaces, stoves produced before 1998 and stoves produced after 1998. The model recognizes and classifies the wood appliances with a precision of 81%, 85% and 91% for open fireplaces, old stoves and new stoves, respectively. Emission factors are heavily dependent on technology and this information is therefore essential for determining accurate emissions. The collected data are compared with existing information from the statistical register at county and national level in Norway. The comparison shows good agreement for the proportion of residential heating systems between the webcrawled data and the official statistics. The high resolution and level of detail of the extracted data show the value of open data to improve emission inventories. With the increased amount and availability of data, the techniques presented here add significant value to emission accuracy and potential applications should also be considered across all emission sectors.