#gated recurrent unit neural network (GRU-Net)

[bookpiofficial]

Study on Neural Network-Guided Sparse Recovery for Interrupted-Sampling Repeater Jamming Suppression | Chapter  14 | Novel Perspectives of Engineering Research Vol. 8

ISRJ (interrupted-sampling repeater jamming) is a novel sort of DRFM-based jamming for linear frequency modulation (LFM) transmissions. ISRJ can obtain radar coherent processing gain by intercepting the radar signal slice and retransmitting it numerous times, resulting in the formation of multiple false target groups following pulse compression (PC), posing significant risks to radar imaging and target detection. However, in the time-frequency (TF) domain, the features of an ISRJ fragment interception can be exploited to discriminate from the true target signal. This study investigates the feasibility of exploiting the discontinuous distribution characteristics of ISRJ in the TF domain relative to the real target to create an adaptive interference suppression approach using a neural network. This work proposes a new method for ISRJ suppression based on the distribution characteristic of the echo signal and the coherence of ISRJ to radar signal. The position of the real target is calculated using a gated recurrent unit neural network (GRU-Net) in this method, and the real target may therefore be reconstructed by adaptive filtering in the sparse representation of the echo signal based on the target locating result. The reconstruction result comprises only the true target, and ISRJ's bogus target groups are totally suppressed. The projected GRU-Net has a target locating accuracy of up to. Simulations have demonstrated the efficacy of the proposed strategy. Author(S) Details Zijian Wang Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China and School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing 101408, China. Wenbo Yu Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China and School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing 101408, China. Zhongjun Yu Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China and School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing 101408, China. Yunhua Luo Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China. Jiamu Li Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China and School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing 101408, China. View Book:- https://stm.bookpi.org/NPER-V8/article/view/6120

[clawsandaffect]

Angry stomach is angry - or how we can translate tweet-speak for the medical world

“Ow my stomach is killing me”

“That burrito is really not sitting well with me”

“Uhhhg angry stomach is angry”

“My tum is :( :(”

Not only does everyday language incorporate a lot of figurative sayings, but sometimes what we write online seems like it could be another language altogether. 

Now consider the problem of mapping this internet-speak to medical concepts. Chances are your doctor didn’t dissect a stomach emoji in medical school. There needs to be some system to translate the kind of language we find online to formal medical terminology. This task is called concept normalization.

Limsopatham and Collier (2016) [1] explain what has been suboptimal about previous approaches to associating informal language with medical language and propose a method of their own. In brief, they argue that there needs to be some understanding of text at the semantic level, that is lower-level meaning, before it can be understood as health-related information. The successful approach uses a convolutional neural network (CNN), which outperforms their other model of a recurrent neural network (RNN).

Both CNNs and RNNs, when used in natural language processing (NLP), typically take word embeddings as their input. The most intuitive way we can understand word embeddings is that "a word is characterized by the company it keeps." [2] Each word in an embedding is represented by a vector, typically reduced to 300 dimensions, that is the result of some statistical analysis that quantifies the relationship among all words. In this paper, the preexisting, widely used GNews (based on 100B words pulled from Google News) and BMC (based on 854MN words from medical articles) word embeddings are used as model input.

Click here for an explanation of word embeddings

A CNN applies convolution over a sliding window across words in a sentence, which here is a tweet or a phrase from a blog post. Convolution refers to a filtering function being applied to a subset of the word embeddings, resulting in a new value that gets stored to summarize that subset. The result is a feature matrix where each window of words has a value corresponding to each feature. The maximum value at each feature goes on to represent the fully connected layer, the output of the neural net. The CNN used in the paper uses a single convolutional and pooling layer, meaning that this procedure is done only once.

Click here for a primer on CNN

Instead of using a sliding window, an RNN sequentially goes through words in the sentence and at each state produces an intermediate output called the hidden state. Each subsequent word’s embedding is processed with the previous hidden state as the input, which is what makes the network recurrent. A gated recurrent unit (GRU) is used as one type of gating function that chooses what information is relevant to maintain or forget throughout the sequence of words.

Click here for a primer on RNN and click here for details on gating functions such as GRU

When a sentence is given to the trained neural net, the output is passed to the softmax activation function, which gives a probability for the sentence being assigned to each of the selected medical terms. The probability of a phrase belonging to a given term is calculated as the exponential of the network output for that term divided by the sum of all exponentiated outputs.

Click here for a quick video explaining softmax

There are six baseline models that the two neural networks are compared against and three evaluation datasets, two based on tweets and one based on blog posts that have health-related phrases. One of the Twitter-derived sets is a novel dataset created by the authors centered around adverse drug reactions (ADRs). The models are evaluated on the basis of accuracy, where each tweet or blog post phrase is labeled with some ground truth medical topic label that the neural network then has to predict. The blog dataset resulted in higher accuracy than the other two in every case, and the researchers reason that this is because written posts tend to have more linguistic structure than brief quips like tweets. In terms of models, CNN and RNN both greatly outperform the chosen baselines, especially when using GNews embeddings. The CNN achieves a stunning 44% improvement in accuracy over the highest baseline, from 0.3099 to 0.4478.

The authors discuss where their neural network models have advantages over other previously used methods for this task. One historically used class of methods is based on string matching and finding similarities in words. The pitfall here is that the models cannot derive semantic meaning and would be mislead to believe that “i don’t hunger or thirst” is indicating hunger rather than loss of appetite as a health phenomenon. A phrase such as “appetite on 10” is complex and doesn’t make sense to a model that is unaware of semantics, or the meaning being conveyed by a word. By contrast, these neural networks can make use of co-occurence of words to understand something about their underlying meanings and understand “appetite on 10” as signaling “increased appetite.”

Hungry for more? I will be starting a new blog called Code Blue that interfaces topics in healthcare with data science. More posts in this vein (ha ha) are to come soon!

Related Work

1 - Adapting Phrase-based Machine Translation to Normalise Medical Terms in Social Media Messages (Limsopatham & Collier 2015) [3]

The authors of the reviewed paper also developed a model using not neural networks but phrase-based machine translation to address this same problem of mapping informal language to medical terminology. This model is also built on the foundation of word embeddings. Like in the reviewed paper, they strive to derive a semantic sense to words and go beyond past work simply considering lexical features.

2 -  Pharmacovigilance from social media: mining adverse drug reaction mentions using sequence labeling with word embedding cluster features (Nikfarjam et al 2015) [4]

This paper also models semantic similarities in words to try and derive medical meaning from the type of language used on social media. Specifically, they focus on the task of mining adverse drug reactions (ADR) based on what people have shared online. Their model, ADRMine, is based on conditional random fields (CRFs).

3 - Utilizing social media data for pharmacovigilance: A review (Sarker et al 2015) [5]

This is a survey of studies that detect ADRs from social media. They found that while there were 22 studies done on the topic, only six of them had their annotations publicly available, which is what would allow the methods to be compared on the basis of performance. They use these insights to propose a systematic way to collect ADR information from social media.

4 - Automagically Encoding Adverse Drug Reactions in MedDRA (Zorzi et al 2015) [6]

This paper continues on the theme of identifying and classifying ADRs. It uses the MedDRA database, the standard terminology set for reporting adverse events related to medications. The authors describe an algorithm to automatically derive MedDRA codes from freeform text, making it so that experts don’t have to manually annotate descriptions but only need to validate them.

5 - Twitter as a Lifeline: Human-annotated Twitter Corpora for NLP of Crisis-related Messages (Imran et al 2016) [7]

Although this paper does not deal with medical data, it still uses the Twitter universe as its subject of study for learning information from informal, noisy, and short messages. It trains an impressive word2vec model based on 52 million tweets from 19 different disaster situations that happened between 2013 and 2015. What is interesting here is that the language found in the tweets is hand-, or human-, annotated. This is something the reviewed paper did not do in coming up with a model for online to medical terminology. However, in both this paper and the one reviewed, the “ground truth” labels used to evaluate accuracy were based on hand annotation.

References

[1] Limsopatham, N., & Collier, N. (2016). Normalising Medical Concepts in Social Media Texts by Learning Semantic Representation. Apollo - University of Cambridge Repository. https://doi.org/10.17863/CAM.378

[2] "A synopsis of linguistic theory 1930-1955". Studies in Linguistic Analysis: 1–32. Reprinted in F.R. Palmer, ed. (1968). Selected Papers of J.R. Firth 1952-1959. London: Longman.

[3] Limsopatham, N., & Collier, N. (2015). Adapting Phrase-based Machine Translation to Normalise Medical Terms in Social Media Messages. In arXiv [cs.CL]. arXiv. http://arxiv.org/abs/1508.02285

[4] Nikfarjam, A., Sarker, A., O’Connor, K., Ginn, R., & Gonzalez, G. (2015). Pharmacovigilance from social media: mining adverse drug reaction mentions using sequence labeling with word embedding cluster features. Journal of the American Medical Informatics Association: JAMIA, 22(3), 671–681.

[5] Sarker, A., Ginn, R., Nikfarjam, A., O’Connor, K., Smith, K., Jayaraman, S., Upadhaya, T., & Gonzalez, G. (2015). Utilizing social media data for pharmacovigilance: A review. Journal of Biomedical Informatics, 54, 202–212.

[6] Zorzi, M., Combi, C., Lora, R., Pagliarini, M., & Moretti, U. (2015). Automagically Encoding Adverse Drug Reactions in MedDRA. 2015 International Conference on Healthcare Informatics, 90–99.

[7] Imran, M., Mitra, P., & Castillo, C. (2016). Twitter as a Lifeline: Human-annotated Twitter Corpora for NLP of Crisis-related Messages. In arXiv [cs.CL]. arXiv. http://arxiv.org/abs/1605.05894