Enhancement of Deep Neural Networks and Their Application to Text Mining

Many current application domains of machine learning and arti cial intelligence
involve knowledge discovery from text, such as sentiment analysis, document
ontology, and spam detection. Humans have years of experience and training with
language, enabling them to understand complicated, nuanced text passages with relative
ease. A text classi er attempts to emulate or replicate this knowledge so that
computers can discriminate between concepts encountered in text; however, learning
high-level concepts from text, such as those found in many applications of text classi-
cation, is a challenging task due to the many challenges associated with text mining
and classi cation. Recently, classi ers trained using arti cial neural networks have
been shown to be e ective for a variety of text mining tasks. Convolutional neural
networks have been trained to classify text from character-level input, automatically
learn high-level abstract representations and avoiding the need for human engineered
features.
This dissertation proposes two new techniques for character-level learning,
log(m) character embedding and convolutional window classi cation. Log(m) embedding
is a new character-vector representation for text data that is more compact and memory e cient than previous embedding vectors. Convolutional window classi
cation is a technique for classifying long documents, i.e. documents with lengths
exceeding the input dimension of the neural network. Additionally, we investigate the
performance of convolutional neural networks combined with long short-term memory
networks, explore how document length impacts classi cation performance and
compare performance of neural networks against non-neural network-based learners
in text classi cation tasks.