Neural networks (Computer science)

With the rapid development of networking platforms and data intensive applications, networks (or graphs) are becoming convenient and fundamental tools to model the complex inter-dependence among big scale data. As a result, networks (or graphs) are being widely used in many applications, including citation networks [40], social media networks [71], and so on. However, the high complexity (containing many important information) as well as the dynamic nature of the network makes the graph learning task more difficult. To have better graph representations (capture both node content and graph structure), many research efforts have been made to develop reliable and efficient algorithms. Therefore, the good graph representation learning is the key factor in performing well on downstream tasks. The dissertation mainly focuses on the graph representation learning, which aims to embed both structure and node content information of graphs into a compact and low dimensional space for a new representation learning. More specifically, in order to achieve an efficient and robust graph representation, the following four problems will be studied from different perspectives: 1) We study the problem of positive unlabeled graph learning for network node classification, and present a new deep learning model as a solution; 2) We formulate a new open-world learning problem for graph data, and propose an uncertain node representation learning approach and sampling strategy to solve the problem; 3) For cross-domain graph learning, we present a novel unsupervised graph domain adaptation problem, and propose an effective graph convolutional network algorithm to solve it; 4) We consider a dynamic graph as a network with changing nodes and edges in temporal order and propose a temporal adaptive aggregation network (TAAN) for dynamic graph learning. Finally, the proposed models are verified and evaluated on various real-world datasets.

Financial time-series data are noisy, volatile, and nonlinear. The classic statistical linear models may not capture those underlying structures of the data. The rapid advancement in artificial intelligence and machine learning techniques, availability of large-scale data, and increased computational capabilities of a machine opens the door to developing sophisticated deep learning models to capture the nonlinearity and hidden information in the data. Creating a robust model by unlocking the power of a deep neural network and using real-time data is essential in this tech era. This study constructs a new computational framework to uncover the information in the financial time-series data and better inform the related parties. It carries out the comparative analysis of the performance of the deep learning models on stock price prediction with a well-balanced set of factors from fundamental data, macroeconomic data, and technical indicators responsible for stock price movement. We further build a novel computational framework through a merger of recurrent neural networks and random compression for the time-series analysis. The performance of the model is tested on a benchmark anomaly time-series dataset. This new computational framework in a compressed paradigm leads to improved computational efficiency and data privacy. Finally, this study develops a custom trading simulator and an agent-based hybrid model by combining gradient and gradient-free optimization methods. In particular, we explore the use of simulated annealing with stochastic gradient descent. The model trains a population of agents to predict appropriate trading behaviors such as buy, hold, or sell by optimizing the portfolio returns. Experimental results on S&P 500 index show that the proposed model outperforms the baseline models.

Within Deep CNNs there is great excitement over breakthroughs in network performance on benchmark datasets such as ImageNet. Around the world competitive teams work on new ways to innovate and modify existing networks, or create new ones that can reach higher and higher accuracy levels. We believe that this important research must be supplemented with research into the computational dynamics of the networks themselves. We present research into network behavior as it is affected by: variations in the number of filters per layer, pruning filters during and after training, collapsing the weight space of the trained network using a basic quantization, and the effect of Image Size and Input Layer Stride on training time and test accuracy. We provide insights into how the total number of updatable parameters can affect training time and accuracy, and how “time per epoch” and “number of epochs” affect network training time. We conclude with statistically significant models that allow us to predict training time as a function of total number of updatable parameters in the network.

Medical professionals use CT images to get information about the size, shape, and location of any lung nodules. This information will help radiologist and oncologist to identify the type of cancer and create a treatment plan. However, most of the time, the diagnosis regarding the types of lung cancer is error-prone and time-consuming. One way to address these problems is by using convolutional neural networks. In this Thesis, we developed a convolutional neural network that can detect abnormalities in lung CT scans and further categorize the abnormalities to benign, malignant adenocarcinoma and malignant squamous cell carcinoma. Our network is based on DenseNet, which utilizes dense connections between layers (dense blocks), so that all layers are connected. Because of all layers being connected, different layers can reuse features from previous layers which speeds up the process and make this network computationally efficient. To retrain this network we used CT images for 314 patients (over 1500 CT images) consistent of 42 Lung Adenocarcinoma and 78 Squamous Cell Carcinoma, 118 Non cancer and 76 benign were acquired from the National Lung Screening Trial (NLST). These images were divided to two categories of Training and Validation with 70% being training dataset and 30% as validation dataset. We trained our network on Training dataset and then checked the accuracy of our model using the validation dataset. Our model was able to categorize lung cancer with an accuracy of 88%. Afterwards we calculated the the confusion matrix, Precision (Sensitivity), Recall (Positivity) and F1 score of our model for each category. Our model is able to classify Normal CT images with Normal Accuracy of 89% Precision of 94% and F1 score of 93%. For benign nodules Accuracy was 92% precision of 97% and F1 score 86%, while for Adenocarcinoma and squamous cell cancer the Accuracy was 98% and 93%, Precision 85% and 84% and F1 score 92% and 86.9%. The relatively high accuracy of our model shows that convolutional neural networks can be a valuable tool for the classification of lung cancer, especially in a small city or underdeveloped rural hospital settings and can play a role in achieving healthcare equality.

One basic goal of artificial learning systems is the ability to continually learn throughout that system’s lifetime. Transitioning between tasks and re-deploying prior knowledge is thus a desired feature of artificial learning. However, in the deep-learning approaches, the problem of catastrophic forgetting of prior knowledge persists. As a field, we want to solve the catastrophic forgetting problem without requiring exponential computations or time, while demonstrating real-world relevance. This work proposes a novel model which uses an evolutionary algorithm similar to a meta-learning objective, that is fitted with a resource constraint metrics. Four reinforcement learning environments are considered with the shared concept of depth although the collection of environments is multi-modal. This system shows preservation of some knowledge in sequential task learning and protection of catastrophic forgetting in deep neural networks.

The recent rise of artificial intelligence (AI) using deep learning networks allowed the development of automatic solutions for many tasks that, in the past, were seen as impossible to be performed by a machine. However, deep learning models are getting larger, need significant processing power to train, and powerful machines to use it. As deep learning applications become ubiquitous, another trend is taking place: the growing use of edge devices. This dissertation addresses selected technical issues associated with edge AI, proposes novel solutions to them, and demonstrates the effectiveness of the proposed approaches. The technical contributions of this dissertation include: (i) architectural optimizations to deep neural networks, particularly the use of patterned stride in convolutional neural networks used for image classification; (ii) use of weight quantization to reduce model size without hurting its accuracy; (iii) systematic evaluation of the impact of image imperfections on skin lesion classifiers' performance in the context of teledermatology; and (iv) a new approach for code prediction using natural language processing techniques, targeted at edge devices.

Positive natural selection leaves detectable, distinctive patterns in the genome in the form of a selective sweep. Identifying areas of the genome that have undergone selective sweeps is an area of high interest as it enables understanding of species and population evolution. Previous work has accomplished this by evaluating patterns within summary statistics computed across the genome and through application of machine learning techniques to raw population genomic data. When using raw population genomic data, convolutional neural networks have most recently been employed as they can handle large input arrays and maintain correlations among elements. Yet, such models often require massive amounts of training data and can be computationally expensive to train for a given problem. Instead, transfer learning has recently been used in the image analysis literature to improve machine learning models by learning the important features of images from large unrelated datasets beforehand, and then refining these models through subsequent application on smaller and more relevant datasets. We combine transfer learning with convolutional neural networks to improve classification of selective sweeps from raw population genomic data. We show that the combination of transfer learning with convolutional neural networks allows for accurate classification of selective sweeps.

Liver cancer is the sixth most common type of cancer worldwide and is the third leading cause of cancer related mortality. Several types of cancer can form in the liver. Hepatocellular carcinoma (HCC) makes up 75%-85% of all primary liver cancers and it is a malignant disease with limited therapeutic options due to its aggressive progression. While the exact cause of liver cancer may not be known, habits/lifestyle may increase the risk of developing the disease. Several risk prediction models for HCC are available for individuals with hepatitis B and C virus infections who are at high risk but not for general population. To address this challenge, an artificial neural network (ANN) was developed, trained, and tested using the health data to predict liver cancer risk. Our results indicate that our ANN can be used to predict liver cancer risk with changes with lifestyle and may provide a novel approach to identify patients at higher risk and can be bene ted from early diagnosis.

Tropical cyclones are among the most devastating natural disasters for human beings and the natural and manmade assets near to Atlantic basin. Estimating the current and future intensity of these powerful storms is crucial to protect life and property. Many methods and models exist for predicting the evolution of Atlantic basin cyclones, including numerical weather prediction models that simulate the dynamics of the atmosphere which require accurate measurements of the current state of the atmosphere (NHC, 2019). Often these models fail to capture dangerous aspects of storm evolution, such as rapid intensification (RI), in which a storm undergoes a steep increase in intensity over a short time. To improve prediction of these events, scientists have turned to statistical models to predict current and future intensity using readily collected satellite image data (Pradhan, 2018). However, even the current-intensity prediction models have shown limited success in generalizing to unseen data, a result we confirm in this study. Therefore, building models for the estimating the current and future intensity of hurricanes is valuable and challenging.
In this study we focus on to estimating cyclone intensity using Geostationary Operational Environmental Satellite images. These images represent five spectral bands covering the visible and infrared spectrum. We have built and compared various types of deep neural models, including convolutional networks based on long short term memory models and convolutional regression models that have been trained to predict the intensity, as measured by maximum sustained wind speed.

Network embedding or representation learning is important for analyzing many real-world applications and systems, i.e., social networks, citation networks and communication networks. It targets at learning low-dimensional vector representations of nodes with preserved graph structure (e.g., link relations) and content (e.g., texts) information. The derived node representations can be directly applied in many downstream applications, including node classification, clustering and visualization.
In addition to the complex network structures, nodes may have rich non structure information such as labels and contents. Therefore, structure, label and content constitute different aspects of the entire network system that reflect node similarities from multiple complementary facets. This thesis focuses on multifaceted network embedding learning, which aims to efficiently incorporate distinct aspects of information such as node labels and node contents for cooperative low-dimensional representation learning together with node topology.