Marques, Oge

Skin cancer is a prevalent cancer that significantly contributes to global mortality rates. Early detection is crucial for a high survival rate. Dermatologists primarily rely on visual inspection to diagnose skin cancers, but this method is inaccurate. Deep learning algorithms can enhance the diagnostic accuracy of skin cancers. However, these algorithms require substantial labeled data for effective training. Acquiring annotated data for skin cancer classification is time-consuming, expensive, and necessitates expert annotation. Moreover, skin cancer datasets often suffer from imbalanced data distribution.
Generative Adversarial Networks (GANs) can be used to overcome the challenges of data scarcity and lack of labels by automatically generating skin cancer images. However, training and testing data from different distributions can introduce domain shift and bias, impacting the model’s performance. This dissertation addresses this issue by developing deep learning-based domain adaptation models.
Additionally, this research emphasizes deploying deep learning models on hardware to enable real-time skin cancer detection, facilitating accurate diagnoses by dermatologists. Deploying conventional deep learning algorithms on hardware is not preferred due to the problem of high resource consumption. Therefore, this dissertation presents spiking neural network-based (SNN) models designed specifically for hardware implementation. SNNs are preferred for their power-efficient behavior and suitability for hardware deployment.

Advancements in Artificial Intelligence (AI) and Machine Learning (ML) have significantly improved their application in dermatology. However, bias issues in AI systems can result in missed diagnoses and disparities in healthcare, especially for individuals with different skin types. This dissertation aims to investigate and improve the fairness and bias in machine learning models for dermatology by evaluating and enhancing their performance across different Fitzpatrick skin types.
The technical contributions of the dissertation include generating metadata for Fitzpatrick Skin Type using Individual Typology Angle; outlining best practices for Explainable AI (XAI) and the use of colormaps; developing and enhancing ML models through skin color transformation and extending the models to include XAI methods for better interpretation and improvement of fairness and bias; and providing a list of steps for successful application of deep learning in medical image analysis.
The research findings of this dissertation have the potential to contribute to the development of fair and unbiased AI/ML models in dermatology. This can ultimately lead to better health outcomes and reduced healthcare costs, particularly for individuals with different skin types.

Digital transformation is rapidly changing the healthcare industry, and artificial intelligence (AI) is a critical component in this evolution. This thesis investigates three selected challenges that might delay the adoption of AI in healthcare and proposes ways to address them successfully. Challenge #1 states that healthcare professionals may not feel sufficiently knowledgeable about AI. This is addressed by Contribution #1 which is a guide for self-actualization in AI for healthcare professionals. Challenge #2 explores the concept of transdisciplinary teams needing a work protocol to deliver successful results. This is addressed by Contribution #2 which is a step-by-step protocol for medical and AI researchers working on data-intensive projects. Challenge #3 states that the NIH All of Us Research Hub has a steep learning curve, and this is addressed by Contribution #3 which is a pilot project involving transdisciplinary teams using All of Us datasets.

The recent rise of artificial intelligence (AI) using deep learning networks allowed the development of automatic solutions for many tasks that, in the past, were seen as impossible to be performed by a machine. However, deep learning models are getting larger, need significant processing power to train, and powerful machines to use it. As deep learning applications become ubiquitous, another trend is taking place: the growing use of edge devices. This dissertation addresses selected technical issues associated with edge AI, proposes novel solutions to them, and demonstrates the effectiveness of the proposed approaches. The technical contributions of this dissertation include: (i) architectural optimizations to deep neural networks, particularly the use of patterned stride in convolutional neural networks used for image classification; (ii) use of weight quantization to reduce model size without hurting its accuracy; (iii) systematic evaluation of the impact of image imperfections on skin lesion classifiers' performance in the context of teledermatology; and (iv) a new approach for code prediction using natural language processing techniques, targeted at edge devices.

Artificial intelligence (AI) had a few false starts – the AI winters of the 1970s and 1980s. We are now in what looks like an AI summer. There are many useful applications of AI in the field. But there are still unfulfilled promises and outright failures. From self-driving cars that work only in constrained cases, to medical image analysis products that would replace radiologists but never did, we still struggle to translate successful research into successful real-world applications.
The software engineering community has accumulated a large body of knowledge over the decades on how to develop, release, and maintain products. AI products, being software products, benefit from some of that accumulated knowledge, but not all of it. AI products diverge from traditional software products in fundamental ways: their main component is not a specific piece of code, written for a specific purpose, but a generic piece of code, a model, customized by a training process driven by hyperparameters and a dataset. Datasets are usually large and models are opaque. We cannot directly inspect them as we can inspect the code of traditional software products. We need other methods to detect failures in AI products.

Melanoma, a severe and life-threatening skin cancer, is commonly misdiagnosed
or left undiagnosed. Advances in artificial intelligence, particularly deep learning,
have enabled the design and implementation of intelligent solutions to skin lesion
detection and classification from visible light images, which are capable of performing
early and accurate diagnosis of melanoma and other types of skin diseases. This work
presents solutions to the problems of skin lesion segmentation and classification. The
proposed classification approach leverages convolutional neural networks and transfer
learning. Additionally, the impact of segmentation (i.e., isolating the lesion from the
rest of the image) on the performance of the classifier is investigated, leading to the
conclusion that there is an optimal region between “dermatologist segmented” and
“not segmented” that produces best results, suggesting that the context around a
lesion is helpful as the model is trained and built. Generative adversarial networks,
in the context of extending limited datasets by creating synthetic samples of skin
lesions, are also explored. The robustness and security of skin lesion classifiers using
convolutional neural networks are examined and stress-tested by implementing
adversarial examples.

Lower prices of video sensors, security concerns and the need for better and faster
algorithms to extract high level information from video sequences are all factors which
have stimulated research in the area of automated video surveillance systems. In the
context of security the analysis of human interrelations and their environment provides
hints to proactively identify anomalous behavior. However, human detection is a
necessary component in systems where the automatic extraction of higher level
information, such as recognizing individuals' activities, is required. The human detection
problem is one of classification. In general, motion, appearance and shape are the
classification approaches a system can employ to perform human detection. Techniques
representative of these approaches, such us periodic motion detection, skin color
detection and MPEG-7 shape descriptors are implemented in this work. An infrastructure
that allows data collection for such techniques was also implemented.

Autonomous video surveillance systems are usually built with several functional blocks
such as motion detection, foreground and background separation, object tracking, depth
estimation, feature extraction and behavioral analysis of tracked objects. Each of those
blocks is usually designed with different techniques and algorithms, which may need
significant computational and hardware resources. In this thesis we present a surveillance
system based on an optical flow concept, as a main unit on which other functional blocks
depend. Optical flow limitations, capabilities and possible problem solutions are
discussed in this thesis. Moreover, performance evaluation of various methods in
handling occlusions, rigid and non-rigid object classification, segmentation and tracking
is provided for a variety of video sequences under different ambient conditions. Finally,
processing time is measured with software that shows an optical flow hardware block can
improve system performance and increase scalability while reducing the processing time
by more than fifty percent.

In the past few years, violence detection has become an increasingly rele-
vant topic in computer vision with many proposed solutions by researchers. This
thesis proposes a solution called Criminal Aggression Recognition Engine (CARE),
an OpenCV based Java implementation of a violence detection system that can be
trained with video datasets to classify action in a live feed as non-violent or violent.
The algorithm extends existing work on fast ght detection by implementing violence
detection of live video, in addition to prerecorded video. The results for violence
detection in prerecorded videos are comparable to other popular detection systems
and the results for live video are also very encouraging, making the work proposed in
this thesis a solid foundation for improved live violence detection systems.

Image Processing and Computer Vision solutions have become commodities
for software developers, thanks to the growing availability of Application Program-
ming Interfaces (APIs) that encapsulate rich functionality, powered by advanced al-
gorithms. To understand and create an e cient method to process faces in images
by computers, one must understand how the human visual system processes them.
Face processing by computers has been an active research area for about 50
years now. Face detection has become a commodity and is now incorporated into
simple devices such as digital cameras and smartphones.
An iOS app was implemented in Objective-C using Microsoft Cognitive Ser-
vices APIs, as a tool for human vision and face processing research. Experimental
work on image compression, upside-down orientation, the Thatcher e ect, negative
inversion, high frequency, facial artifacts, caricatures and image degradation were
completed on the Radboud and 10k US Adult Faces Databases along with other
images.