Cardenas, Erika

In today’s world, data is generated at an unprecedented rate, and a significant portion of it is unstructured text data. The recent advancements in Natural Language Processing have enabled computers to understand and interpret human language. Data mining techniques were once unable to use text data due to the high dimensionality of text processing models. This limitation was overcome with the ability to represent data as text. This thesis aims to compare the predictive performance of structured versus unstructured text data in two different applications. The first application is in the field of real estate. We compare the performance of tabular real-estate data and unstructured text descriptions of homes to predict the house price. The second application is in translating Electronic Health Records (EHR) tabular data to text data for survival classification of COVID-19 patients. Lastly, we present a range of strategies and perspectives for future research.

New software technologies are rapidly changing the economy. These changes have presented problems such as job displacement, high barrier to entry, and a gender gap in the engineering communities. In order to see the views of Americans regarding the challenges of software technologies, we conducted an online survey, gathering 500 responses. In recent news stories, it has been shown that there is a gender gap in the tech industry, but the women that participated in our survey are interested in learning software engineering as much as men. Additionally, our research found that younger people are not only required to use software tools more frequently but are the most interested in learning how to build them.  Finally, we found that a majority of people do not have any experience developing software. Our survey highlights some of the challenges of software technologies in the economy.