首页|Exploiting Network Science for Feature Extraction and Representation Learning
Exploiting Network Science for Feature Extraction and Representation Learning
扫码查看
点击上方二维码区域，可以放大扫码查看
原文链接
NETL
Networks are ubiquitous for many real-world problems such as information diffusion over social networks, transportation systems, among others. Recently, due to the ongoing Big Data revolution, the fields of machine learning and Artificial Intelligence (AI) have also become extremely important, with AI mostly being dominated by representation learning techniques such as deep learning. However, research at the intersection of network science, machine learning and AI has been mostly unexplored. Specifically, most prior research focuses on how machine learning can be used to solve "network" problems such as predicting information diffusion on social networks or classifying blogger interests in a blog network, etc. On the contrary, in this thesis, we answer the following key question: How can we exploit network science to improve machine learning and representation learning models when addressing general problems? To answer the above question, we address four fundamental research challenges: (i) Network Science for Traditional Machine Learning, (ii) Representation Learning for Small-Sample Datasets, (iii) Network Science-Based Deep Learning Model Compression, and (iv) Network Science for Neural Architecture Space Exploration. In other words, we show that many problems are governed by latent network dynamics which must be incorporated into the machine learning or representation learning models.To this end, we first demonstrate how network science can be used for traditional machine learning problems such as spatiotemporal timeseries prediction and application-specific feature extraction. We then discuss how network science can be used to address general representation learning problems with high-dimensional and small-sample datasets. Here, we propose a new network community-based dimensionality reduction framework called FeatureNet. We demonstrate the effectiveness of FeatureNet on many diverse small-sample problems. We then focus on problems like image classification for which deep learning models such as Convolutional Neural Networks (CNN) achieve state-of-the-art accuracy. Indeed, in the era of Internet-of-Things (IoT), when a computationally expensive CNN (or even a compressed model) cannot fit within the memory-budget of a single IoT-device, it must be distributed across multiple devices which leads to significant inter-device communication. To alleviate this problem, we propose a new network science-based model compression framework called the Network-of-Neural Networks (NoNN). Extensive experiments, followed by validation on real hardware such as Raspberry Pi's and Odroids, demonstrate that NoNN results in up to 12× reduction in latency, and up to 14× reduction in energy per device with negligible loss of accuracy.Finally, we exploit network science for Neural Architecture Space Exploration. To this end, we propose new, theoretically-grounded metrics to study the architecture design space of deep networks. We further show that our metrics reveal previously unknown relationship between model architectures and generalization of deep networks. We then present extensive empirical evidence towards our theoretical insights by conducting experiments on real image classification tasks. In summary, in this thesis, we address several problems at the intersection of network science, machine learning, and representation learning. Our research comprehensively demonstrates that network science can not only play a significant role, but also lead to excellent results in both machine learning and representation learning.
deep learningmachine learningmodel compressionnetwork science
NETL
