Estimating Intrinsic Dimension To Architect Compact Neural Networks

This project is supported by Grant H98230-21-1-0317.

Information security professionals around the world are always looking for ways to improve the robustness of threat detection and response. A machine learning paradigm such as an autoencoder can be used to automate flagging and increase the accuracy of threat detection. Autoencoders are unsupervised neural networks that are able to take input data, encode and compress the data, and decompress the data into a reconstruction extremely similar to the original data (Badr, 2019). Compression schemes such as autoencoders need to encode input to a size that maximizes space savings while minimizing information loss – a size governed by the intrinsic dimension of the data. This research focuses on a suite of intrinsic dimension estimation techniques implemented in the Scikit-dimension Python package and subsequent use of the Pytorch package to develop an autoencoder model. This is an extremely important interplay because without the correct intrinsic dimension, adversaries can take advantage of the excess space in the autoencoder and insert malicious packets to the system. On the other hand, if the intrinsic dimension of an autoencoder is set too low, it will struggle to learn the representation of even normal, non-malicious data. It is for these reasons that it is imperative to discover a reliable method of intrinsic dimension estimation.