Introduction
Deep neural networks such as GoogLeNet and ResNet have achieved superhuman performance in tasks like image classification. To understand how such superior performance is achieved, we can probe a trained deep neural network by studying neuron activations, that is, combinations of neuron firings, at any layer of the network in response to a particular input. With a large set of input images, we aim to obtain a global view of what neurons detect by studying their activations. We ask the following questions: What is the shape of the activation space? That is, what is the organizational principle behind neuron activations, and how are the activations related within a layer and across layers? Using tools from topological data analysis, we present TopoAct, a visual exploration system used to study topological summaries of activation vectors for a single layer as well as the evolution of such summaries across multiple layers. We present visual exploration scenarios using TopoAct that provide valuable insights towards learned representations of an image classifier.
Interactive Demos
Explore the structure of activations from a single selected layer. Inspect nodes to
see images from dataset and the corresponding feature visualization.
Compare structures of activations across multiple consecutive layers. Use filters to search of a
specific class or a set of classes across multiple graphs.
Explore the structure of activations from a single selected layer on the CIFAR-10 and CIFAR-100 datasets using ResNet architecture.