Wider Vision: Enriching Convolutional Neural Networks via Alignment to External Knowledge Bases

Currently, Deep Learning neural network models of Computer Vision are considered black boxes. There have been many research works about revealing what is inside the black boxes. An explanation of CNNs that could fully un-boxes a CNN model and explains the CNN features detected is still missing. This explanation will help other real-world problems, such as Visual Question Answering and Zero-shot Learning.

The intuition is to build a graph from CNN using visual features and to use Entity Alignment methods to map these features to an external Knowledge Graph. The corresponding nodes for a feature provides an explanation for that feature.