AAAI 2021

Multidimensional Uncertainty-Aware Evidential Neural Networks

Yibo Hu, Yuzhe Ou, Xujiang Zhao, Jin-Hee Cho, Feng Chen

A classifier can be unsure because its evidence points to several familiar classes at once, or because it has almost no evidence at all. The model in this paper reports these two kinds of uncertainty as separate numbers, and the second one is what flags inputs unlike anything it was trained on.

Take three classes of points that overlap in the middle, shown as white, red and orange dots below. A point in the overlap is hard to call. A point far from all three is unlike any training example. The usual uncertainty score, the entropy of the predicted class probabilities, treats the two cases alike: in the left map it is high in the overlap (red circle) and high everywhere outside the data (yellow).

One spot, three readings of uncertainty
Three maps of the same synthetic data. Entropy is high both where the three classes overlap and everywhere away from the data. Dissonance is high only in narrow streaks where the classes meet. Vacuity is low over the whole region covered by data, including the overlap, and high outside it.
Color shows the model’s uncertainty, from 0 (dark purple) to 1 (yellow); each circle marks the same spot where the classes overlap. Red circle: entropy is high there. Green circles: the other two scores tell overlap from unfamiliar. Adapted from Figure 3 of the paper; highlights added.

The other two maps split that single signal in two. Dissonance (middle) is high only where the classes meet: look for the teal streaks that run out of the circle to the upper right and straight down, along the borders between classes. Inside the circle the dots hide the same color. There the model has evidence for more than one class. Vacuity (right) is low across the whole region covered by training data, including the overlap, and high everywhere outside it: the model has little evidence for any class. Both terms come from subjective logic, a framework for reasoning with evidence. The network outputs an amount of evidence for each class rather than a probability, and the two measures read different patterns in that evidence.

To make vacuity high in the right places, we train the classifier together with a generator, a second network that learns to produce examples ranging from far away to close to the training data. The classifier learns to report high vacuity on them. We call the combined model WENN.

The same split holds on real images. For a model trained on five CIFAR-10 classes, images from those classes that it got wrong had low vacuity and higher dissonance than the images it got right, so a mistake on a familiar kind of image does not look like an unfamiliar image. In the paper’s hardest detection test, a model trained on CIFAR-10 had to tell CIFAR-100 images apart from CIFAR-10 images. Measured by AUROC, where 50 means chance and 100 means every unfamiliar image is ranked above every familiar one, vacuity scored 99.5, the same model’s entropy 98.6, and the best of the other methods compared 92.5. These experiments use synthetic data and small image benchmarks.