Confidence Networks for Providing Output Assurance on Spectrogram Classifiers

Machine learning research has increasingly focused on enhancing the ability of neural networks to predict confidence levels of their outputs is particularly important in safety-critical applications. In safety-critical instances, understanding the uncertainty of predictions can significantly impact decision-making processes, such as classifying and predicting attributes of waveforms in spectrogram format. Confidence networks (CNs) are secondary supporting models trained to predict the confidence of a primary model’s output based on activation properties of the primary model’s forward pass. After having trained ConvNext models to predict waveform attributes of spectrogram images, it was found that the distribution of values in ConvNext confidence maps from samples that were correctly classified differed significantly from the distribution of values of incorrectly classified samples. This work details our efforts towards training these secondary CN models, and their ability to accurately predict non-confident outputs, including out-of-distribution outputs. We will outline an experiment with a specific waveform classifier, and provide comparisons to a traditional confidence metric called Attribute Based Confidence (ABC) to determine the efficacy of CN’s