A Deep Learning Method For Spatio-temporal Detection Of Atypical Activity In Ngss Camera Data

This paper presents a deep learning-based method for detecting and visualizing atypical activity in video sequences to assist in the review of data by nuclear safeguards inspectors. Surveillance review is time consuming and requires painstaking attention to detail, and our method can increase review productivity by flagging unexpected objects and activities. We are investigating unsupervised learning methods for reviewing datasets that do not require prior annotation of training data to highlight atypical activities within any given nuclear facility. The algorithms learn normal activity from video sequences and then highlight areas in each video frame that deviate from what the model was expecting. Given a set of training videos with representative activity, the deep learning model learns to predict the next frame in the video. Atypical activity can then be identified based on deviations from the predicted and the observed video sequence. The entire workflow is run via Python notebooks and is packaged in a Docker container. Use cases are tested using Next Generation Surveillance System (NGSS) cameras at Sandia National Laboratories’ gamma irradiation facility where atypical activites were staged. Results demonstrated that atypical activities were successfully identified.