Characterizing Long-Term Behavior Patterns and Anomalies in Surveillance Videos

To identify any malicious activity or deviation from established protocols in safeguarded facilities, IAEA inspectors review large amounts of previously recorded surveillance video, a manually intensive and tedious effort that is prone to fatigue-induced errors. To help alleviate this demanding task, we develop time series analysis and clustering techniques to capture long-term behavior patterns and anomalies in videos. The application of time series analysis to capture long-term dependencies in video analysis is relatively unexplored. Previous research has developed methods for efficient and effective local spatial-temporal anomaly detection in surveillance videos; however, the capability to detect long-term dependent anomalies (e.g., a nuclear container taken out of the drying area at an irregular time of day or week) and recurring patterns (e.g., nuclear containers being brought to and taken out of the drying area at the same time each day) remains a significant and unresolved challenge. By representing sequences of 2D video frames as local 1D time series, we employ time series algorithms and clustering techniques to label sequences of events, facilitating the detection of long-term patterns and deviations.