Malicious behaviors identification in nuclear security based
on visual relationships extraction and knowledge
reasoning

Intrusion and sabotage to nuclear facilities pose serious consequences to society safety and economic loss, or which the protection against human malicious behaviors is also necessary and critical. However, commonly employed physical protection systems usually rely on manual monitoring and extensive sensor deployment, which proves to be easily missed and costly. To this end, works have been conducted to introduce deep learning-based object detection and action recognition models to automatically perform identification of human malicious behaviors. However, such approaches allow the identification of only malicious behaviors with relatively obvious features, such as carrying of malicious weapons or directly aggressive actions. For human behaviors with ambiguous features or non-unique intents, i.e., the reasoning result would vary according to the situation, a unified model with comprehensive reasoning capabilities turns out to be necessary and applicable. In this paper, a novel human malicious behaviors identification approach based on visual relationships extraction and knowledge reasoning is proposed. First of all, it should be noted that the main entities in surveillance videos, i.e., objects, humans, and actions, are detected or predicted using deep learning-based calculation models. Subsequently, the visual relationships in videos are extracted by temporalspatial relationships analysis, and are then converted to knowledge units. Finally, human malicious behaviors identification is performed based on knowledge reasoning. In order to complete this task, a nuclear security-specific knowledge base is pre-generated according to the statistical information of training dataset, which contains features of typical knowledge unit element sequences annotated as malicious behaviors in nuclear security. Therefore, in the testing phase, the reasoning process could be carried out by checking the existence of items in the knowledge base. With the implementation of the proposed approach, a preliminarily identification of human malicious behaviors in four scenarios, i.e., fence climbing, wire net cutting, weapons holding, and normal status, has been conducted.