On the performance of a hybrid intrusion detection architecture for voice over IP systems

Abstract

Voice over IP (VoIP) environments pose challenging threats to Intrusion Detection Systems (IDSs). Services over VoIP systems are provided by multiple interacting protocols, each with its own vulnerabilities. This scheme could result in novel and more complex attacks, and requires cross-protocol aware IDSs. Furthermore, VoIP devices may suffer a full or partial service loss if the syntax or semantics of the aforementioned protocols are violated. Usually, a single detection approach is suited to identify a subset of the security violations to which a system is subject in VoIP environments. Therefore, a hybrid approach that combines the strengths and avoids the weaknesses of various approaches is needed. In this paper, we discuss the performance and the detection accuracy of a hybrid, host-based intrusion detection system suitable for VoIP environments. Our system has two combined detection modules, namely, a specification-based and a signature-based module. Both modules use State Machines and State Transition Analysis Techniques to model proper protocols' behaviors and potential attacks. Both modules address the issues related to syntax and semantics anomaly detection for the monitored protocols. In addition, our architecture provides a cross-protocol framework for various protocols to exchange useful detection information in real time. We implement our proposed architecture in a network simulator, alongside implementing a variety of attacks to test the credibility of the design. The implemented IDS shows an excellent detection accuracy, and low runtime impact on the performance of the VoIP system.