A cross-protocol approach to detect TCP hijacking attacks

Abstract

More efficient intrusion detection systems (IDSs) have become a necessity because the nature of Internet attacks and the methods used by attackers are changing significantly. Many recent attacks take advantage of more than one protocol at a time, which results in poor detection accuracy in traditional IDSs. In this paper, we propose a novel design and implementation of TCP extended finite state machine with TCP hijacking in mind. Our design is based on a cross-protocol detection mechanism which assists TCP detection module with information from other protocols involved (especially IP), and makes TCP parameters available for other protocols participating in the session. The way our system is designed enables it to help a wide range of applications that use TCP protocol, to detect session attacks. The system is tested with TCP hijacking attacks among others and shows promising detection accuracy.