On an efficient and scalable architecture for mimicry attacks detection using probabilistic methods

Abstract

An intrusion detection system (IDS) aims at signalling an alarm for every activity that compromises a secure state of an IT system. It often amounts to detecting a known pattern of computer misuse, a deviation to ordinary, expected user behaviour, or a combination thereof. Regardless of which of these approaches is adopted, current Intrusion Detection Systems (IDSs) are easy to bypass. This thesis addresses about the three most important limitations of existing IDSs: i) current IDSs are easily overwhelmed by the the amount of information they ought to analyse; ii) current IDSs are not sufficient to monitor dynamic environments where the monitored services are changed according to the needs of the organisation; and iii) current IDSs are easy to bypass using a mimicry attack (attacks that simulate normal sequences of system calls). These kinds of attacks simulate normal activity (eg traffic, interaction) by varying an attack signature in a way that does not affect the harmfulness of the attack. Instead of creating a lightweight detection method capable of dealing with large volumes of information, at the probable cost of loss of accuracy, we focus on making intrusion detection more tractable, scalable and efficient (without compromising accuracy). We make intrusion detection more tractable by pre- processing the information. Whether it is a sequence of network packets or a sequence of system calls, the information an IDS analyses is often redundant in at least, two respects: first, every entry in the sequence may contain spurious information; second, any sequence may contain redundant subsequences. To make probabilistic intrusion detection more scalable, efficient and flexible, we propose a novel architecture that includes a service selection mechanism. Instead of analysing a single stream of data, the stream is partitioned in services, each of which is analysed by a very specialised sensor. New sensors can be added on demand; if a new service needs to be monitored another sensor is placed. To make mimicry attack intrusion detection more accurate (reduce false positives) we propose to divide attacks into smaller segments. For each segmentwe will create a detector that classifies the segment and all its variants. By combining these smaller detectors we hope to detect all variations of an attack. By using rough sets we have identified key attributes to eliminate spurious information, without missing chief details. Using n-gram theory we have identified the most redundant subsequences within a sequence, substitution of these subsequences with a fresh tag results in a reduction of the sequence length. To approach service selection, we suggest the use of hidden Markov models (HMMs), trained to detect a specific service described by a family of n-gram.s In this thesis, we introduce a method which is capable of successfully detecting a significant, interesting sub-class of mimicry attacks. The key behind our method's effectiveness lies on the use of a word network [Pereira and Riley, 1997, Young et al., 2002]. A word network conveniently decomposes a pattern matching problem into a chain of smaller, noise- tolerant pattern matchers, thereby making it more tractable and robust. A word network is realised as a finite state machine, where every state is an HMM. In our experiments, our mechanism shows an accuracy of 93%. .By contrast., the rate of false positive occurrence is only 3%. Our log reduction methods are among the best in reduction ratio and features a minimal loss of information. Ours is one of the first techniques to successfully detect a sub-class of mimicry attacks.