Flow reconstruction and normalization in Suricata

The naive approach would consider that an IDS is just taking packet and doing a lot of matching on it. In fact, this is not at all what is happening. An IDS/IPS like Suricata is in fact rebuilding the data stream and in case of known protocols it is even normalizing the data stream and providing keyword which can be used to match on specific field of a protocol.

Let’s say, we a rule to match on a HTTP request where method is GET and the URL is “/download.php”.

But what happen inside Suricata when want to do such a match ? Let’s try to visualize this matching on a fictive example packets stream. With such a stream, we could have the following process:

Flow reconstruction by Suricata

If you click on the image, you will get access to an interactive svg showing the alerting signatures.

A series of 7 IP packets are seen. They belong to the same flow but because of fragmentation they need to be assembled by the defrag engine. The #4 packet is invalid due to an invalid checksum.
Suricata can alert on this packet if the following rule is activated:

alert ip any any -> any any (msg:"SURICATA IPv4 invalid checksum"; ipv4-csum:invalid; sid:2200073; rev:1;)

This rule is included in the provided signature file decoder-events.rules.

To match on individual TCP packet after defragmentation, one can use the following rule:

alert tcp-pkt any any -> any 80 (msg:"HTTP dl"; content:"Get /download.php"; sid:1; rev:1;)

It will try to find a per-packet match. This means that if the request is cut in two parts, there will be no detection.

At the TCP level, we’ve got three packets but one of them is invalid because of an invalid TCP windows. Suricata can alert on this by using the following rules:

alert tcp any any -> any any (msg:"SURICATA STREAM ESTABLISHED packet out of window"; stream-event:est_packet_out_of_window; sid:2210020; rev:1;)

This rule is included in the provided signature file stream-events.rules.

So at the stream level, we’ve got data made of packets #1, #2, #3 and #5. A matching rule at this level would be:

alert tcp any any -> any any (msg:"HTTP download"; flow:established,to_server; content:"Get /download.php";)

This is a case sensitive rule and this is not resistant to basic transformation on the request such as adding spaces between Get and the URI.

The data is part of an HTTP stream and it is normalized to avoid any application level manipulation that could alter the detection by signatures.
In Suricata, we can use the following rule to have a match on normalized field:

alert http any any -> any any (msg:"Download"; content: "GET"; http_method; content: "/download.php"; http_uri)

The match is made when the traffic is identified as HTTP and we want the HTTP request method to be GET and the URL to match “/download.php”. We’ve got here one of the biggest advantage of Suricata, dedicated keywords and protocol recognition allow to write rule which are almost direct expression of our thinking.

I’m sure most of you are happy not to have this job cleanly done by Suricata!

Installing Debian sid on a XPS 15

Since this morning, I’m the owner of a XPS 15 end-2012 edition. The model I have come with a hard drive and a SSD and it is pre-installed with Windows 8. As it is not a good choice for a OS you want to use the laptop for real work, I’ve installed a Debian sid on it.

On the laptop, I’ve received the 32Go SSD was formatted with a single 8Go partition which was not used by Windows. The Windows installation was made on the classic hard drive.

From Windows, I was able to change the size of the system partition on the hard drive (the big one host OS and data) and doing so, I’ve managed to create a new partition to host my Linux data.

I’ve made the installation using an USB stick prepared with unetbootin. I had to choose the HD media mode to be able to correctly run the install procedure.

To start the install, you need to go in the bios by pressing F2. You then have to activate the legacy mode which will allow you to run an OS like Debian. To switch between the different OS, you have to hit F12 at start to be able to choose between Legacy (for GNU/Linux) and UEFI (for Windows).

The major trick during the install was to delete the partition table on the SSD which was in GPT style (and thus difficultly supported by grub). I’ve created instead a standard old-style partition table on it. I was then able to put my Debian system on the SSD without any problem.

All devices seems to work fine with Debian Sid. So if we omit the optimus Nvidia card issue, this is a computer that I recommend.