Suricata on To Linux and beyond !

Slides of my talks at Lecce

Wed, 18 Feb 2015 10:44:41 +0000

I’ve been invited by SaLUG to Lecce to give some talks during their Geek Evening. I’ve done a talk on nftables and one of suricata.

Lecce by night

The nftables talk was about the motivation behind the change from iptables.

Here are the slides: Nftables

The talk on Suricata was explaining the different feature of Suricata and was showing how I’ve used it to make a study of SSH bruteforce.

pshitt: collect passwords used in SSH bruteforce

Thu, 26 Jun 2014 08:41:02 +0000

Introduction

I’ve been playing lately on analysis SSH bruteforce caracterization. I was a bit frustrated of just getting partial information:

ulogd can give information about scanner settings
suricata can give me information about software version
sshd server logs shows username

But having username without having the password is really frustrating.

So I decided to try to get them. Looking for a SSH server honeypot, I did find kippo but it was going too far for me
by providing a fake shell access. So I’ve decided to build my own based on paramiko.

Suricata and Ulogd meet Logstash and Splunk

Fri, 07 Mar 2014 23:19:37 +0000

Some progress on the JSON side

Suricata 2.0-rc2 is out and it brings some progress on the JSON side. The logging of SSH protocol has been added:

and the format of timestamp has been updated to be ISO 8601 compliant and it is now named timestamp instead of time.

Suricata and Nftables

Wed, 05 Feb 2014 09:03:28 +0000

Iptables and suricata as IPS

Building a Suricata ruleset with iptables has always been a complicated task when trying to combined the rules that are necessary for the IPS with the firewall rules. Suricata has always used Netfilter advanced features allowing some more or less tricky methods to be used.

For the one not familiar with IPS using Netfilter, here’s a few starting points:

IPS receives the packet coming from kernel via rules using the NFQUEUE target
The IPS must received all packets of a given flow to be able to handle detection cleanly
The NFQUEUE target is a terminal target: when the IPS verdicts a packet, it is or accepted (and leave current chain)

Investigation on an attack tool used in China

Sun, 02 Feb 2014 15:28:32 +0000

Log analysis experiment

I’ve been playing lately with logstash using data from the ulogd JSON output plugin and the Suricata full JSON output as well as standard system logs.

Ulogd is getting Netfilter firewall logs from Linux kernel and is writing them in JSON format. Suricata is doing the same with alert and other traces. Logstash is getting both log as well as sytem log. This allows to create some dashboard with information coming from multiple sources. If you want to know how to configure ulogd for JSON output check this post. For suricata, you can have a look at this one.

Using linux perf tools for Suricata performance analysis

Mon, 18 Nov 2013 12:59:59 +0000

Introduction

Perf is a great tool to analyse performances on Linux boxes. For example, perf top will give you this type of output on a box running Suricata on a high speed network:

Events: 32K cycles                                                                                                                                                                                                                            
 28.41%  suricata            [.] SCACSearch
 19.86%  libc-2.15.so        [.] tolower
 17.83%  suricata            [.] SigMatchSignaturesBuildMatchArray
  6.11%  suricata            [.] SigMatchSignaturesBuildMatchArrayAddSignature
  2.06%  suricata            [.] tolower@plt
  1.70%  libpthread-2.15.so  [.] pthread_mutex_trylock
  1.17%  suricata            [.] StreamTcpGetFlowState
  1.10%  libc-2.15.so        [.] __memcpy_ssse3_back
  0.90%  libpthread-2.15.so  [.] pthread_mutex_lock

The functions are sorted by CPU consumption. Using arrow key it is possible to jump into the annotated code to see where most CPU cycles are used.

Logstash and Suricata for the old guys

Mon, 28 Oct 2013 10:47:31 +0000

Introduction

logstash an opensource tool for managing events and logs. It is using elasticsearch for the storage and has a really nice interface named Kibana. One of the easiest to use entry format is JSON.

Suricata is an IDS/IPS which has some interesting logging features. Version 2.0 will feature a JSON export for all logging subsystem. It will then be possible to output in JSON format:

HTTP log
DNS log
TLS log
File log
IDS Alerts

For now, only File log is available in JSON format. This extract meta data from files transferred over HTTP.

WiFi interface and suricata AF_PACKET IPS mode

Tue, 26 Mar 2013 15:24:45 +0000

Not usual setup can lead to surprise

The 5th of December 2012, I’ve setup suricata in AF_PACKET IPS mode between a WiFi interface and an Ethernet interface. The result was surprising as it was leading to a crash after some time:

The issue was linked with the defrag option of AF_PACKEt fanout. I’ve proposed a patch the 7th Dec 2012 and after a discussion with David Miller and Johannes Berg, Johannes has proposed a better patch which was included in official tree. So the problem is fixed for kernel superior or equal to 3.7.

Victor Julien, Suricata and Netfilter

Mon, 11 Mar 2013 15:04:14 +0000

Suricata and Netfilter can be better friend as they are doing some common work like decoding packet and maintaining flow table.

In IPS mode, Suricata is receiving raw packet from libnetfilter_queue. It has to made the parsing of this packet but this kind of thing has also been done by kernel. So it should be possible to avoid to duplicate the work.

Some statistics about Suricata 1.4

Thu, 13 Dec 2012 16:11:00 +0000

A huge work

Suricata 1.4 has been released December 13th 2012 and it has been a huge work. The number of modifications is just impressing:

390 files changed, 25299 insertions(+), 11982 deletions(-)

The following video is using gource to display the evolution of Suricata IDS/IPS source code between version 1.3 and version 1.4. It only displays the modified files and do not show the files existing at start.

A collaborative work

A total of 11 different authors have participated to this release. The following graph generated by gitstats shows the number of lines of code by author:

About Suricata and a kernel oops in AF_PACKET

Wed, 05 Dec 2012 10:38:19 +0000

Introduction

Kernel oops have been reported by some users running Suricata with AF_PACKET multiple thread capture activated. This is due to a bug I’ve introduced in AF_PACKET when fixing an other bug.

Which kernel not to use with Suricata in AF_PACKET mode

The following kernel version will surely crash if Suricata or any other program is used with AF_PACKET capture with multiple capture threads:

Linux 3.2.30 to 3.2.33
Linux 3.4.12 to 3.4.18
Linux 3.5.5 to 3.5.7
Linux 3.6.0 to 3.6.6

If only one capture thread is used there is no risk of crash. If you are running a vulnerable kernel, your configuration should looks like:

Flow reconstruction and normalization in Suricata

Thu, 15 Nov 2012 17:36:06 +0000

The naive approach would consider that an IDS is just taking packet and doing a lot of matching on it. In fact, this is not at all what is happening. An IDS/IPS like Suricata is in fact rebuilding the data stream and in case of known protocols it is even normalizing the data stream and providing keyword which can be used to match on specific field of a protocol.

Let’s say, we a rule to match on a HTTP request where method is GET and the URL is “/download.php”.

Display suricata signatures in Latex

Tue, 23 Oct 2012 18:04:11 +0000

lstlisting is a convenient way to display code when using latex. It has no definition for suricata rules language and I’ve cooked one:

\lstdefinelanguage{suricata}
{morekeywords= {alert, tcp, http, tls, ip, ipv4, ipv4, drop, pass, sid, priority, rev, classtype, threshold, metadata, reference, tag, msg, content, uricontent, pcre, ack, seq, depth, distance, within, offset, replace, nocase, fast\_pattern, rawbytes, byte\_test, byte\_jump, sameip, ip\_proto, flow, window, ftpbounce, isdataat, id, rpc, dsize, flowvar, flowint, pktvar, noalert, flowbits, stream\_size, ttl, itype, icode, tos, icmp\_id, icmp\_seq, detection\_filter, ipopts, flags, fragbits, fragoffset, gid, nfq\_set\_mark, tls.version, tls.subject, tls.issuerdn, tls.fingerprint, tls.store, http\_cookie, http\_method, urilen, http\_client\_body, http\_server\_body, http\_header, http\_raw\_header, http\_uri, http\_raw\_uri, http\_stat\_msg, http\_stat\_code, http\_user\_agent, ssh.protoversion, ssh.softwareversion, ssl\_version, ssl\_state, byte\_extract, file\_data, dce\_iface, dce\_opnum, dce\_stub\_data, asn1, filename, fileext, filestore, filemagic, filemd5, filesize, l3\_proto, luajit},
otherkeywords={ipv4-csum, tcpv4-csum, tcpv6-csum, udpv4-csum, udpv6-csum, icmpv4-csum, icmpv6-csum, decode-event, app-layer-event, engine-event, stream-event},
sensitive=true,
morecomment=[l]{//},
morecomment=[s]{/*}{*/},
morestring=[b]",
}

To use it, you can simply add this code at start of your tex file and you can then use it:

Defend your network from Microsoft Word upload with Suricata and Netfilter

Tue, 09 Oct 2012 14:17:13 +0000

Introduction

Some times ago, I’ve blogged about new IPS features in Suricata 1.1 and did not find at the time
any killer application of the nfq_set_mark keyword. When using Suricata in Netfilter IPS mode, this keyword allows you to set the Netfilter mark on the packet when a rule match.
This mark can be used by Netfilter or by other network subsystem to differentiate the treatment to apply to the packet.

A new unix command mode in Suricata

Tue, 18 Sep 2012 22:21:05 +0000

Introduction

I’ve been working for the past few days on a new Suricata feature. It is available in Suricata 1.4rc1.

Suricata can now listen to a unix socket and accept commands from the user. The exchange protocol is JSON-based and the format of the message has been done to be generic and it is described in this commit message. An example script called suricatasc is provided in the source and installed automatically when updating Suricata.

New AF_PACKET IPS mode in Suricata

Tue, 04 Sep 2012 20:53:53 +0000

A new Suricata IPS mode

Suricata IPS capabilities are not new. It is possible to use Suricata with Netfilter or ipfw to build a state-of-the-art IPS. On Linux, this system has not the best throughput performance. Patrick McHardy’s work on netlink: memory mapped I/O should bring some real improvement but this is not yet available.

I’ve thus decided to do an implementation of IPS based on AF_PACKET (read raw socket). The idea is based on one of the snort’s running mode. It peers two network interfaces and all packets received from one interface are sent to the other interface (if a signature with drop keyword does not fired on the packet). This requires to dedicate two network interfaces for Suricata but this provide a simple bridge system. As suricata is using latest AF_PACKET features (read load balancing), it was possible to build something really promising.

Suricata new TLS fingerprint and TLS store keywords.

Mon, 27 Aug 2012 17:10:49 +0000

Suricata TLS support

Victor Julien has just merged to main tree a branch containing some interesting new TLS related features. They have been contributed by me and Jean-Paul Roliers.

This patchset introduces TLS logging and brings some new keywords to Suricata engine.
Here’s the list of all TLS related keywords that are available in latest Suricata git:

tls.version: match on version of protocol
tls.subject: match on subject of certificate
tls.issuerdn: match on issuer DN of certificate
tls.fingerprint: match on SHA1 fingerprint of certificate
tls.store: store the certificate on disk

You will find detailed explanation below.

Suricata, to 10Gbps and beyond

Mon, 30 Jul 2012 21:03:19 +0000

Introduction

Since the beginning of July 2012, OISF team is able to access to a server where one interface is receiving
some mirrored real European traffic. When reading “some”, think between 5Gbps and 9.5Gbps
constant traffic. With that traffic, this is around 1Mpps to 1.5M packet per seconds we have to study.

The box itself is a standard server with the following characteristics:

CPU: One Intel(R) Xeon(R) CPU E5-2680 0 @ 2.70GHz (16 cores counting Hyperthreading)
Memory: 32Go
capture NIC: Intel 82599EB 10-Gigabit SFI/SFP+

The objective is simple: be able to run Suricata on this box and treat the whole
traffic with a decent number of rules. With the constraint not to use any non
official system code (plain system and kernel if we omit a driver).

Building Suricata for OpenBSD 4.9 and over

Fri, 27 Apr 2012 14:57:47 +0000

It seems OpenBSD upgrade are done to give maintenance work to the developers of third-party application. In a way, OpenBSD fight against the economic crisis: It gives jobs to developers and if you want some performance you need a powerful thus new computer.

Let’s stop bashing and be serious: Suricata was building fine on OpenBSD 4.8 but the build was failing on subsequent version. This was link with an include modification around the “socket.h” file. It is now mandatory to include “types.h” before “socket.h” to avoid compilation error. The patch 0001-Fix-OpenBSD-compilation.patch.gz fixes the build.

Using AF_PACKET zero copy mode in Suricata

Thu, 23 Feb 2012 18:25:15 +0000

Victor Julien has just pushed a new feature to suricata’s git tree. It brings improvements to the AF_PACKET capture mode.

This capture mode can be used on Linux. It is the native way to capture packet. Suricata is able to use the interesting new multithreading feature provided by AF_PACKET on recent kernels: it is possible to have multiple capture threads receiving the packet of a single interface.

The commits add mmaped ring buffer support to AF_PACKET capture and also provide a zero copy mode. Mmaped ring buffer is mechanism similar to the one used by PF_RING. The kernel allocates some memory to store the packets and share this memory with the capture process. Instead of sending messages, the kernel just write to the shared memory and the process capture reads it. This is less consuming in term of CPU ressource and helps to increase the capture rate. But the main avantage of this technique is that the capture process can treat the packets without making a copy and this saves a lot of time

Ecosystem of Suricata

Mon, 13 Feb 2012 16:46:32 +0000

Suricata is an IDS/IPS engine. To build a complete solution, you will need to use other tools.

The following schema is a representation of a possible software setup in the case Suricata is used as IDS or IPS on the network. It only uses opensource components:

Suricata is used to sniff and analyse the traffic. To detect malicious traffic, it uses signatures (or rules). You can download a set of specialised rules from EmergingThreats.

Acquisition systems and running modes evolution of Suricata

Thu, 06 Oct 2011 23:06:24 +0000

Some new features have recently reach Suricata’s git tree and will be available in the next development release. I’ve worked on some of them that I will describe here.

Multi interfaces support and new running modes

Configuration update

IDS live mode in suricata (pcap, pf_ring, af_packet) now supports the capture on multiple interfaces. The syntax of the YAML configuration file has evolved and it is now possible to set per-interface variables.

For example, it is possible to define pfring configuration with the following syntax:

OISF brainstorming: planning phase 3 (take 3)

Mon, 19 Sep 2011 23:42:25 +0000

GEO IP

Idea is to add a keyword that would be used to interact with GEOIP database (free at least) and be able to use it to detect things like control canal. For example, an IRC server in an non common country is certainly a control canal.

Live ruleset swap

A must have! This is vital for critical environnement. This is very costly in memory and this should be an option to avoid exploding low memory boxes.

OISF brainstorming: planning phase 3 (take 2)

Mon, 19 Sep 2011 22:49:17 +0000

DNS fast flux/anomaly detection

The idea is to detect malware and other things by collecting the DNS request and their answer and detecting anomaly. For example, if an host is making a lot of request to a domain.

First part of the job on Suricata is to log all requests and their answer. Then analysis can occurs in the database.

File extraction

This is a work under progress linked with a third party contract. It permit to store exchanged files on disk for some application level protocol. It is possible to say: “store the file, if the content type is different from the extension”. File extraction works currently on HTTP. It focus on POST request to detect uploaded file.

Oisf brainstorming: planning phase 3 (take 1)

Mon, 19 Sep 2011 21:52:28 +0000

Performance improvement

As shown by Victor’s latest work on performance counters, there is a lot of work that can be done to improve performance. They are currently good but there is place for improvement. Proposal to provide off-loading or clustering is done. This is heavily discussed but as pointed out by Victor, it will be more interesting to do this in the next phase. Phase 3 should focus in improvement of current code. This will permit to use the upcoming Suricata killing features like global flow variable.

Matt Jonkman: development avancement

Mon, 19 Sep 2011 21:14:59 +0000

Phase 2 development is almost over now. Among the completed major features:

Multithread
protocol discovery
smb logging
HTTP logging
flowvars

One of the advantage of Suricata over Snort is protocol discovery combined to HTTP parsing by libhtp. It provides a huge improvement over Snort as a lot of bad flow are using HTTP on non standard ports.

Victor Julien: Development status

Mon, 19 Sep 2011 20:49:42 +0000

Work has started in september 2007. The work depends on some externel library like multithread of input handling library. The main external depedency is libhtp which is initally developped by Ivan Ristic.

The development is managed in a single git repository. Victor is the only one with commit right. The review are done by Victor and cross review are made by developpers.

Work unit for developers are tasks which are written by Victor and describe a specific task to do. This task are mainly done by OISF funded developers. Some simpler task are let to the comunity and everyone can help with this.

Slides of my Suricata talk at Libre Software Meeting

Wed, 13 Jul 2011 08:55:15 +0000

I gave a talk about Suricata entitled Suricata, rethinking IDS/IPS at Libre Software Meeting (RMLL in french). The slides can be downloaded from the RMLL website.

Thanks a lot to Christophe Brocas and Mathieu Blanc for the organisation of the security track of LSM.

About Suricata performance boost between 1.0 and 1.1beta2

Thu, 02 Jun 2011 19:45:03 +0000

Discovering the performance boost

When doing some coding on both 1.0 and 1.1 branch of suricata, I’ve remarked that there was a huge performance improvement of the 1.1 branch over the 1.0 branch. The parsing of a given real-life pcap file was taking 200 seconds with 1.0 but only 30 seconds with 1.1. This performance boost was huge and I decide to double check and to study how such a performance boost was possible and how it was obtained:

Suricata conference at Solutions Linux 2011

Wed, 11 May 2011 20:18:59 +0000

I’ve gived today a presentation about Suricata at the Solutions Linux event. It was part of the security track presided by Herve Schauer.

The slides are in french and are available here: 2011_sollinux_suricata

Joining the OISF coding staff

Wed, 20 Apr 2011 21:19:55 +0000

My collaboration with OISF has been announced today. This is an honor for me to join this excellent team on this wonderful project. I’ve taken a lot of pleasure in the past months contributing to the project and I’m sure the start of an official collaboration will lead to good things. The challenge is high and I will do my best to merit the trust.

A big thanks to all people who congrat me for this nomination.

Building Suricata under OpenBSD

Sun, 17 Apr 2011 08:17:35 +0000

Suricata 1.1beta2 has brought OpenBSD to the list of supported operating system. I’m a total newbie to OpenBSD so excuse me for the lack of respect of OpenBSD standards and usages in this documentation.

Here’s the different step, I’ve used to finalize the port starting from a fresh install of OpenBSD.

If you want to use source taken from git, you will need to install building tools:

pkg_add git libtool

automake and autoconf need to be installed to. For a OpenBSD 4.8, one can run:

Some new features of IPS mode in Suricata 1.1beta2

Wed, 13 Apr 2011 22:37:12 +0000

The IDS/IPS suricata has a native support for Netfilter queue. This brings IPS functionnalities to users running Suricata on Linux.

Suricata 1.1beta2 introduces a lot of new features related to the NFQ mode.

New stream inline mode

One of the main improvement of Suricata IPS mode is related with the new stream engine dedicated to inline. Victor Julien has a great blog post about it.

Multiqueue support

Suricata can now be started on multiple queue by using a comma separated list of queue identifier on the command line. The following syntax:

More about Suricata multithread performance

Tue, 15 Feb 2011 23:30:49 +0000

Following my preceding post on suricata multithread performance I’ve decided to continue to work on the subject.

By using perf-tool, I found out that when the number of detect threads was increasing, more and more time was used in a spin lock. One of the possible explanation is that the default running mode for pcap file (RunModeFilePcapAuto) is not optimal. The only decode thread take some time to treat the packets and he is not fast enough to send data to the multiple detect threads. This is triggering a lot of wait and a CPU usage increase. Following a discussion with Victor Julien, I decide to give a try to an alternate run mode for working on pcap file, RunModeFilePcapAutoFp.

Optimizing Suricata on multicore CPUs

Wed, 26 Jan 2011 00:20:28 +0000

Suricata IDS/IPS architecture is heavily using multithreading. On almost every runmode (PCAP, PCAP file, NFQ, …) it is possible to setup the number of thread that are used for detection. This is the most CPU intensive task as it does the detection of alert by checking the packet on the signatures. The configuration of the number of threads is done by setting
a ratio which decide of the number of threads to be run by available CPUs (detect_thread_ratio variable).

Building a suricata compliant ruleset

Sun, 16 Jan 2011 16:13:08 +0000

Introduction

During Nefilter Workshop 2008, we had an interesting discussion about the fact that NFQUEUE is a terminal decision. This has some strong implication and in particular when working with an IPS like suricata (or snort-inline at the time of the discussion): the IPS must received all packets routed by the gateway and can only issue a terminal DROP or ACCEPT verdict. It thus take precedence over all subsequent rules in the ruleset: any ACCEPT rules before the IPS rules will remove packets from IPS analysis and in the other way, any decision after the IPS rules will be ignored.

Using Suricata with CUDA

Sun, 23 May 2010 19:10:25 +0000

Suricata is a next generation IDS/IPS engineÂ developedÂ by the Open Information Security Foundation.

This article describes the installation, setup and usage of Suricata with CUDA support on a Ubuntu 10.04 64bit. For 32 bit users, simply remove 64 occurances where you find them.

Preparation

You need to download both Developper driver and Cuda driver from nvidia website. I really mean both because Ubuntu nvidia drivers are not working with CUDA.

I’ve first downloaded and installed CUDA toolkit for Ubuntu 9.04. It was straightforward: