Development on To Linux and beyond !

Slides of my talks at Lecce

Wed, 18 Feb 2015 10:44:41 +0000

I’ve been invited by SaLUG to Lecce to give some talks during their Geek Evening. I’ve done a talk on nftables and one of suricata.

Lecce by night

The nftables talk was about the motivation behind the change from iptables.

Here are the slides: Nftables

The talk on Suricata was explaining the different feature of Suricata and was showing how I’ve used it to make a study of SSH bruteforce.

Speeding up scapy packets sending

Thu, 17 Apr 2014 14:11:58 +0000

Sending packets with scapy

I’m currently doing some code based on scapy. This code reads data from a possibly huge file and send a packet for each line in the file using the contained information.
So the code contains a simple loop and uses sendp because the frame must be sent at layer 2.

def run(self):
         filedesc = open(self.filename, 'r')
         # loop on read line
         for line in filedesc:
             # Build and send packet
             sendp(pkt, iface = self.iface, verbose = verbose)
             # Inter packet treatment

Doing that the performance are a bit deceptive. For 18 packets, we’ve got:

Using linux perf tools for Suricata performance analysis

Mon, 18 Nov 2013 12:59:59 +0000

Introduction

Perf is a great tool to analyse performances on Linux boxes. For example, perf top will give you this type of output on a box running Suricata on a high speed network:

Events: 32K cycles                                                                                                                                                                                                                            
 28.41%  suricata            [.] SCACSearch
 19.86%  libc-2.15.so        [.] tolower
 17.83%  suricata            [.] SigMatchSignaturesBuildMatchArray
  6.11%  suricata            [.] SigMatchSignaturesBuildMatchArrayAddSignature
  2.06%  suricata            [.] tolower@plt
  1.70%  libpthread-2.15.so  [.] pthread_mutex_trylock
  1.17%  suricata            [.] StreamTcpGetFlowState
  1.10%  libc-2.15.so        [.] __memcpy_ssse3_back
  0.90%  libpthread-2.15.so  [.] pthread_mutex_lock

The functions are sorted by CPU consumption. Using arrow key it is possible to jump into the annotated code to see where most CPU cycles are used.

Adding a force build to all builders

Fri, 20 Sep 2013 10:56:26 +0000

Recent versions of buildbot, the continuous integration framework don’t allow by default the force build feature.
This feature can be used to start a build on demand. It is really useful when you’ve updated the build procedure or when you want to test new branches.

It was a little tricky to add it, so I decided to share it. If c is the name of the configuration you build in your master.cfg, you can add after all builders declarations:

Ulogd 2.0.2, my first release as maintainer

Mon, 04 Mar 2013 00:13:13 +0000

Objectives of this release

So it is my first ulogd2 release as maintainer. I’ve been in charge of the project since 2012 October 30th and this was an opportunity for me to increase my developments on the project. Roadmap was almost empty so I’ve decided to work on issues that were bothering me as a user of the project. I’ve also included two features which are connection tracking event filtering and a Graphite output module. Ulogd is available on Netfilter web site

Coccigrep improved func operation

Mon, 10 Sep 2012 15:22:02 +0000

Coccigrep 1.11 is now available and mainly features some improvements related to the func search. The func operation can be used to search when a structure is used as argument of a function. For example, to search where the Packet structures are freed inside Suricata project, one can run:

$ coccigrep -t Packet -a "SCFree" -o func src/
src/alert-unified2-alert.c:1156 (Packet *p):         SCFree(p);
src/alert-unified2-alert.c:1161 (Packet *p):         SCFree(p);
...
src/alert-unified2-alert.c:1368 (Packet *pkt):         SCFree(pkt);

Run a build on all commits in a git branch

Tue, 07 Aug 2012 09:37:15 +0000

Sometime, you need to check that all the commits in a branch are building correctly. For example, when a rebase has been done, it is possible you or diff has made a mistake during the operation. The building operation can be run against all commits of the current branch with the following one-liner (splitted here for more readability):

for COMMIT in $(git log --reverse --format=format:%H origin/master..HEAD); do
    git checkout ${COMMIT} ;
    make -j8 1>/dev/null || { echo "Commit $COMMIT don't build";  break; }
done

The idea is trivial, we build the list of commits with git log using a simple format string (to get only the hash). We add the reverse tag to start from the oldest commit.
For each commit, we checkout and run the build command. If the build fails, we exit from the loop.

Set or unset define variables in Coccigrep

Tue, 31 Jul 2012 09:10:09 +0000

Following a discussion with the great Julia Lawall, she added a new feature in coccinelle: it is now possible to define as set or unset some variables. This option has been added in coccigrep 1.9 and requires coccinelle 1.0-rc14.

For example, let’s have a code like Suricata where a lot of unit tests are implemented. The structure of the code is the following:

REGULAR CODE

#ifdef UNITTESTS
 TEST CODE
#endif

When doing search in the regular code, you don’t want to be bothered by results found in the test code. To obtain this result, you can pass the -U UNITTESTS option to coccigrep to tell him to consider UNITTESTS variable as undefined. If you want to define a variable, you can use the -D flag.

What’s new in coccigrep 1.6?

Mon, 07 Nov 2011 23:07:29 +0000

I did not write any article on coccigrep since the 1.0 release. Here is an update on what has been added to the software since that release.

C++ support

Coccinelle has a basic C++ support which can be activated by using the –cpp flag in coccigrep.

Patches information

The -L -v options on command line will display a description of the match available on the system.

$ coccigrep -L -v
set: Search where a given attribute of structure 'type' is set
 * Confidence: 80%
 * Author: Eric Leblond <eric@regit.org>
 * Arguments: type, attribute
 * Revision: 2

For the developer, this is obtained from structured comments put at the start of the cocci file:

Acquisition systems and running modes evolution of Suricata

Thu, 06 Oct 2011 23:06:24 +0000

Some new features have recently reach Suricata’s git tree and will be available in the next development release. I’ve worked on some of them that I will describe here.

Multi interfaces support and new running modes

Configuration update

IDS live mode in suricata (pcap, pf_ring, af_packet) now supports the capture on multiple interfaces. The syntax of the YAML configuration file has evolved and it is now possible to set per-interface variables.

For example, it is possible to define pfring configuration with the following syntax:

Playing a bit with vim macros

Mon, 27 Jun 2011 06:31:31 +0000

During one of my recent coding, I had to modify a signature file for suricata. The file was looking like this:

alert pkthdr any any -> any any (msg:"SURICATA ICMPv4 unknown code"; decode-event:icmpv4.unknown_code; sid:2200024; rev:1;)
alert pkthdr any any -> any any (msg:"SURICATA ICMPv4 truncated packet"; decode-event:icmpv4.ipv4_trunc_pkt; sid:2200025; rev:1;)
alert pkthdr any any -> any any (msg:"SURICATA ICMPv4 unknown version"; decode-event:icmpv4.ipv4_unknown_ver; sid:2200026; rev:1;)
alert pkthdr any any -> any any (msg:"SURICATA ICMPv6 packet too small"; decode-event:icmpv6.pkt_too_small; sid:2200027; rev:1;)
alert pkthdr any any -> any any (msg:"SURICATA ICMPv6 unknown type"; decode-event:icmpv6.unknown_type; sid:2200028; rev:1;)
alert pkthdr any any -> any any (msg:"SURICATA ICMPv6 unknown code"; decode-event:icmpv6.unknown_code; sid:2200029; rev:1;)

The modification was to decrease the number behind <em<sid by 24 for each signatures.