Jun 082012
 

Les transparents de ma présentation du SSTIC sont disponibles : Utilisation malveillante des suivis de connexions. Merci aux organisateurs du SSTIC d’avoir accepté mon papier!

Des vidéos de démonstration sont disponibles sur ce post: Playing with Network Layers to Bypass Firewalls’ Filtering Policy

L’outil de test openvsp est disponible sur cette page.

Jun 072012
 
Scapy BPF filtering is not working when some exotic interface are used. This includes Virtualbox interface such as vboxnet. For example, the following code will not work if the interface is a virtualbox interface:
build_filter = "src host %s and src port 21"
sniff(iface=iface, prn=callback, filter=build_filter)
To fix this, you can use the lfilter option. The filtering is now done inside Scapy. This is powerful but less efficient. The code can be modified like this:
build_lfilter = lambda (r): TCP in r and r[TCP].sport == 21 and r[IP].src == ip
sniff(iface=iface, prn=callback, lfilter=build_lfilter)
Tanks a lot to Guillaume Valadon for the tips!
Mar 092012
 

The slides of my CansecWest talk can now be downloaded: Playing with Network Layers to Bypass Firewalls’ Filtering Policy.

The required counter-measures are described in the Secure use of iptables and connection tracking helpers document

The associated video demonstrations are available:

First video demonstrates how to use forged IRC protocol command (DCC request) to be able to open connection to a NATed client from internet.

Second video demonstrates the effect of the attack on helpers on a non protected Netfilter Firewall.

Third video demonstrates the effect of the attack on helpers on a badly configured Checkpoint firewall.

More information will come in upcoming posts.

Feb 132012
 
Suricata is an IDS/IPS engine. To build a complete solution, you will need to use other tools. The following schema is a representation of a possible software setup in the case Suricata is used as IDS or IPS on the network. It only uses opensource components:

Suricata is used to sniff and analyse the traffic. To detect malicious traffic, it uses signatures (or rules). You can download a set of specialised rules from EmergingThreats.

To analyse the alerts generated by Suricata, you will need an interface such as, for example, snorby.

But this interface will need to have an access to the data. This is done by storing the alerts in a database. The feed of this database can be done by barnyard2 which take the files outputted by suricata in unified2 and convert and send them to a database (or to some other format).

If you want counter-measure to be taken (such as blocking for a moment attacker at the firewall level), you can use snortsam.

More complex setup can include interactions with a SIEM solution.

Nov 302011
 
Following the presentation I’ve made during the 8th Netfilter Workshop, it was decided to write a document containing the best practices for a secure use of iptables and connection tracking helpers. This document called “Secure use of iptables and connection tracking helpers” is now available on this site. It contains recommendations that should be followed carefully if you are the administrator of a Netfilter/Iptables or the developer of a Netfilter based software.
Jul 132011
 
I gave a talk about Suricata entitled Suricata, rethinking IDS/IPS at Libre Software Meeting (RMLL in french). The slides can be downloaded from the RMLL website. Thanks a lot to Christophe Brocas and Mathieu Blanc for the organisation of the security track of LSM.
Jun 022011
 

Discovering the performance boost

When doing some coding on both 1.0 and 1.1 branch of suricata, I’ve remarked that there was a huge performance improvement of the 1.1 branch over the 1.0 branch. The parsing of a given real-life pcap file was taking 200 seconds with 1.0 but only 30 seconds with 1.1. This performance boost was huge and I decide to double check and to study how such a performance boost was possible and how it was obtained: A git bisection shows me that the performance improvement was done in at least two main steps. I then decide to do a more systematic study on the performance improvement by iterating over the revision and by each time running the same test with the same basic and not tuned configuration:
suricata -c ~eric/builds/suricata/etc/suricata.yaml  -r benches/sandnet.pcap
and storing the log output.

Graphing the improvements

The following graph shows the evolution of treatment time by commits between suricata 1.0.2 and suricata 1.1beta2: It is impressive to see that improvements are located over a really short period. In term of commit date, almost everything has happened between the December 1th and December 9th. The following graph shows the same data with a zoom on the critical period: One can see that there is two big steps and a last less noticeable phase.

Identifiying the commits

The first big step in the improvement is due to commit c61c68fd:
commit c61c68fd365bf2274325bb77c8092dfd20f6ca87
Author: Anoop Saldanha
Date:   Wed Dec 1 13:50:34 2010 +0530

    mpm and fast pattern support for http_header. Also support relative modifiers for http_header
This commit has more than double previous performance. The second step is commit which also double performance. It is again by Anoop Saldanha:
commit 72b0fcf4197761292342254e07a8284ba04169f0
Author: Anoop Saldanha
Date:   Tue Dec 7 16:22:59 2010 +0530

    modify detection engine to carry out uri mpm run before build match array if alproto is http and if sgh has atleast one sig with uri mpm set
Other improvements were made a few hours later by Anoop who succeeded in a 20% improvements with:
commit b140ed1c9c395d7014564ce077e4dd3b4ae5304e
Author: Anoop Saldanha
Date:   Tue Dec 7 19:22:06 2010 +0530

    modify detection engine to run hhd mpm before building the match array
The motivation of this development was the fact that the developers were knowing that the match on http_headers was not optimal because it was using a single pattern search algorithm. By switching to a multi-pattern match algorithm, they know it will do a great prefilter job and increase the speed. Here’s the quote of Victor Julien’s comment and explanation:
We knew that at the time we inspected the http_headers and a few other buffers for each potential signature match over and over again using a single pattern search algorithm. We already knew this was inefficient, so moving to a multi-pattern match algorithm that would prefilter the signatures made a lot of sense even without benching it.
Finaly two days later, there is a serie of two commits which brings a other 20-30% improvements :
commit 8bd6a38318838632b019171b9710b217771e4133
Author: Anoop Saldanha
Date:   Thu Dec 9 17:37:34 2010 +0530

    support relative pcre for http header. All pcre processing for http header moved to hhd engine

commit 2b781f00d7ec118690b0e94405d80f0ff918c871
Author: Anoop Saldanha
Date:   Thu Dec 9 12:33:40 2010 +0530

    support relative pcre for client body. All pcre processing for client body moved to hcbd engine

Conclusion

It appears that all the improvements are linked with modifications on the HTTP handling. Working hard on improving HTTP feature has lead to an impressive performance boost. Thanks a lot to Anoop for this awesome work. As HTTP is now most of the trafic on internet this is a really good news for suricata users !
Apr 292011
 

IPv6 global address

The global address is used in IPv6 to communicate with the outside world. This is thus the one that is used as source for any communication and thus in a way identify you on Internet. Below is a dump of an interface configuration:
eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:22:15:64:42:bd brd ff:ff:ff:ff:ff:ff
    inet6 2a01:f123:1234:5bd0:222:15ff:fe64:42bd/64 scope global dynamic 
       valid_lft 86314sec preferred_lft 86314sec
    inet6 fe80::222:15ff:fe64:42bd/64 scope link 
       valid_lft forever preferred_lft forever
The global address is here 2a01:f123:1234:5bd0:222:15ff:fe64:42bd/64. It is build by using the prefix and adding an identifier build with the hardware address. For example, here the hardware address is 00:22:15:64:42:bd and the global IPv6 address is ending with 22:15ff:fe64:42bd. It is thus easy to go from the IPv6 global address to the hardware address. To fix this issue and increase the privacy of network user, privacy extensions have been developed.

Privacy extensions

The RFC 3041 describes how to build and use temporary addresses that will be used as source address for connection to the outside world. To activate this feature, you simply have to modify an entry in /proc. For example to activate the feature on eth0, you can do
echo "2">/proc/sys/net/ipv6/conff/eth0/use_tempaddr
The usage of the option is detailled in the must-read ip-sysctl.txt file:
use_tempaddr - INTEGER
        Preference for Privacy Extensions (RFC3041).
          <= 0 : disable Privacy Extensions
          == 1 : enable Privacy Extensions, but prefer public
                 addresses over temporary addresses.
          >  1 : enable Privacy Extensions and prefer temporary
                 addresses over public addresses.
        Default:  0 (for most devices)
                 -1 (for point-to-point devices and loopback devices)
After network restart (a simple ifdown, ifup of the interface is enough), the output of the ip a command looks like that:
eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:22:15:64:42:bd brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.129/24 brd 192.168.1.255 scope global eth0
    inet6 2a01:f123:1234:5bd0:21f1:f624:d2b8:3702/64 scope global temporary dynamic 
       valid_lft 86314sec preferred_lft 2914sec
    inet6 2a01:f123:1234:5bd0:222:15ff:fe64:42bd/64 scope global dynamic 
       valid_lft 86314sec preferred_lft 86314sec
    inet6 fe80::222:15ff:fe64:42bd/64 scope link 
       valid_lft forever preferred_lft forever
A new temporary address has been added. After preferred_lft seconds, it becomes deprecated and a new address is added:
eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:22:15:64:42:bd brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.129/24 brd 192.168.1.255 scope global eth0
    inet6 2a01:f123:1234:5bd0:55c3:7efd:93d1:5057/64 scope global temporary dynamic 
       valid_lft 85009sec preferred_lft 1672sec
    inet6 2a01:f123:1234:5bd0:21f1:f624:d2b8:3702/64 scope global temporary deprecated dynamic 
       valid_lft 82077sec preferred_lft 0sec
    inet6 2a01:f123:1234:5bd0:222:15ff:fe64:42bd/64 scope global dynamic 
       valid_lft 86398sec preferred_lft 86398sec
    inet6 fe80::222:15ff:fe64:42bd/64 scope link 
       valid_lft forever preferred_lft foreverr
The deprecated address is removed when the valid_lft counter reach zero second.

Some more tuning

The default duration for a prefered adress is of one day. This can be changed by modifying the temp_prefered_lft variable. For example, you can add to sysctl.conf:
net.ipv6.conf.eth0.temp_prefered_lft = 7200
The default validity length of the addresses can be changed via the temp_valid_lft variable. The max_desync_factor set the max random time to wait before asking a new address. This is used to avoid that all computers in network ask for an address at the same time. On side effect is that if you set the prefered or valid time to a low value, the max_desync_factor must also be decreased. If not, there will be long time period without temporary address. If temp_prefered_lft is multiple time lower than temp_valid_lft, then the deprecated addresses will accumulate. To avoid overloading the kernel, a maximum number of addresses is set. Equal to 16 by default, it can be changed by setting the max_addresses sysctl variable.

Known issues and problems

As the temporary address is used for connection to the outside and has a limited duration, some long duration connections (tink ssh) will be cut when the temporary address is removed. I’ve also observed a problem when the maximum number of addresses is reached:
ipv6_create_tempaddr(): retry temporary address regeneration.
ipv6_create_tempaddr(): regeneration time exceeded. disabled temporary address support.
The result was that the temporary address support was disabled and the standard global address was used again. When setting temp_prefered_lft to 3600 and keeping temp_valid_ft to default value, the problem is reproduced easily.

Conclusion

The support of IPv6 privacy extensions is correct but the lack of link with existing connection can cause the some services to be disrupted. A easy to use per-software selection of address could be really interesting to avoid these problems.
Feb 162011
 
Following my preceding post on suricata multithread performance I’ve decided to continue to work on the subject. By using perf-tool, I found out that when the number of detect threads was increasing, more and more time was used in a spin lock. One of the possible explanation is that the default running mode for pcap file (RunModeFilePcapAuto) is not optimal. The only decode thread take some time to treat the packets and he is not fast enough to send data to the multiple detect threads. This is triggering a lot of wait and a CPU usage increase. Following a discussion with Victor Julien, I decide to give a try to an alternate run mode for working on pcap file, RunModeFilePcapAutoFp. The architecture of this run mode is different. A thread is in charge of the reading of the file and the treatment of packets is done in a pool of threads (from decode to output). The augmentation of the power of decoding and the limitation of the ratio decode/detect would possibly bring some scalability. The following graph is a comparison of the Auto mode and the FP mode on the test system described in the previous post (A 24 threads/core server parses a 6.1Go pcap file). It displays the number of packets per second in function of the number of threads: The performance difference is really interesting. The FP mode shows a increase of the performance with the number of threads. This is far better than the Auto run mode where performance decrease with the number of threads. As pointed out in a discussion on the OISF-users mailing list, multithread tuning has a real impact on performance. The result of the tests I’ve done are significant but they only apply to the parsing of a big pcap file. You will have to tune Suricata to see how to take the best of it on your system.