Linux 3.13 is out Linux 3.13 is out bringing among other thing the first official release of nftables. nftables is the project that aims to replace the existing {ip,ip6,arp,eb}tables framework aka iptables. nftables version in Linux 3.13 is not yet complete. Some important features are missing and will be introduced in the following Linux versions. It is already usable in most cases but a complete support (read nftables at a better level than iptables) should be available in Linux 3.15. ...

Introduction I’m running a dedicated server to host some internet services. The server runs Debian. I’ve installed logstash on it to do a bit of monitoring of my system logs and suricata. I’ve build a set of dashboards. The screenshot below shows a part of the one being dedicated to suricata: ...

New features in ulogd 2.0.3 release Database framework update ulogd 2.0.3 implements two new optional modes for database connections: backlog system to avoid event loss in case of database downtime running mode where acquisition is made in one thread and queries to databases are made in separate threads to reduce latency in the treatment of kernel messages These two modes are described below. Postgresql update Postgresql output plugin was only offering a small subset of Postgresql connection-related options. It is now possible to use the connstring to use all possible parameters of libpq param keywords. If set, this variable has precedence on other variables. ...

Introduction Perf is a great tool to analyse performances on Linux boxes. For example, perf top will give you this type of output on a box running Suricata on a high speed network: Events: 32K cycles 28.41% suricata [.] SCACSearch 19.86% libc-2.15.so [.] tolower 17.83% suricata [.] SigMatchSignaturesBuildMatchArray 6.11% suricata [.] SigMatchSignaturesBuildMatchArrayAddSignature 2.06% suricata [.] tolower@plt 1.70% libpthread-2.15.so [.] pthread_mutex_trylock 1.17% suricata [.] StreamTcpGetFlowState 1.10% libc-2.15.so [.] __memcpy_ssse3_back 0.90% libpthread-2.15.so [.] pthread_mutex_lock The functions are sorted by CPU consumption. Using arrow key it is possible to jump into the annotated code to see where most CPU cycles are used. ...

Introduction logstash an opensource tool for managing events and logs. It is using elasticsearch for the storage and has a really nice interface named Kibana. One of the easiest to use entry format is JSON. Suricata is an IDS/IPS which has some interesting logging features. Version 2.0 will feature a JSON export for all logging subsystem. It will then be possible to output in JSON format: HTTP log DNS log TLS log File log IDS Alerts For now, only File log is available in JSON format. This extract meta data from files transferred over HTTP. ...

When doing some tests on Suricata, I needed to setup a small IPv6 network. The setup is simple with one laptop which is Ethernet connected to a desktop. And the desktop host a Virtualbox system. This way, the desktop can act as a router with laptop on eth0 and Vbox on vboxnet0. To setup the desktop/router, I’ve used: ip a a 4::1/64 dev eth0 ip a a 2::1/64 dev vboxnet0 echo "1">/proc/sys/net/ipv6/conf/all/forwarding To setup the laptop who already has a IPv6 public address on eth0, I’ve done: ...

I’ve just gave a talk about nftables, the iptables successor, at Kernel Recipes 2013. You can find the slides here: 2013_kernel_recipes_nftables A description of the talk as well as slides and video are available on Kernel Recipes website Here’s the video of my talk: I’ve presented a video of nftables source code evolution: The video has been generated with gource. Git history of various components have been merged and the file path has been prefixed with project name.

Recent versions of buildbot, the continuous integration framework don’t allow by default the force build feature. This feature can be used to start a build on demand. It is really useful when you’ve updated the build procedure or when you want to test new branches. It was a little tricky to add it, so I decided to share it. If c is the name of the configuration you build in your master.cfg, you can add after all builders declarations: ...

The first news is that it works! It is possible to use tc to setup QoS on IPv6 but the filter have to be updated. When working on adding IPv6 support to lagfactory, I found out by reading tc sources and specifically ll_proto.c that the keyword to use for IPv6 was ipv6. Please read that file if you need to find the keyword for an other protocol. So to send packet with Netfilter mark 5000 to a specific queue, one can use: ...

Some new features I’ve just pushed to ulogd tree a series of patches. They bring two major improvements to database handling: Backlog system: temporary store SQL query in memory if database is down. Ring buffer system: a special mode with a thread to read data from kernel and a thread to do the SQL query. The first mode is attended for preventing data loss when database is temporary down. The second one is an attempt to improve performance and the resistance to netlink buffer overrun problem. The modification has been done in the database abstraction layer and it is thus available in MySQL, PostgreSQL and DBI. ...

The problem I’ve been recently working for a customer which needed consultancy because of some unexplained Netfilter behaviors related to ICMP error messages. He authorizes me to share the result of my study and I thank him for making this blog entry possible. His problem was that one of his firewalls is using a private interconnexion with their border router and the customer did not manage to NAT all outgoing ICMP error messages. ...

Visualizing Debian packages upload Ultimate Debian Database provide a way to get information about all packages upload on Debian repositories accros time. After a discussion with Lucas Nussbaum at Distro Recipes, he made available a webpage to access to a gource compatible file format of packages upload. Using this I was able to create videos of Debian evolution over time. I’ve generated two videos showing on month of packages upload in 2000 and to compare one month in 2012. ...

Not usual setup can lead to surprise The 5th of December 2012, I’ve setup suricata in AF_PACKET IPS mode between a WiFi interface and an Ethernet interface. The result was surprising as it was leading to a crash after some time: The issue was linked with the defrag option of AF_PACKEt fanout. I’ve proposed a patch the 7th Dec 2012 and after a discussion with David Miller and Johannes Berg, Johannes has proposed a better patch which was included in official tree. So the problem is fixed for kernel superior or equal to 3.7. ...

Xtables2 xtables 2 suppress the different tables that exits in current Netfilter. If a rule only apply to a specific type of traffic (read owner id match per-example) then it just don’t match. One of the interest to have one single table is that it is possible to easily update the ruleset by just doing a single atomic swap. Manual chains can be created by hand as there are very useful to create factorized rules. ...

Top starting from left: Jan Engelhardt, Tomasz Bursztyka, Daniel Borkmann, Julien Vehent, Holger Eitzenberger, Victor Julien, Eric Leblond, Eric Dumazet, Nicolas Dichtel, David Miller, S. Park Bottom starting from left: Martin Topholm, Jesper Sander Lindgren, Pablo Neira Ayuso, Simon Horman, Jozsef Kadlecsik, Jesper Dangaard Brouer, Patrick McHardy, Thomas Graf

Introduction connMan is a network manager which has support for a lot of different layers from ethernet and WiFi to NFC and link sharing. It features automatic link switch and allow you to select your preferred type of support. The communication with UI is event based so it is easy to do as only a few windows type are needed. Discussion David Miller pointed out the fact that DHCP client is really often putting the interface in promiscuous mode and this is not a good idea as it is like having a tcpdump started on every laptop. As connMann does ahave its own implementation, they could maybe take this into account and improved the situation. This is in fact already the case as the DHCP client is using an alternate method.

Tc interaction tc interaction has been contributed by Florian Westphal. It is thus now possible to use a set match to differentiate Qos or routing of packet. This opens a wide area for experimentation. Packet and byte counters This is a fairly larger rewriting of set element and extensions which adds packets and bytes counters to the element. The syntax has been updated: ipset add <set> <elem> packets n bytes m It is also possible to do check on counters !! For example, ipset will be able to do a match on a set and to refine the selection by specifying the number of packets we must have seen before matching. Counters can also be updated in the set match. ...

Introduction This is a new kernel packet filtering framework. The only change is on iptables. Netfilter hooks, connection tracking system, NAT are unchanged. It provides a backward compatibility. nftables was released in March 2009 by Patrick Mchardy. It has been revived in the precedent months by Pablo Neira Ayuso and other hackers. Architecture It uses a pseudo-state machine in kernel-space which is similar to BPF: 4 registers: 4 general purpose (128 bits long each) + 1 verdict provides instruction set (which can be extended) Here’s a example of existing instructions: ...

Open vSwitch is a multi-layer switch. It is designed to enable network automation through programmatic extension, while still supporting standard management interfaces and protocols. Openflow is a management protocol that is supported by Open vSwitch. Openflow is has a basic support for MPLS. It features a minimum operation set to enable to configure MPLS correclty. Openflow MPLS support is partially implemented in Open vSwitch but there is some difficulties. SOme of the operations feature update of L3+ parameter like TTL. They must be updated in same manner in the MPLS header and in the packet header. And this is quite complicated as it supposed to decode the packet below MPLS. But MPLS header does not include the encapsulated ethernet type so it is almost impossible to access correctly to the packet structure. ...

Suricata and Netfilter can be better friend as they are doing some common work like decoding packet and maintaining flow table. In IPS mode, Suricata is receiving raw packet from libnetfilter_queue. It has to made the parsing of this packet but this kind of thing has also been done by kernel. So it should be possible to avoid to duplicate the work. ...