NFWS on To Linux and beyond !

Jan Engelhardt, â€œMerge Meâ€

Tue, 12 Mar 2013 14:43:50 +0000

Xtables2

xtables 2 suppress the different tables that exits in current Netfilter. If a rule only apply to a specific type of traffic (read owner id match per-example) then it just don’t match.

One of the interest to have one single table is that it is possible to easily update the ruleset by just doing a single atomic swap.

Manual chains can be created by hand as there are very useful to create factorized rules.

NFWS group photo

Tue, 12 Mar 2013 13:34:59 +0000

Top starting from left:
Jan Engelhardt, Tomasz Bursztyka, Daniel Borkmann, Julien Vehent, Holger Eitzenberger, Victor Julien, Eric Leblond, Eric Dumazet, Nicolas Dichtel, David Miller, S. Park

Bottom starting from left:
Martin Topholm, Jesper Sander Lindgren, Pablo Neira Ayuso, Simon Horman, Jozsef Kadlecsik, Jesper Dangaard Brouer, Patrick McHardy, Thomas Graf

Tomasz Bursztyka, connMan usage of Netfilter

Tue, 12 Mar 2013 12:52:16 +0000

Introduction

connMan is a network manager which has support for a lot of different layers from ethernet and WiFi to NFC and link sharing.

It features automatic link switch and allow you to select your preferred type of support. The communication with UI is event based so it is easy to do as only a few windows type are needed.

Discussion

David Miller pointed out the fact that DHCP client is really often putting the interface in promiscuous mode and this is not a good idea as it is like having a tcpdump started on every laptop. As connMann does ahave its own implementation, they could maybe take this into account and improved the situation. This is in fact already the case as the DHCP client is using an alternate method.

Jozsef Kadlecsik, ipset status

Tue, 12 Mar 2013 11:00:51 +0000

Tc interaction

tc interaction has been contributed by Florian Westphal. It is thus now possible to use a set match to differentiate Qos or routing of packet. This opens a wide area for experimentation.

Packet and byte counters

This is a fairly larger rewriting of set element and extensions which adds packets and bytes counters to the element.

The syntax has been updated:

ipset add <set> <elem> packets n bytes m

It is also possible to do check on counters !! For example, ipset will be able to do a match on a set and to refine the selection by specifying the number of packets we must have seen before matching. Counters can also be updated in the set match.

Pablo Neira Ayuso, nftables strikes back

Tue, 12 Mar 2013 10:13:42 +0000

Introduction

This is a new kernel packet filtering framework. The only change is on iptables. Netfilter hooks, connection tracking system, NAT are unchanged.
It provides a backward compatibility. nftables was released in March 2009 by Patrick Mchardy. It has been revived in the precedent months by Pablo Neira Ayuso and other hackers.

Architecture

It uses a pseudo-state machine in kernel-space which is similar to BPF:

4 registers: 4 general purpose (128 bits long each) + 1 verdict
provides instruction set (which can be extended)

Here’s a example of existing instructions:

Simon Horman, MPLS Enlightened Open vSwitch

Mon, 11 Mar 2013 15:58:36 +0000

Open vSwitch is a multi-layer switch. It is designed to enable network automation through programmatic extension, while still supporting standard management interfaces and protocols.

Openflow is a management protocol that is supported by Open vSwitch. Openflow is has a basic support for MPLS. It features a minimum operation set to enable to configure MPLS correclty.
Openflow MPLS support is partially implemented in Open vSwitch but there is some difficulties.

SOme of the operations feature update of L3+ parameter like TTL. They must be updated in same manner in the MPLS header and in the packet header. And this is quite complicated as it supposed to decode the packet below MPLS. But MPLS header does not include the encapsulated ethernet type so it is almost impossible to access correctly to the packet structure.

Victor Julien, Suricata and Netfilter

Mon, 11 Mar 2013 15:04:14 +0000

Suricata and Netfilter can be better friend as they are doing some common work like decoding packet and maintaining flow table.

In IPS mode, Suricata is receiving raw packet from libnetfilter_queue. It has to made the parsing of this packet but this kind of thing has also been done by kernel. So it should be possible to avoid to duplicate the work.

Pablo Neira Ayuso, Netfilter summary of changes since last workshop

Mon, 11 Mar 2013 14:05:00 +0000

Pablo Neira Ayuso has made a panorama of Netfilter changes since last workshop.

On user side, the first main change to be published after last workshop, is libnetfilter_cttimeout. It allows you to define different timeout policies and to apply them to connections by using the CT target.

An other important new “feature” is a possibility to disable to automatic helper assignment. More information on
Secure use of iptables and connection tracking helpers.

Martin Topholm: DDoS experiences with Linux and Netfilter

Mon, 11 Mar 2013 10:54:17 +0000

Martin is working for one.com a local ISP and is facing some DDoS. SYN cookie was implemented but the performance were too low with performance below 300kpps which is not what was expected. In fact SYN is on a slow path with a single spin lock protecting the SYN backtrack queue. So the system behave like a single core system relatively to SYN attacks.

Jesper Dangaard Brouer has proposed a patch to move the syn cookie out of the lock but it has some downside and could not be accepted. In particular, the syncookie system needs to check every type of packet to see if they belong to a previous syn cookie response and thus a central point is needed.

David Miller: routing cache is dead, now what ?

Mon, 11 Mar 2013 10:17:21 +0000

The routing cache was maintaining a list of routing decisions. This was an hash table which was highly dynamic and was changing due to traffic. One of the major problem was the garbage collector. An other severe issue was the possibility of DoS using the increase

The routing cache has been suppressed in Linux 3.6 after a 2 years effort by David and the other Linux kernel developers. The global cache has been suppressed and some stored information have been moved to more separate resources like socket.

Fabio Massimo Di Nitto: Kronosnet.org

Mon, 11 Mar 2013 09:03:03 +0000

Kronosnet is a “I conceived it when drunk but it works well” VPN implementation. It is using an Ether TAP for the VPN to provide a lyaer 2 vpn. To avoid reinventing the wheel, it is delegating most of the work to the kernel. It supports multilink and redundancy of servers. On multilink side, 8 links can be done per-host to help redundancy.

One of the use of this project is the creation of private network in the cloud as it can be easily setup to provide redundancy and connection for a lot of clients (64k simultaneous clients). And because a layer 2 VPN is really useful for this type of usage.

Eric Leblond: ulogd2, Netfilter logging reloaded

Mon, 11 Mar 2013 07:30:30 +0000

Introduction

I’ve made yesterday a presentation of ulogd2 at Open Source Days in Copenhagen. After a brief history of Netfilter logging, I’ve described the key features of ulogd2 and demonstrate two interfaces, nf3d and djedi.

The slides are available:
Ulogd2, Netfilter logging reloaded.

Screencasts

This video demonstrates some features of nf3d:

This screencast is showing some of the capabilities of djedi:

Thanks a lot to the organizers for this cool event.

Jan Engelhardt, Xtables2: Packet Filter Evolved

Sun, 10 Mar 2013 16:27:58 +0000

Introduction

Iptables duplicate work for each family and is using a socket protocol which is far too static. Xtables2 is an ongoing effort to evolve the packet filter.
It aims at providing finer frained modification (and not the whole ruleset modification).

Capabilities

rule packing: increase cache hit.
family independent: no more IPv4 and IPv6 specific code. Only the hook remains specific as they are dependant of core network.
xt extension support
atomic replace support

xtables syntax is quite similar but not the same. libxtadm is a high-level library for ruleset inspection/manipulation.

Daniel Borkmann: Packets Sockets, BPF and Netsniff-NG

Sun, 10 Mar 2013 16:11:29 +0000

PF_PACKET introduction

This is access to raw packet inside Linux. It is used by libpcap and by other projects like Suricata.
PF_PACKET performance can be improved via dedicated features:

Zero-copy RX/TX
Socket clustering
Linux socket filtering (BPF)

BPF architecture looks like a small virtual machine with register and memory stores. It has different instructions and the kernel has its own kernel extensions to access to cpu number, vlan tag.

Netsniff-NG

Netsniff-ng is a set of minimal tools:

Tomasz Bursztyka, ConnMan usage of Netfilter: a close overview

Sun, 10 Mar 2013 15:36:08 +0000

Introduction

ConnMan is a connection manager which integrate all critical networking components. It provides a smart D-Bus API to develop an User Interface. It is plugin oriented and all different network stacks are implemented in different modules.
Connection sharing (aka tethering) is using Netfilter to setup NAT masquerading. So it is a simple usage.

Switching to nftables

Application connectivity is a more advanced part involving Netfilter as it makes a use of statistics and differenciated routing. For example, in a car, service data must be sent to manufacturer operator and not on the owner network.

Julien Vehent, AFW: Automating host-based firewalls with Chef

Sun, 10 Mar 2013 15:12:52 +0000

The problem

Centralized firewall design does not scale well when dealing with a lot of servers. It begins to collapse after a few thousands rules.
Furthermore, to be able to have an application A to connect to server B, it would take a workflow and possibly 3 weeks to get the opening.

From Service Oriented Architecture to Service Oriented Security

Service are autonomous. They call each other using a standard protocol. The architecture is described by a list of dependencies between services.
You can then specify security via things like ACCEPT Caching TO Frontend ON PORT 80.
But this force you to do provisioning each time a server start.

Jozsef Kadlecsik, Faster firewalling with ipset

Sun, 10 Mar 2013 13:51:19 +0000

Why ipset ?

iptables is enough sufficient but in some cases limit are found:

High number of rules: iptables is linear
Need to change the rules often

Independant study available at d(a)emonkeeper’s purgatory has shown that the performance of ipset are almost constant with respect to the number of filtered hosts:

History

The originating project was ippool featuring a a basic set and after some time it has been taken over by Jozsef and renamed ipset. A lot of type of sets are now handled.

Patrick McHardy: Oops, I did it: IPv6 NAT

Sun, 10 Mar 2013 13:01:41 +0000

Introduction

Harald Welte when asked about IPv6 NAT was answering: “it will be over my dead body”. It is now available in official kernel.

Reasons for adding IPv6 NAT

Dynamic IPv6 Prefixes : ISP assigning dynamic IPv6 prefixes so Internal network address change. NAT can bring you stability.
Easier test setup.
Users are asking and most operating systems have it.

To resume the arguments of NAT, Patrick McHardy used this video:

Pablo Neira Ayuso: nftables, a new packet filtering framework for Netfilter

Sun, 10 Mar 2013 12:37:15 +0000

Introduction

nftable is a kernel packet filtering framework to replaces iptables. It brings no changes in the core (conntrack, hooks).

Match logic is changed: you fetch keys and once you have your key set, you make operation on them. Advanced and specialized matchs are built upon this system.

nftables vs iptables

In iptables, extension were coded in separate files and they must be put in iptables source tree. To act, they must modify on a binary array storing the ruleset and injecting it back to the kernel. So every update involve a full download and upload of the whole ruleset.

Patrick McHardy: memory mapped netlink tree is available for testing

Fri, 26 Aug 2011 09:03:08 +0000

Patrick (aka kaber) has just made available his work on memory mapped netlink. Both the kernel and the libmnl part are available on git.kernel.org.

You can pull kernel code other net-next tree:

git pull git://git.kernel.org/pub/scm/linux/kernel/git/kaber/nl-mmap-2.6.git

Libmnl code can be fetched:

git clone git://git.kernel.org/pub/scm/linux/kernel/git/kaber/libmnl-mmap.git

Once done a NETLINK_MMAP kernel compilation option is then available via make config.

Documentation is available in the Linux tree. It is in the file: Documentation/networking/netlink_mmap.txt

Eric Leblond: Introduction to coccinelle

Wed, 24 Aug 2011 14:00:59 +0000

The Netfilter workshop being a developer conference, I’ve decided to presente an introduction to the coccinelle tool. Coccinelle is a program matching and transformation engine for the C language which is used in many place and among them in the Linux kernel. It is able to perform C clever modification in the code. If you ever had to modify multiple code files following an API change, I invite you to have a look at the slides or my Coccinelle for the newbie page. I’ve also presented my coccigrep tool which is a easy to use semantic grep.

Jesper Dangaard Brouer: CPAN module IPTables::libiptc

Wed, 24 Aug 2011 13:17:05 +0000

Jesper’s IPTables::libiptc is a perl module which allow you to modify Netfilter rules from Perl. He’s the maintener and this is available on CPAN. It currently supports up-to iptables 1.4.10 (version 0.51 of IPTables::libiptc).

It dynamically load xtables.so and libiptc.so to access to iptables feature. It is fast as it does not suffer of iptables limitation (which is running modification one by one). Performance are quite good: it takes only 16 sec to generate and implement a 80000 rules ruleset (which is quite good compare to the 42h hours that would be take by direct iptables calls)

Patrick McHardy: getting rid of the second tuple

Wed, 24 Aug 2011 10:55:21 +0000

Patrick presents one work that is aiming at getting rid of the second tuple in the connection tracking. This second tuple is only necessary when NAT is used. idea is not new but at the time the ct-extention where not available and thus it would not be possible to add it when needed. Patrick has done most of the work but there is still a missing point which is the hash function. It has to be symetrical: hash_func(src,dst) = hash_func(dst, src) and it must be very fast to avoid slowdown of the conntrack.

Ulrich Weber: IPV6 NAT

Wed, 24 Aug 2011 10:05:48 +0000

We have been ignoring the fact that NAT could have some interest in IPv6 during the latest 5 years. IPv6 will not fix everything and it may be time to reconsider NAT. There is some reasons for that:

Dynamic IPv6 prefixes: some ISP decide to not give fixed address to people
Server load balancing, DMZ
Uplink Balancing (multi-homing): this is one of the most important reason. IPv6 client can handle multiple addresses but you may want not having your user to choose their internet output.

Pablo Neira Ayuso: nfgrep: traffic classification for Netfilter/iptables

Wed, 24 Aug 2011 09:11:08 +0000

Pablo is presenting is work on protocol classification. As you may not have guess, nfgrep is not using regular expression but a descriptive language.

The basic architecture is the following:

developped layer-7 filter in userspace
filter is passed to a tool that generates byte-code
it loads the byte-code to the kernel via nfnetlink
The kernel does the classification
nfgrep match can then be used to select or mark the flow

In userspace, nfgrep and libnfgrep can be used to interact with the system. There’s also a nfgrep-test to validate filter before sending them.

Nishit Shah & Jimit Mahadevia: TCP Session Load-balancing in Active-Active HA Cluster

Wed, 24 Aug 2011 08:31:46 +0000

Cyberoam team presents their work on active active cluster. They’ve done a 2 nodes active active setup, with a primary and an auxiliary sytem. The primary take care of load balancing. The setup is using virtual MAC addresses.

To avoid split-brain problem, the primary take all decisions by always treating the SYN packet. It also transfer the NAT, marks to the auxiliary thanks to a module. This is done via a module called ipt_SYNDATA. It is placed in PREROUTING

Holger Eitzenberger: speeding up selective conntrack flush

Tue, 23 Aug 2011 13:58:52 +0000

At times it is necessary to flush UNREPLIED connection tracking entries for connectionless protocols if there are NAT rules involved. For example this is the case when a ipsec or a ppp connection goes up. Without doing that the connection are not correctly NATed because the topology change has not been taken into account.

Doing this in userspace with the conntrack-tools was taking long like minutes on some setup. They thus decide to put in kernel space and this is now only taking milliseconds instead of minutes.

Jesper Dangaard Brouer: the missing conntrack garbage collector

Tue, 23 Aug 2011 13:38:01 +0000

There is a fixed number of connection tracking entries. When reaching the maximum, new connections are simply dropped. Default maximum size is ridicully too low like using 20Mbytes oon a 12GB memory computer.

Kernel syslog message “nf_conntrack: table full, dropping, packet” is not correct because packet have just no state relatively to conntrack. Usually they get blocked by invalid rules but an adapted ruleset could let them go through.

One other problem is that adjusting the connection tracking size does not change the hash size. This results in longer search because conntrack has often to go through a list.

Jan Engelhardt: Free form discussion

Tue, 23 Aug 2011 10:29:32 +0000

Jan starts its presentation by talking about its Distro Availability Matrix of Netfilter tech page. It contains the software and their versions in a lot of distributions.

Next subject is the discussion about maintaining translations of iptables man page. The team is international and could translate in a few language the man pages. But the question is about finding volunteers in the long term. Jan is alright with taking in charge the synchronization of translation. Any volunteers for translation is welcome.

Florian Westphal: Moving rp_filter into netfilter

Tue, 23 Aug 2011 09:17:32 +0000

Reverse Path filtering is currently only implemented in IPv4. Eric Leblond sends a patch to add support for IPv6 but it was refused by David Miller who, among other points, wanted to get rid of rp_filter and would like to see it in the Netfilter code.

Reverse patch filter implementation is a single function called fib_validate_source. Looking at the problem, it seem relatively simple to implement because, it is just to reverse source and destination and then get the output interface. if it match with the incoming interface, then this is ok.

Eric Leblond: In need of reverse path filtering

Tue, 23 Aug 2011 08:53:14 +0000

I just gave a presentation to explain that it is necessary to implement carefully reverse path filtering in IPv4 and IPv6.

More to come later.

Patrick McHardy: memory mapped netlink and nfnetlink_queue

Mon, 22 Aug 2011 13:56:34 +0000

Patrick McHardy presents his work on a modification of netlink and nfnetlink_queue which is using memory map.

One of the problem of netlink is that netlink uses regular socket I/O and data need to be copied to the socket buffer data areas before being send. This is a problem for performance.

The basic concept or memory mapped netlink is to used a shared memory area which can be used by kernel and userspace. A ring buffer is set and instead of copying the data, we just move a pointer to the correct memory area and the userspace reads
It is necessary to synchronize kernel and user spaces to avoid a read on a non significative area. This is done by using a area ownership.

Jesper Dangaard Brouer: IPTV-analyzer

Mon, 22 Aug 2011 13:17:14 +0000

Jesper presents its IP TV analyser know called IPTV-analyser.

He starts the project when encountering problem in the IP TV system in the company he works for. Proprietary analyser exists but they are expensive and the tested equipment were not able to show the burstiness directly. To fix this, he started using wireshark and add it a burstiness detector. It was not enough because pcap was not scaling enough and they decide to build their own probe. One of the decisive point was the 192000â‚¬ necessary to buy the necessary probes.

Holger Eitzenberger: experiences from making Network Stack Multicore

Mon, 22 Aug 2011 10:51:38 +0000

Holger want to describe its experience when switching from monocore system to mutiticore system at ~~Astaro~~ Sophos.
They used:

RPS: Receive packet steering
RFS:Receive flow steering
XPS: Transmit flow steering

They are using a 2.6.32 kernel and they had to backport the code but this was quite easy because the code is self-contained. irqbalance is not RPS and XPS aware and it is know to degrade performance. Holger decide then to start a new project.

Sanket Shah: An alternate way to use IPSet framework for increasing firewall throughput

Mon, 22 Aug 2011 10:25:14 +0000

When doing matching on iptables, the sequential test of the rules is costly. By using ipset this is possible to limit the number of matches by using the sets.

For their use, they decide to use the connection mark to determine the fate of the packet. It is used to jumb on the correct chain. This logic, combined with a connectionmark set they have developed this lead to a filtering system with a really limited number of rules. In fact, this was switching from something like 10000 rules to one single rule. Ipset is doing all the classification work. The performance increase is huge as on the test system, it goes from a bandwith of 256Mb with iptables to a bandwith 1.8Gb with their system.

JÃ³zsef Kadlecsik: ipset status

Mon, 22 Aug 2011 09:38:11 +0000

Ipset is now included in the kernel and that’s the main event of ipset in the previous year. JÃ³zsef recommands to use the 6.8 version which is included in kernel 3.1. If your kernel is older, using a separately compiler ipset is recommanded.

If we omit the bugfixes, a lot of of new features have been introduced sinced version 6.0. It is possible to list the sets defined on a system without getting everything which is useful when big set have been defined.

Eric Leblond: degree of freedom offered by connection tracking helpers

Mon, 22 Aug 2011 09:37:22 +0000

I gave a small presentation about a study I’ve made on connection tracking helpers. The slides are here: nfws_helper_freedom

Discussion following the speech was interesting. The main subject was automatic testing of the connection tracking helpers (as well as testing the other components). Pablo Neira Ayuso came with the idea of injecting the packet inside the kernel via a mechanism similar to NFQUEUE. This would then be easy to replay traffic. An extended discussion about the subject should take place during the week.

Samir Bellabes: userspace security for network syscalls – snet

Mon, 22 Aug 2011 08:35:50 +0000

Snet is an LSM module which treat network access. It is composed of a kernel part, a library and a tool.

In the kernel, event are generated for protocol and syscall, for example tcp and listen. It is then possible through a ticket system to decide if a process has the right to the event. For example, you can tell firefox can open connections to outside. A netlink protocol is used to communicate with userspace. Thus this is possible in userspace to take the decision by issuing ticket and sending it to kernel.

Opening of 8th Netfilter Workshop

Mon, 22 Aug 2011 08:10:50 +0000

The 8th Netfilter Workshop has been opened by Patrick McHardy in Freiburg. It is hosted by the Freiburg University.

The schedule is available on the workshop wiki.

NFWS on To Linux and beyond !

Jan Engelhardt, â€œMerge Meâ€

Xtables2

NFWS group photo

Tomasz Bursztyka, connMan usage of Netfilter

Introduction

Discussion

Jozsef Kadlecsik, ipset status

Tc interaction

Packet and byte counters

Pablo Neira Ayuso, nftables strikes back

Introduction

Architecture

Simon Horman, MPLS Enlightened Open vSwitch

Victor Julien, Suricata and Netfilter

Pablo Neira Ayuso, Netfilter summary of changes since last workshop

Martin Topholm: DDoS experiences with Linux and Netfilter

David Miller: routing cache is dead, now what ?

Fabio Massimo Di Nitto: Kronosnet.org

Eric Leblond: ulogd2, Netfilter logging reloaded

Introduction

Screencasts

Jan Engelhardt, Xtables2: Packet Filter Evolved

Introduction

Capabilities

Daniel Borkmann: Packets Sockets, BPF and Netsniff-NG

PF_PACKET introduction

Netsniff-NG

Tomasz Bursztyka, ConnMan usage of Netfilter: a close overview

Introduction

Switching to nftables

Julien Vehent, AFW: Automating host-based firewalls with Chef

The problem

From Service Oriented Architecture to Service Oriented Security

Jozsef Kadlecsik, Faster firewalling with ipset

Why ipset ?

History

Patrick McHardy: Oops, I did it: IPv6 NAT

Introduction

Reasons for adding IPv6 NAT

Pablo Neira Ayuso: nftables, a new packet filtering framework for Netfilter

Introduction

nftables vs iptables

Patrick McHardy: memory mapped netlink tree is available for testing

Eric Leblond: Introduction to coccinelle

Jesper Dangaard Brouer: CPAN module IPTables::libiptc

Patrick McHardy: getting rid of the second tuple

Ulrich Weber: IPV6 NAT

Pablo Neira Ayuso: nfgrep: traffic classification for Netfilter/iptables

Nishit Shah & Jimit Mahadevia: TCP Session Load-balancing in Active-Active HA Cluster

Holger Eitzenberger: speeding up selective conntrack flush

Jesper Dangaard Brouer: the missing conntrack garbage collector

Jan Engelhardt: Free form discussion

Florian Westphal: Moving rp_filter into netfilter

Eric Leblond: In need of reverse path filtering

Patrick McHardy: memory mapped netlink and nfnetlink_queue

Jesper Dangaard Brouer: IPTV-analyzer

Holger Eitzenberger: experiences from making Network Stack Multicore

Sanket Shah: An alternate way to use IPSet framework for increasing firewall throughput

JÃ³zsef Kadlecsik: ipset status

Eric Leblond: degree of freedom offered by connection tracking helpers

Samir Bellabes: userspace security for network syscalls – snet

Opening of 8th Netfilter Workshop

Jan Engelhardt, â€œMerge Meâ€