Network on To Linux and beyond !

Speeding up scapy packets sending

Thu, 17 Apr 2014 14:11:58 +0000

Sending packets with scapy

I’m currently doing some code based on scapy. This code reads data from a possibly huge file and send a packet for each line in the file using the contained information.
So the code contains a simple loop and uses sendp because the frame must be sent at layer 2.

def run(self):
         filedesc = open(self.filename, 'r')
         # loop on read line
         for line in filedesc:
             # Build and send packet
             sendp(pkt, iface = self.iface, verbose = verbose)
             # Inter packet treatment

Doing that the performance are a bit deceptive. For 18 packets, we’ve got:

A bit of fun with IPv6 setup

Thu, 26 Sep 2013 09:28:38 +0000

When doing some tests on Suricata, I needed to setup a small IPv6 network. The setup is simple with one laptop which is Ethernet connected to a desktop. And the desktop host a Virtualbox system.
This way, the desktop can act as a router with laptop on eth0 and Vbox on vboxnet0.

To setup the desktop/router, I’ve used:

ip a a 4::1/64 dev eth0
ip a a 2::1/64 dev vboxnet0
echo "1">/proc/sys/net/ipv6/conf/all/forwarding

To setup the laptop who already has a IPv6 public address on eth0, I’ve done:

Talk about nftables at Kernel Recipes 2013

Tue, 24 Sep 2013 13:48:19 +0000

I’ve just gave a talk about nftables, the iptables successor, at Kernel Recipes 2013. You can find the slides here:
2013_kernel_recipes_nftables

A description of the talk as well as slides and video are available on Kernel Recipes website

Here’s the video of my talk:

I’ve presented a video of nftables source code evolution:

The video has been generated with gource. Git history of various components have been merged and the file path has been prefixed with project name.

Using tc with IPv6 and IPv4

Wed, 18 Sep 2013 12:57:21 +0000

The first news is that it works! It is possible to use tc to setup QoS on IPv6 but the filter have to be updated.

When working on adding IPv6 support to lagfactory, I found out by reading tc sources and specifically ll_proto.c that the keyword to use for IPv6 was ipv6. Please read that file if you need to find the keyword for an other protocol.
So to send packet with Netfilter mark 5000 to a specific queue, one can use:

Netfilter and the NAT of ICMP error messages

Wed, 24 Apr 2013 22:30:00 +0000

The problem

I’ve been recently working for a customer which needed consultancy because of some unexplained Netfilter behaviors related to ICMP error messages. He authorizes me to share the result of my study and I thank him for making this blog entry possible.
His problem was that one of his firewalls is using a private interconnexion with their border router and the customer did not manage to NAT all outgoing ICMP error messages.

Tomasz Bursztyka, connMan usage of Netfilter

Tue, 12 Mar 2013 12:52:16 +0000

Introduction

connMan is a network manager which has support for a lot of different layers from ethernet and WiFi to NFC and link sharing.

It features automatic link switch and allow you to select your preferred type of support. The communication with UI is event based so it is easy to do as only a few windows type are needed.

Discussion

David Miller pointed out the fact that DHCP client is really often putting the interface in promiscuous mode and this is not a good idea as it is like having a tcpdump started on every laptop. As connMann does ahave its own implementation, they could maybe take this into account and improved the situation. This is in fact already the case as the DHCP client is using an alternate method.

Simon Horman, MPLS Enlightened Open vSwitch

Mon, 11 Mar 2013 15:58:36 +0000

Open vSwitch is a multi-layer switch. It is designed to enable network automation through programmatic extension, while still supporting standard management interfaces and protocols.

Openflow is a management protocol that is supported by Open vSwitch. Openflow is has a basic support for MPLS. It features a minimum operation set to enable to configure MPLS correclty.
Openflow MPLS support is partially implemented in Open vSwitch but there is some difficulties.

SOme of the operations feature update of L3+ parameter like TTL. They must be updated in same manner in the MPLS header and in the packet header. And this is quite complicated as it supposed to decode the packet below MPLS. But MPLS header does not include the encapsulated ethernet type so it is almost impossible to access correctly to the packet structure.

David Miller: routing cache is dead, now what ?

Mon, 11 Mar 2013 10:17:21 +0000

The routing cache was maintaining a list of routing decisions. This was an hash table which was highly dynamic and was changing due to traffic. One of the major problem was the garbage collector. An other severe issue was the possibility of DoS using the increase

The routing cache has been suppressed in Linux 3.6 after a 2 years effort by David and the other Linux kernel developers. The global cache has been suppressed and some stored information have been moved to more separate resources like socket.

Fabio Massimo Di Nitto: Kronosnet.org

Mon, 11 Mar 2013 09:03:03 +0000

Kronosnet is a “I conceived it when drunk but it works well” VPN implementation. It is using an Ether TAP for the VPN to provide a lyaer 2 vpn. To avoid reinventing the wheel, it is delegating most of the work to the kernel. It supports multilink and redundancy of servers. On multilink side, 8 links can be done per-host to help redundancy.

One of the use of this project is the creation of private network in the cloud as it can be easily setup to provide redundancy and connection for a lot of clients (64k simultaneous clients). And because a layer 2 VPN is really useful for this type of usage.

Daniel Borkmann: Packets Sockets, BPF and Netsniff-NG

Sun, 10 Mar 2013 16:11:29 +0000

PF_PACKET introduction

This is access to raw packet inside Linux. It is used by libpcap and by other projects like Suricata.
PF_PACKET performance can be improved via dedicated features:

Zero-copy RX/TX
Socket clustering
Linux socket filtering (BPF)

BPF architecture looks like a small virtual machine with register and memory stores. It has different instructions and the kernel has its own kernel extensions to access to cpu number, vlan tag.

Netsniff-NG

Netsniff-ng is a set of minimal tools:

Ulogd 2.0.2, my first release as maintainer

Mon, 04 Mar 2013 00:13:13 +0000

Objectives of this release

So it is my first ulogd2 release as maintainer. I’ve been in charge of the project since 2012 October 30th and this was an opportunity for me to increase my developments on the project. Roadmap was almost empty so I’ve decided to work on issues that were bothering me as a user of the project. I’ve also included two features which are connection tracking event filtering and a Graphite output module. Ulogd is available on Netfilter web site

Visualize Netfilter accounting in Graphite

Sat, 22 Dec 2012 11:27:09 +0000

Ulogd Graphite output plugin

I’m committed a new output plugin for ulogd. The idea is to send NFACCT accounting data to a graphite server to be able to display the received data. Graphite is a web application which provide real-time visualization and storage of numeric time-series data.

Once data are sent to the graphite server, it is possible to use the web interface to setup different dashboard and graphs (including combination and mathematical operation):

Flow reconstruction and normalization in Suricata

Thu, 15 Nov 2012 17:36:06 +0000

The naive approach would consider that an IDS is just taking packet and doing a lot of matching on it. In fact, this is not at all what is happening. An IDS/IPS like Suricata is in fact rebuilding the data stream and in case of known protocols it is even normalizing the data stream and providing keyword which can be used to match on specific field of a protocol.

Let’s say, we a rule to match on a HTTP request where method is GET and the URL is “/download.php”.

Suricata, to 10Gbps and beyond

Mon, 30 Jul 2012 21:03:19 +0000

Introduction

Since the beginning of July 2012, OISF team is able to access to a server where one interface is receiving
some mirrored real European traffic. When reading “some”, think between 5Gbps and 9.5Gbps
constant traffic. With that traffic, this is around 1Mpps to 1.5M packet per seconds we have to study.

The box itself is a standard server with the following characteristics:

CPU: One Intel(R) Xeon(R) CPU E5-2680 0 @ 2.70GHz (16 cores counting Hyperthreading)
Memory: 32Go
capture NIC: Intel 82599EB 10-Gigabit SFI/SFP+

The objective is simple: be able to run Suricata on this box and treat the whole
traffic with a decent number of rules. With the constraint not to use any non
official system code (plain system and kernel if we omit a driver).

Flow accounting with Netfilter and ulogd2

Sat, 14 Jul 2012 21:11:44 +0000

Introduction

Starting with Linux kernel 3.3, there’s a new module called nfnetlink_acct.
This new feature added by Pablo Neira brings interesting accountig capabilities to Netfilter.
Pablo has made an extensive description of the feature in the commit.

System setup

We need to build a set of tools to get all that’s necessary:

libmnl
libnetfilter_acct
nfacct

The build is the same for all projects:

git clone git://git.netfilter.org/PROJECT
cd PROJECT
autoreconf -i
./configure
make
sudo make install

Opensvp, a new tool to analyse the security of firewalls using ALGs

Fri, 08 Jun 2012 13:34:07 +0000

Following my talk at SSTIC, I’ve released a new tool called opensvp. Its aim is to cover the attacks described in this talk. It has been published to be able to determine if the firewall policy related to Application Layer Gateways is correctly implemented.

Opensvp implements two type of attacks:

Abusive usage of protocol commands: an protocol message can be forged to open pinhole into firewall. Opensvp currently implements message sending for IRC and FTP ALGs.
Spoofing attack: if anti-spooofing is not correctly setup, an attacker can send command which result in arbitrary pinhole being opened to a server.

It has been developed in Python and uses scapy to implement the spoofing attack on ALGs.

Fri, 08 Jun 2012 10:26:13 +0000

Les transparents de ma prÃ©sentation du SSTIC sont disponibles : Utilisation malveillante des suivis de connexions. Merci aux organisateurs du SSTIC d’avoir acceptÃ© mon papier!

Des vidÃ©os de dÃ©monstration sont disponibles sur ce post: Playing with Network Layers to Bypass Firewalls’ Filtering Policy

L’outil de test openvsp est disponible sur cette page.

Using Scapy lfilter

Thu, 07 Jun 2012 13:13:45 +0000

Scapy BPF filtering is not working when some exotic interface are used. This includes Virtualbox interface such as vboxnet.

For example, the following code will not work if the interface is a virtualbox interface:

build_filter = "src host %s and src port 21"
sniff(iface=iface, prn=callback, filter=build_filter)

To fix this, you can use the lfilter option. The filtering is now done inside Scapy. This is powerful but less efficient.

The code can be modified like this:

IPv6 privacy extensions on Linux

Fri, 29 Apr 2011 17:41:24 +0000

IPv6 global address

The global address is used in IPv6 to communicate with the outside world. This is thus the one that is used as source for any communication and thus in a way identify you on Internet.

Below is a dump of an interface configuration:

eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

link/ether 00:22:15:64:42:bd brd ff:ff:ff:ff:ff:ff
inet6 2a01:f123:1234:5bd0:222:15ff:fe64:42bd/64 scope global dynamic 
   valid_lft 86314sec preferred_lft 86314sec
inet6 fe80::222:15ff:fe64:42bd/64 scope link 
   valid_lft forever preferred_lft forever</pre>

The global address is here 2a01:f123:1234:5bd0:222:15ff:fe64:42bd/64. It is build by using the prefix and adding an identifier build with the hardware address. For example, here the hardware address is 00:22:15:64:42:bd and the global IPv6 address is ending with 22:15_ff:fe_64:42bd.

Network on To Linux and beyond !

Speeding up scapy packets sending

Sending packets with scapy

A bit of fun with IPv6 setup

Talk about nftables at Kernel Recipes 2013

Using tc with IPv6 and IPv4

Netfilter and the NAT of ICMP error messages

The problem

Tomasz Bursztyka, connMan usage of Netfilter

Introduction

Discussion

Simon Horman, MPLS Enlightened Open vSwitch

David Miller: routing cache is dead, now what ?

Fabio Massimo Di Nitto: Kronosnet.org

Daniel Borkmann: Packets Sockets, BPF and Netsniff-NG

PF_PACKET introduction

Netsniff-NG

Ulogd 2.0.2, my first release as maintainer

Objectives of this release

Visualize Netfilter accounting in Graphite

Ulogd Graphite output plugin

Flow reconstruction and normalization in Suricata

Suricata, to 10Gbps and beyond

Introduction

Flow accounting with Netfilter and ulogd2

Introduction

System setup

Opensvp, a new tool to analyse the security of firewalls using ALGs

Transparents de ma prÃ©sentation au SSTIC

Using Scapy lfilter

IPv6 privacy extensions on Linux

IPv6 global address