A new Suricata IPS mode
Suricata IPS capabilities are not new. It is possible to use Suricata with Netfilter or ipfw to build a state-of-the-art IPS. On Linux, this system has not the best throughput performance. Patrick McHardy’s work on netlink: memory mapped I/O should bring some real improvement but this is not yet available.
I’ve thus decided to do an implementation of IPS based on AF_PACKET (read raw socket). The idea is based on one of the snort’s running mode. It peers two network interfaces and all packets received from one interface are sent to the other interface (if a signature with drop keyword does not fired on the packet). This requires to dedicate two network interfaces for Suricata but this provide a simple bridge system. As suricata is using latest AF_PACKET features (read load balancing), it was possible to build something really promising.
Victor Julien has commited my implementation which is now available in current git tree and will be part of Suricata 1.4beta1.
Configuration
You need to dedicate two network interfaces for this mode. The configuration is made via some new configuration variable available in the description of an AF_PACKET interface.
For example, the following configuration will create a Suricata acting as IPS between interface eth0 and eth1:
af-packet:
- interface: eth0
threads: 1
defrag: yes
cluster-type: cluster_flow
cluster-id: 98
copy-mode: ips
copy-iface: eth1
buffer-size: 64535
use-mmap: yes
- interface: eth1
threads: 1
cluster-id: 97
defrag: yes
cluster-type: cluster_flow
copy-mode: ips
copy-iface: eth0
buffer-size: 64535
use-mmap: yes
Basically, we’ve got an af-packet configuration with two interfaces. Interface eth0 will copy all received packets to eth1 because of the copy-* configuration variable:
copy-mode: ips
copy-iface: eth1
The configuration on eth1 is symetric:
copy-mode: ips
copy-iface: eth0
There is some important facts to consider when setting up this mode:
- The implementation of this mode is dependent of the zero copy mode of AF_PACKET. Thus you need to set use-mmap to yes on both interface.
- MTU on both interfaces have to be equal: the copy from one interface to the other is direct and packet bigger then the MTU will be dropped by kernel.
- Set different values of cluster-id on both interfaces to avoid conflict.
The copy-mode variable can take the following values:
- ips: the drop keyword is honored and matching packet are dropped.
- tap: no drop occurs, Suricata act as a bridge
Please note, that mode can be different on the peered interfaces.
Current limitation
One other important point is that the threads variable must be equal on both interface. This is due to the fact the peering is done at the socket level: each packet received on a socket is sent to a peered socket listening to the peered interface.
At last but not least, you need at least Linux kernel 3.6 (or 3.4.12) to be able to set a threads value higher than 1. There is a bug in prior kernel that causes an infinite loop (packet being sent from an interface and reread, …). If you can’t wait or can’t made a full upgrade of your kernel you can use my patch which will be part of Linux 3.6 and of Linux 3.4.12.
Running it
Running Suricata in AF_PACKET IPS mode is straight forward. Once the configuration has been updated, you can simply run:
suricata -c /etc/suricata/suricata.yaml --af-packet
Please note that is possible to have normal IDS interface running simultaneously. For example, eth3 could be added to the af-packet configuration and used a regular interface.
Some implementation details
You can find some additional technical information in the commit.
As mentioned below, the peering is done at the socket level. In Suricata, each capture thread is opening a socket and is binding it to the cluster-id of the interface.
Once a thread is started, it is peered with an thread listening to the copy-iface interface which socket will be used as a send interface.
This system permit to avoid locking issue in workers running mode as only one thread will write to the peered socket at a time because we have only one packet treated by a capture thread at a time.
We need to reuse the interface for a simple reason: if a new dedicated socket was to be used, the packet sent to this socket would be received by Suricata read socket instead of being ignored. Linux kernel implement the natural feature of avoiding to send back packet to the sending socket.
This behavior was implemented in Linux kernel for regular socket but this was not the case for clustered sockets. This is why at least Linux 3.6 or my patch is needed to be able to use a value of threads bigger than one.
Conclusion
This mode is new in Suricata and is is really promising. It has suffered limited testing in high bandwidth environment. So, as usual, feedback is more than welcome. Don’t hesitate to send me a mail or to ask question on Suricata Mailing lists.
HÃ,
suricatasc error:
~/suricata$ sudo suricatasc
Traceback (most recent call last):
File “/usr/local/bin/suricatasc”, line 30, in
socket.connect(SOCKET_PATH)
File “/usr/lib/python2.7/socket.py”, line 224, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 2] No such file or directory
Best Regards,
OK, did you activate the feature ? You have to put in your suricata.yaml something like:
Hi,
I tried it, but suricata won’ start, my config part:
af-packet:
– interface: eth1 # Number of receive threads (>1 will enable experimental flow pinned # runmode)
threads: 1
cluster-id: 99
cluster-type: cluster_flow
defrag: yes
use-mmap: yes
copy-mode: ips
copy-iface: eth2
When I run suricata (suricata c /etc/suricata/suricata-debian.yaml –af-packet)
I receive this error:
29/5/2013 – 12:39:17 – – Adding interface eth1 from config file
29/5/2013 — 12:39:17 – – Using 1 live device(s).
29/5/2013 — 12:39:17 – – Enabling mmaped capture on iface eth1
29/5/2013 — 12:39:17 – – AF_PACKET TAP mode activated eth1->eth2
29/5/2013 — 12:39:17 – – Using flow cluster mode for AF_PACKET (iface eth1)
29/5/2013 — 12:39:17 – – Using defrag kernel functionality for AF_PACKET (iface eth1)
29/5/2013 — 12:39:17 – – Enabling zero copy mode by using data release call
29/5/2013 — 12:39:17 – – [ERRCODE: SC_ERR_AFP_CREATE(190)] – Threads number not equals
29/5/2013 — 12:39:17 – – [ERRCODE: SC_ERR_RUNMODE(187)] – Some IPS capture threads did not peer.
A new entry appears in kern.log:
Loading kernel module for a network device with CAP_SYS_MODULE (deprecated). Use CAP_NET_ADMIN and alias netdev- instead.
versions:
suricata 1.4.1
kernel: 3.8
system: debian sid
What could be the problem?
Laszlo
Fix is explained in https://redmine.openinfosecfoundation.org/issues/818 😉
Does the AF_PACKET IPS mode in Suricata just support the bridge mode, does it ok in route mode
Hello Alex. No, this is a transparent mode where suricata only act as a transparent bridge.
Could you please post a valid IPtables configuration that works with this setup? while Suricata is reporting:
11/23/2013-13:42:20.196528 [wDrop] [**] [1:1:1] facebook is blocked [**] [Classification: Potential Corpo
rate Privacy Violation] [Priority: 1] {TCP} 192.168.X.X:XXXX -> XX.XX.XX.XX:80
Packets are not dropped, they still pass to the internal network.
My IPTables Config:
Chain INPUT (policy DROP)
target prot opt source destination
ACCEPT all — anywhere anywhere
ACCEPT all — anywhere anywhere state RELATED,ESTABLISHED
ACCEPT icmp — anywhere anywhere limit: avg 1/sec burst 1
LOG icmp — anywhere anywhere limit: avg 1/sec burst 1 LOG level warning prefix “PING-DROP:”
DROP icmp — anywhere anywhere
ACCEPT tcp — anywhere anywhere multiport dports ssh
ACCEPT tcp — anywhere anywhere tcp dpt:domain
ACCEPT udp — anywhere anywhere udp dpt:domain
ACCEPT tcp — anywhere anywhere tcp dpt:http
ACCEPT udp — 192.168.1.15 anywhere multiport dports radius:tdp-suite
ACCEPT gre — anywhere anywhere
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Thank you!
Hello Joseph. This AF_PACKET IPS mode is complete separated from firewall stack. It establishes a software bridge between two interfaces by copying packet from one interface to the other (and reverse).
So packet are not seen by the firewall stack.
Thanks, Regit, I’ll keep my iptables rules then… The problem is although I see the wDrop statements in fast.log based on website content blocking rules, I still can access the blocked content, while with NFQUEUE it worked like a charm. I’m only getting 45MBps throughtput with a dual-core intel atom 1.86GHz and 2GiB DDR3 1333Mhz in NFQUEUE mode, so I wanted to test AF_PACKET in order to improve performance.. Suggestions are welcome.
Same here:
12/11/2013-20:34:45.359743 [wDrop] [**] [1:2100498:7] GPL ATTACK_RESPONSE id check returned root [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 82.165.177.154:80 -> 192.168.1.33:56836
traffic passes.
Any Ideas?
Chris, did you activate inline streaming in the YAML ?
No I did not. Pass the dunce cap.
Statically assigned and working now. I assumed ‘auto’ meant it was active based upon the note in yaml: “# auto will use inline mode in IPS mode”
Was expecting Drop still…is there any reason why the fast log shows a “wDrop?”
Thanks for your help!
I also had “inline yes” streaming from the beginning and traffic still passes. “wDrop” logs show, despite not dropping.
I am also having the problem “wDrop†logs showing, despite not dropping. Inline is set to “yes” in suricata.yaml. If I create a “reject” rule instead of a “drop” rule, the reject seems to prevent the packet being passed, where as “drop” still passes.
Is this a bug?
btw, my IPTABLES are flushed and accepting ALL on the IPS machine
Hello Chris,
Do you have a bridge setup ? You don’t need a bridge in AF_PACKET IPS mode. Suricata will be in charge of transferring packets from one interface to the other.
Regit, thanks for the reply.
I did not setup a bridge manually or intentionally. “brctl show” does not list any bridges.
I have eth0 set to DHCP. I have eth1 setup to “Share to other computers”. I have a router setup to DHCP
So it goes INTERNET > cable modem > eth0 of IPS > eth1 of IPS> linksys router > LAN. The LAN can get to INTERNET even when IPS is not running, oddly. So maybe the packets not “dropping” is related to this. But again, packets will be “rejected” properly by the IPS when I switch a “drop rule” to be a “reject rule”.
I am thinking /etc/NetworkManager is messing things up, since I set “managed” to true. I am going to try to flush all and set the IPS route and gateways up manually, but I am not sure how to manually setup the gateways,routes, and DNS properly. I am doing more reading and trial and error with IP addressing.
Any thoughts, Regit?
Hi Regit, it is possible to use several instances of af-packet? that is: if I have several interfaces on my machine, it is posible to create in suricata.yaml: afpacket_instance1(eth0ð1), afpacket_instance2(eth0ð2), afpacket_instance3(eth0ð3) and so on… How do you see it?
Hello Mark. Yes this is possible. Just look at the suricata.yaml configuration file. It is already specified in the af-packet section.
Once you have wrote your config file, you can start ‘suricata –af-packet’ and it will liste on all interfaces.
Hi Regit, this mode look sound interesting. Did you has some test results when Suricata work as IPS in this mode?
Hello Huy, some users are using it and it seems to perform well. It is said has been faster than NFQ mode.
Thanks, i will try it ^_^
Chris, Joseph, and Regit,
Did you ever solve the issue with the wDrop logging to fast.log but not actually dropping the packets? I’m seeing the same thing and I do have inline:yes
Hi Regit,
I run suricata in af-packet mode by command suricata -D -c suricata.yaml -i br0.
With br0 is bridged of eth1ð2.
My drop rules is not performed.
Reject rules is not performed too .
This is rule:
reject ip any any -> 8.8.8.8 any (msg:”Block ping DNS google”; sid:10000002; rev:1;)
and
drop ip any any -> 8.8.8.8 any (msg:”Block ping DNS google”; sid:10000002; rev:1;)
Could u pls confirm with af-packet mode, Can I do drop/reject ?
Hi all,
I’m confused on the following statements about in IPS/inline mode with a host with 2 nics:
“AF_PACKET IPS mode is completely separated from firewall stack.It establishes a software bridge between two interfaces by copying packet from one interface to the other (and reverse).”
“Suricata will be in charge of transferring packets from one interface to the other.”
What are then the basic network interface setting requirements related to routing on the OS level ? ip forwarding ? static routing tables ? default gateway ?
I must be doing something wrong here, I have a VM with two nics attached. One is enp0s3 and the other enp0s8. I am running Suricata version 4.0.5.
I have the following in my yaml in the af-packet section.
af-packet:
– interface: enp0s3
threads: 1
defrag: yes
cluster-type: cluster_flow
cluster-id: 98
copy-mode: ips
copy-iface: enp0s8
buffer-size: 64535
use-mmap: yes
– interface: enp0s8
threads: 1
cluster-id: 97
defrag: yes
cluster-type: cluster_flow
copy-mode: ips
copy-iface: enp0s3
buffer-size: 64535
use-mmap: yes
I also have the following in the stream section
stream:
memcap: 32mb
checksum-validation: yes
inline: yes
reassembly:
memcap: 64mb
depth: 1mb
toserver-chunk-size: 2560
toclient-chunk-size: 2560
I have the two network interfaces setup with static IPs
$ ifconfig
enp0s3 Link encap:Ethernet HWaddr 08:00:27:25:ac:2b
inet addr:10.0.19.16 Bcast:10.0.19.255 Mask:255.255.255.0
inet6 addr: fe80::b78f:c69a:b566:cb37/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:4131515 errors:0 dropped:0 overruns:0 frame:0
TX packets:2690412 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:630894339 (630.8 MB) TX bytes:269331975 (269.3 MB)
enp0s8 Link encap:Ethernet HWaddr 08:00:27:41:5b:de
inet addr:10.0.19.17 Bcast:10.0.19.255 Mask:255.255.255.0
inet6 addr: fe80::95a8:d0e9:d138:b746/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2969442 errors:0 dropped:0 overruns:0 frame:0
TX packets:3797496 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:626044650 (626.0 MB) TX bytes:272136409 (272.1 MB)
My routing table only has routes for the enp0s3 nic
$ route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default 10.0.19.1 0.0.0.0 UG 0 0 0 enp0s3
10.0.19.1 * 255.255.255.255 UH 0 0 0 enp0s3
So In my head, I’m thinking that I’d like to have anything routed from enp0s8 go be copied by suricata to enp0s3 only if it doesn’t have a drop rule. However, this doesn’t seem to be possible to create a rule to route from enp0s8 to enp0s3 in that way. I just get the error “Network is unreachable”.
I’m using the command
route add default gw 10.0.19.1 enp0s8
Anyone know what I’m doing wrong?
Hi,
I am running suricata af_packet in IPS mode on marvell board, packets are not copying from one interface to another. When i stop suricata, I can see No. of packets count received on one interface connected to WAN, but packets are not copying on other interface given in af_packet setting.
Hello. https://jayrims.co.uk/