Netfilter Connmark

 

Introduction

CONNMARK is a cool feature of Netfilter. It provides a way to have a mark which is linked to the a connection tracking entry. Once a connmark is set, it also apply for RELATED connection entry. So, if you add a connmark to an FTP connection, the same connmark will be put of connections from ftp-data. All Linux tools (for QoS or routing) are only able to use a mark put on packet. Thus, to be really useful, CONNMARK has the capability to transfer the connection mark to the packet mark (and reverse). This can be used to established connection persistent decision for QoS or routing. More information is available on the netfilter site. It is available in all Linux kernel since 2.6.12.

Using the feature

The most common CONNMARK setup consist in putting connection mark on packet when they arrive and saving packet mark to connection when they leave. In term of iptables, this translates as:
iptables -A PREROUTING -t mangle -j CONNMARK --restore-mark
iptables -A POSTROUTING -t mangle -j CONNMARK --save-mark

Code examples

A simple example

First here’s the code :
iptables -A POSTROUTING -t mangle -j CONNMARK --restore-mark
iptables -A POSTROUTING -t mangle -m mark ! --mark 0 -j ACCEPT
iptables -A POSTROUTING -p tcp --dport 21 -t mangle -j MARK --set-mark 1
iptables -A POSTROUTING -p tcp --dport 80 -t mangle -j MARK --set-mark 2
iptables -A POSTROUTING -t mangle -j CONNMARK --save-mark
The process is the following :
  1. We restore the mark following the CONNMARK mark. In fact, it does a simple MARK=CONNMARK where MARK is the standard mark (usable by tc)
  2. If we’ve got a mark no need to get further[1].
  3. We mark tcp packets with destination port 21 with MARK 1.
  4. We mark tcp packet with destination port 80.
  5. We set the mark of the initial packet as value of the conntrack mark for all the packets of the connection. This mark will be restore for the other packets by the first rule of POSTROUTING (–restore-mark).

A complete example

If you want to do a garbage (or use imbricated filters), you can do so by sorting rule from the more specific to the less specific :
iptables -A POSTROUTING -t mangle -j CONNMARK --restore-mark
iptables -A POSTROUTING -t mangle -m mark ! --mark 0 -j ACCEPT
iptables -A POSTROUTING -m mark --mark 0 -p tcp --dport 21 -t mangle -j MARK --set-mark 1
iptables -A POSTROUTING -m mark --mark 0 -p tcp --dport 80 -t mangle -j MARK --set-mark 2
iptables -A POSTROUTING -m mark --mark 0 -t mangle -p tcp -j MARK --set-mark 3
iptables -A POSTROUTING -t mangle -j CONNMARK --save-mark
We use -m mark –mark 0 to prevent an already put mark to be overwritten by a more generic filter. The second line is optional but it’s a shortcut because all packets relative to an established connection are only tested once.

Related Tips

Connmark and ingress

If you use an ingress policy you can take advantage of CONNMARK by restoring the mark in the PREROUTING chain. So you can use :
iptables -A PREROUTING -t mangle -j CONNMARK --restore-mark

A modified ip_conntrack file

After loading of the connmark module, the file /proc/net/ip_conntrack contains the indication of the connmark :
tcp      6 432000 ESTABLISHED src=62.212.98.117 dst=XXX.XXX.XXX.XXX sport=35027 dport=22 \\
src=217.15.82.6 dst=62.212.98.117 sport=22 dport=35027 [ASSURED] use=1 mark=0
udp      17 24 src=127.0.0.1 dst=127.0.0.1 sport=1024 dport=53 [UNREPLIED] src=127.0.0.1 \\
dst=127.0.0.1 sport=53 dport=1024 use=1 mark=0

Accounting with ip_conntrack

It is easy to replace iptables -A POSTROUTING -t mangle -m mark ! –mark 0 -j ACCEPT by
iptables -A POSTROUTING -t mangle -m mark  --mark 1 -j ACCEPT
iptables -A POSTROUTING -t mangle -m mark  --mark 2 -j ACCEPT
...
With that, a simple iptables -L POSTROUTING -t mangle -v gives you a perfect counter for each flow. And as mentioned before it does also account the traffic of RELATED connections in the category of the parent connection. There is still a problem since we loose the first packet of each connections. Thus it is necessary to add the counters of the –set-mark and of the related ACCEPT lines.
 Posted by at 14:47

  18 Responses to “Netfilter Connmark”

  1. Thanks so much for the information.

  2. Yes, thanks a lot. Very useful.

  3. Hi!

    Thanks for the explanation! Is this extension available for the latest netfilter/iptables ?

    Thanks!

  4. […] a suspect packet and decide to change the way it is handled by the network stack. Thanks to the CONNMARK target, you can modify the way all packets of the connection are […]

  5. Is important to note that ingress policy occurs way before mangle PREROUTING so the mark will never be available in the policier to match against it.

  6. Thanks for the information.

    It seems the behaviour has changed in 2.6 kernel (yes this page is old). I will try to have a look to check this and see what has happened.

  7. Hi Luciano,

    Did you meant “ingress policy occurs way before mangle *POSTROUTING*” ?.

  8. thanks,it’s useful

  9. Hi,
    I am marking packets in PREROUTING chain of mangle table. Is there any log method to see marked packets?
    Actually in my case, packets are fragmented, and want to see that all freagmented packets are mark correctly or not.

  10. Hello,

    By putting a LOG rule just below your marking rules, you should have message reaching syslog.

  11. Hi,
    I am getting packet logs but not information of marking on it. Let me tell you my issue in detail.

    I have 4 machine setup, M1 -> M2 -> M3 | M4. And A laptop that can be reachable through both M3 and M4.

    M2 has 2 NIC conected to M3 and M4. Now I want to divide the flow coming from M1 for laptop.
    At M2, I have created a samle rule,
    ==============================================================
    ip6tables -t mangle -A PREROUTING -p udp –dport 80 -j MARK –set-mark 12
    echo 2 udp_flow >> /etc/iproute2/rt_tables
    ip -6 rule add fwmark 12 table udp_flow
    ip -6 route add $laptop-ip dev eth2 table udp_flow
    ==============================================================

    Now if i send UDP traffic from M1 destined to Laptop IP, then if packet size is less then 1410 bytes then I can see complete UDP packets at M4 and successfully delivered to Laptop.

    But if packet size > 1410 bytes, then fragmentaion happens and I can see only 1st fragmented packet at M4, not successive fragmented packets. I searched and find out that only 1st fragmented packet have information of UDP and destination port and hence marked properly and routed. Other fragmented packets have only UDP information not destination port, so not marked and dropped at M2.

    Is there any solution or rule that can be applied so that all packets (whether fragmentation is there or not) can transfer to M4 only not M3 (destination port rule is compulsory) ?

    If i put only UDP rule not including destination port then there is no issue with both cases with/without fragmentation.

  12. A complete example states ‘POSTROUTING’, this should say ‘PREROUTING’ as POSTROUTING is too late

  13. You really make it appear really easy along with your presentation however I in finding this matter to be actually one thing which I feel
    I might never understand. It kind of feels too complex and very huge for me.
    I’m having a look forward in your subsequent submit, I’ll attempt to get the cling of it!

  14. […] OK now I use iptables to evenly distribute packets to each route tables. More info on how this work can be found here (http://www.diegolima.org/wordpress/?p=36) and here (http://home.regit.org/?page%5Fid=7) […]

  15. […] OK now I use iptables to evenly distribute packets to each route tables. More info on how this work can be found here (http://www.diegolima.org/wordpress/?p=36) and here (http://home.regit.org/?page%5Fid=7) […]

  16. […] OK now I use iptables to evenly distribute packets to each route tables. More info on how this work can be found here (http://www.diegolima.org/wordpress/?p=36) and here (http://home.regit.org/?page%5Fid=7) […]

  17. Hi,
    I’m using your pynetfilter_conntrack and I see the greate example in .\conntrack.py, it’s about list all conntrack events.
    In case I want to filter the conntrack events (for example: all tcp events) I find it so difficult.
    Could you please show me how I can do to filter the conntrack events?

 Leave a Reply

(required)

(required)

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>