Links Load balancing

25 thoughts on “Links Load balancing”

1. Thank you very much for a wonderful tutorial 🙂 Helped me a lot. Please note that ‘-t mange’ option is missing in some commands because of which they’ll fail.

   And the ip route lines should be

   ip route add default via GW_LINK1 table LINK1
   ip route add default via GW_LINK2 table LINK2

   Thanks again 🙂

2. Thanks a lot, I’ve fixed the ip command in the text.

3. Hi! Did you hand patched netfilter to add condition extension? I didnt found it available for iptables 1.4.10.

   Thanks!

4. You’re right. Condition is now available via Xtables-addons.

5. Hi there – I believe this could also work with more than two links right? Also, would this support persistence over the link where the connection is initiated?

   Thanks a lot!

6. Yes, there is no limitation on the number of links, simply change the counter size and the distribution of links. And don’t forget to sync this with ‘ip rule’.
   The persistence is achieved thanks to CONNMARK which transfer a per-connection mark to the mark of the packet. This guarantee that the mark on a packet and hence is link is set for all packets of a connection. This part is one of the most tricky and all is done in these lines:

   iptables -A PREROUTING -t mangle -j CONNMARK --restore-mark
iptables -A PREROUTING -t mangle -m mark --mark 0x0 -m nth --counter 1 \
--every 4 --packet 1 -j MARK --set-mark 1
iptables -A POSTROUTING -j CONNMARK --save-mark


   If we have a packet belonging to a new connection, the first rule will not restore a mark which has never been set. Thus the packet is mark 0 and there is a match in one of the counter rules. This match set a mark on the packet which correspond to a link. This mark is saved by the last rule (--save-mark) on the connection mark. Thus, when a packet of an existing connection comes, the restore-mark rules copy the connection mark to the packet mark and the counter rules do not match anymore.

7. Hi!
   Do You have the same thing but for recent kernels?
   I can’t find it out how to convert all this solution for new versions of iptables. There are no -j MARKING and –counter and so on… ;(

8. P.s. How iptables knows where is eth0 and where is eth1?
   May be You have totally working example from the beginning till end?
   Sorry for noob questions 🙁

9. Hi,

   On recent kernel, you’ve got to use the statistic module for counter and not nth module. The syntax is described in the page.

   MARKING is a custom chain created by the -N command. This is not a old lost extensions 😉

10. Iptables does not care about eth0 and eth1. It puts a mark and that’s the routing who knows about interface.

11. Pingback: Dynamically re-route specific traffic on NIC failure in Linux - Admins Goodies

12. Hi everybody,

    Would you please let me know what should be my configuration for my centOS system on which i have two external (public) IPs and want to load-balance with failover.
    Actually i am confused whether i should write LINK1/LINK2 in the above config or replace them with WANIP1/WANIP2 or names of the interfaces (eth0/eth1)?????

    Kindly help!!!

13. I face the following error on the second line:
    iptables v1.4.7: Couldn’t load match `nth’:/lib64/xtables/libipt_nth.so: cannot open shared object file: No such file or directory

    what is the problem?

14. Masood: the nth module is not available anymore, use statistic instead as described in the page.

15. Thanks Regit. But, there is another problem with the last line:

    iptables -A POSTROUTING -j CONNMARK –save-mark
    iptables: No chain/target/match by that name.

16. OK, ‘-t mangle’ is missing on the line. I’ve updated the page.

17. Hi guys, I want try to load balance and fail over my squid, I have prepare d this iptables conf, can i have your opinion?

    :FORWARD ACCEPT [0:0]
    :OUTPUT ACCEPT [0:0]
    -A INPUT -m state –state ESTABLISHED,RELATED -j ACCEPT
    -A INPUT -p icmp -j ACCEPT
    -A INPUT -i lo -j ACCEPT
    -A INPUT -m state –state NEW -m tcp -p tcp –dport 22 -j ACCEPT
    -A INPUT -m state –state NEW -m udp -p udp –dport 161 -j ACCEPT
    COMMIT
    *mangle
    :FORWARD ACCEPT [0:0]
    :INPUT ACCEPT [0:0]
    :OUTPUT ACCEPT [0:0]
    :PREROUTING ACCEPT [0:0]
    :POSTROUTING ACCEPT [0:0]

    #GUEST
    -A PREROUTING -i eth3 -j MARK –set-mark 1
    #LAN
    -A PREROUTING -j CONNMARK –restore-mark
    -A PREROUTING -m mark –mark 0x0 -J MARKING
    -A PREROUTING -i eth1 -p tcp –dport 80 -m condition –condition link1_up \ –mode nth –every 4 –packet 0 -j MARK –set-mark 2
    -A PREROUTING -i eth1 -p tcp –dport 80 -m condition –condition link2_up \ –mode nth –every 4 –packet 1 -j MARK –set-mark 3
    -A PREROUTING -i eth1 -p tcp –dport 80 -m condition –condition link1_up \ –mode nth –every 4 –packet 2 -j MARK –set-mark 2
    -A PREROUTING -i eth1 -p tcp –dport 80 -m condition –condition link2_up \ –mode nth –every 4 –packet 3 -j MARK –set-mark 3
    #WIRELESS
    -A PREROUTING -i eth2 -p tcp –dport 80 -m mark –mark 0x0 -m statistic \ –mode nth –every 4 –packet 0 -j MARK –set-mark 2
    -A PREROUTING -i eth2 -p tcp –dport 80 -m mark –mark 0x0 -m statistic \ –mode nth –every 4 –packet 1 -j MARK –set-mark 3
    -A PREROUTING -i eth2 -p tcp –dport 80 -m mark –mark 0x0 -m statistic \ –mode nth –every 4 –packet 2 -j MARK –set-mark 2
    -A PREROUTING -i eth2 -p tcp –dport 80 -m mark –mark 0x0 -m statistic \ –mode nth –every 4 –packet 3 -j MARK –set-mark 3

    -A PREROUTING -A POSTROUTNG -t mangle -j CONNMARK –save-mark
    -A PREROUTING -m mark –mark 2 -j ACCEPT
    -A PREROUTING -m mark –mark 3 -j ACCEPT
    COMMIT
    *nat
    :PREROUTING ACCEPT [0:0]
    :OUTPUT ACCEPT [0:0]
    :POSTROUTING ACCEPT [0:0]
    COMMIT

    The squid are in different lan to the cliet 🙂

18. Hi Regit,

    Actually I taken a different approach for load balance which is mentioned in URL http://lartc.org/howto/lartc.rpdb.multiple-links.html.

    Can you provide a comparison between your approach and the one mentioned in URL? What are pros and cons?

    Thanks.

19. Hello lingeek.

    I’ve never tested lartc version. It seems easy to use and this is a good point for this solution. One advantage of what is explained on my page is that you are able to tune the load balancing with respect to the link bandwidth.

20. Hi,
    actually there is one problem with lartc version.
    In case of a Linux router with two ISPs (eth1 and eth2 in lartc load balance) with one local (eth0), if I am having a tcp connection from local machine to server on internet and say it is connected through eth1. So my local ip is translated to public ip of eth1 (masquerading) and connection is made with server.

    Now at every minute my application is sending request to sever to share some date (obviously through eth1). But as soon as kernel calls the gc which flushes the route cache, the traffic goes through eth2.

    As both ISPs are in load balance and when route cache flused, it is possible that next time traffic will go through different ISP, but the problem is that the traffic is still NATed/masqueraded by eth1 public ip and routed through eth2.

    Some how the NATing/masquerading functionality is not working hand in hand with route cache flush for already established connections.

    A small clue to resolve this will be a great help.

    Thanks.

21. Hi Lingeek,

    Just use these rules for nat :

    iptables -t nat -I POSTROUTING -o interface_link1 -j MASQUERADE
    iptables -t nat -I POSTROUTING -o interface link2 -j MASQUERADE

    Use SNAT only if you have severals IP on each interface or if you don’t specify the interface as in the example above. Here you don’t need to nat packet according to the mark because of theses rules :
    ip rule add fwmark 1 lookup table LINK1
    ip rule add fwmark 2 lookup table LINK2

    I have one question.

    the condition’s module just check the value in one file :

    To allow the rule to match:
    echo 1 > /proc/net/nf_condition/link1_up

    To disable the rule:
    echo 0 > /proc/net/nf_condition/link1_up

    How do you manage to write the right value in the file if the interface is up or down ?

    Do you use a script to do this ?
    Thank you.

22. It’s great that you are getting ideas from this piece of writing as well as
    from our argument made at this time.

23. Hey there I am so grateful I found your blog, I really found you by error, while I was
    browsing on Aol for something else, Regardless I am here
    now and would just like to say cheers for a fantastic post and
    a all round thrilling blog (I also love the theme/design), I don’t have time to browse it all at the moment but I have book-marked it and also added your RSS feeds, so
    when I have time I will be back to read more, Please
    do keep up the great b.

24. please cmiiw in fail over first line should be
    iptables -t mangle -N MARKING

25. When I originally left a comment I appear to have clicked on the -Notify me when new comments are added- checkbox and from now on each time a comment is added I recieve four emails with the same comment.
    There has to be a way you are able to remove me from that service?
    Thank you!