- Posts: 145
- Thank you received: 0
Load balancing in Linux
18 years 5 months ago #15032
by Ranger24
Patience - the last reserve of the any engineer
Load balancing in Linux was created by Ranger24
Hi,
Imagine the following set up:
2 DSL lines feeding via ethernet ( eth0 & eth1) into a single linux box then a 3rd ethernet port (eth2) connected to small LAN.
Does anyone know how to load balance the upstream traffic over eth0 & eth1?
Will additional software be required? Or should it be possible via standard distros such as Knoppix?
Thanks,
Pete
And please don't ask why 2 DSL lines... Office Politics!
Imagine the following set up:
2 DSL lines feeding via ethernet ( eth0 & eth1) into a single linux box then a 3rd ethernet port (eth2) connected to small LAN.
Does anyone know how to load balance the upstream traffic over eth0 & eth1?
Will additional software be required? Or should it be possible via standard distros such as Knoppix?
Thanks,
Pete
And please don't ask why 2 DSL lines... Office Politics!
Patience - the last reserve of the any engineer
18 years 5 months ago #15047
by nske
Replied by nske on topic Re: Load balancing in Linux
You can do this through IPTables. There is a module (available through the patch-o-matic patch,
www.netfilter.org
) that allows you to match every N-th packet. So you could define every 2nd packet that initiates a connection, to forward through eth1 and the rest through eth0.
You would need the kernel to support the netfilter infrastructure, "Advanced Routing" and "Route by firewall mark" systems. I'm not sure whether the default kernels of most distributions provide these, most likely they do.
Here's a short guide through the procedure (familiarity with iptables is still needed though).
We use the mangle table to mark the 1st packet of every connection via MARK target, save the mark for the whole session (CONMARK target) and mark the rest of the packets of the session with the same mark, so that the packets of the same connection go through the same interface.
1) Initially we create the chain "marking", that will be used for marking packets:
[code:1]iptables -t mangle -N marking[/code:1]
2) We restore the connection marks (if they exist), in both PREROUTING, and OUTPUT, so that connections will be cycled in a round-robin fashion:
[code:1]iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
iptables -t mangle -A OUTPUT -j CONNMARK --restore-mark[/code:1]
3) We mark the unmarked packets, like said, only if they are initiating a connection:[code:1]iptables -t mangle -A PREROUTING -m mark --mark 0x0 -j marking
iptables -t mangle -A OUTPUT -m mark --mark 0x0 -j marking[/code:1]
[code:1]iptables -t mangle -A marking -m mark --mark 0x0 -m nth --every 2 --packet 0 -j MARK --set-mark 0x1
iptables -t mangle -A marking -m mark --mark 0x0 -m nth --every 2 --packet 1 -j MARK --set-mark 0x2
iptables -t mangle -A marking -j CONNMARK --save-mark[/code:1]
Now all the packets of half the connections will be marked as 0x1 and all the packets of the other half of the connections, as 0x2.
4) We create two routes, one for each interface. We also make sure connections to the internal network (192.168.0.0/16 in the example) do not leave through any of these interfaces.
If the public addresses are ON the box, let's say through ethernet-bridge to the dsl routers, or if you are using PCI/USB adsl interfaces and dial to the ISP through the linux box and ppp (in these cases the interfaces would be named by default ppp0 and ppp1), you would add:
[code:1]ip route add default dev ppp0 table 10
ip route add throw 192.168.0.0/16 table 10
ip route add default dev ppp1 table 20
ip route add throw 192.168.0.0/16 table 20[/code:1]
If on the other hand you are using a seperate router for each dsl connection, you would add routes to the addresses of the internal interfaces of these routers:
[code:1]ip route add default via ip-router1-here table 10
ip route add throw 192.168.0.0/16 table 10
ip route add default via ip-router2-here table 20
ip route add throw 192.168.0.0/16 table 20[/code:1]
5) We define routing according to the fwmark. Packets with mark 0x1 will leave through the 1st interface, packets marked with 0x2 through the 2nd:
[code:1]ip rule add fwmark 0x1 lookup 10
ip rule add fwmark 0x2 lookup 20[/code:1]
*Note*
The above method does not provide redundancy mechanism. If one of the DSL connections goes dead, half of your connections won't work! You could add redundancy by using the condition match module (offered through patch-o-matic), or by using some script to load different rulesets, depending whether there is connectivity or not through each of the interfaces. Unfortunately, condition match doesn't work on 2.6 kernel as of the moment.
Credit for the above instructions goes to respectable fellow apoikos, from insomnia.gr (greek forum).
You would need the kernel to support the netfilter infrastructure, "Advanced Routing" and "Route by firewall mark" systems. I'm not sure whether the default kernels of most distributions provide these, most likely they do.
Here's a short guide through the procedure (familiarity with iptables is still needed though).
We use the mangle table to mark the 1st packet of every connection via MARK target, save the mark for the whole session (CONMARK target) and mark the rest of the packets of the session with the same mark, so that the packets of the same connection go through the same interface.
1) Initially we create the chain "marking", that will be used for marking packets:
[code:1]iptables -t mangle -N marking[/code:1]
2) We restore the connection marks (if they exist), in both PREROUTING, and OUTPUT, so that connections will be cycled in a round-robin fashion:
[code:1]iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
iptables -t mangle -A OUTPUT -j CONNMARK --restore-mark[/code:1]
3) We mark the unmarked packets, like said, only if they are initiating a connection:[code:1]iptables -t mangle -A PREROUTING -m mark --mark 0x0 -j marking
iptables -t mangle -A OUTPUT -m mark --mark 0x0 -j marking[/code:1]
[code:1]iptables -t mangle -A marking -m mark --mark 0x0 -m nth --every 2 --packet 0 -j MARK --set-mark 0x1
iptables -t mangle -A marking -m mark --mark 0x0 -m nth --every 2 --packet 1 -j MARK --set-mark 0x2
iptables -t mangle -A marking -j CONNMARK --save-mark[/code:1]
Now all the packets of half the connections will be marked as 0x1 and all the packets of the other half of the connections, as 0x2.
4) We create two routes, one for each interface. We also make sure connections to the internal network (192.168.0.0/16 in the example) do not leave through any of these interfaces.
If the public addresses are ON the box, let's say through ethernet-bridge to the dsl routers, or if you are using PCI/USB adsl interfaces and dial to the ISP through the linux box and ppp (in these cases the interfaces would be named by default ppp0 and ppp1), you would add:
[code:1]ip route add default dev ppp0 table 10
ip route add throw 192.168.0.0/16 table 10
ip route add default dev ppp1 table 20
ip route add throw 192.168.0.0/16 table 20[/code:1]
If on the other hand you are using a seperate router for each dsl connection, you would add routes to the addresses of the internal interfaces of these routers:
[code:1]ip route add default via ip-router1-here table 10
ip route add throw 192.168.0.0/16 table 10
ip route add default via ip-router2-here table 20
ip route add throw 192.168.0.0/16 table 20[/code:1]
5) We define routing according to the fwmark. Packets with mark 0x1 will leave through the 1st interface, packets marked with 0x2 through the 2nd:
[code:1]ip rule add fwmark 0x1 lookup 10
ip rule add fwmark 0x2 lookup 20[/code:1]
*Note*
The above method does not provide redundancy mechanism. If one of the DSL connections goes dead, half of your connections won't work! You could add redundancy by using the condition match module (offered through patch-o-matic), or by using some script to load different rulesets, depending whether there is connectivity or not through each of the interfaces. Unfortunately, condition match doesn't work on 2.6 kernel as of the moment.
Credit for the above instructions goes to respectable fellow apoikos, from insomnia.gr (greek forum).
18 years 5 months ago #15128
by Ranger24
Patience - the last reserve of the any engineer
Replied by Ranger24 on topic Re: Load balancing in Linux
Thanks,
You make it sound so simple....
You make it sound so simple....
Patience - the last reserve of the any engineer
18 years 5 months ago #15193
by nske
Replied by nske on topic Re: Load balancing in Linux
Actually by using OpenBSD/FreeBSD and PF it is much more simple (in my opinion). Everything is explained greatly at the official
PF FAQ
that (hate to admit), unlikely the fragmented unofficial documentation of IPtables/IProute2, has the quality and thoroughness of a commercial product.
In PF the whole thing can even be done with one single line of configuration and it looks less like a hack than the above, you should prefer it if possible!
In PF the whole thing can even be done with one single line of configuration and it looks less like a hack than the above, you should prefer it if possible!
Time to create page: 0.129 seconds