Bonding internet connections
I live in a rural area where the only means of getting online is 4,5/0,4 Mbit DSL. As you can imagine this becomes a big problem when multiple people are trying to use the line at the same time (read: bufferbloat). Recently we got an extra line from another ISP in hope that we could improve the situation somewhat by combining them.
There are some resources available on this subject already, but they either doesn’t work, or doesn’t cover the whole process of setting it up. With this article I’m trying to change that. So if there’s anything I’ve missed or forgotten to add, please create an issue at the blog’s repository.
- You’re using an EdgeRouter as the client
- You’re running a Debian-based server on the endpoint
- You don’t need OpenVPN’s security functions
Setting up the server
First of all we’ve got to set up the server that’ll act as our gateway to the internet. To accomplish this, we’re going to set up two OpenVPN servers simulating two point-to-point ethernet connections.
/etc/modules file with your editor of choice and add the
bonding module to it. This’ll make the
bonding module load on system boot.
# /etc/modules: kernel modules to load at boot time.
Now we need to create the file
/etc/modprobe.d/bonding.conf. Here we’ll set the options for the module. Full documentation on the
bonding module is available here.
# mode=0 means round-robin
Now we’ll have the
bond0 interface available after booting the server, but it doesn’t have an IP address yet. So we need to edit the
/etc/network/interfaces file and add the following. Change
10.75.0.1 to whatever address you’d like, as with the netmask.
You should also change the MTU to what fits for your network. Find the MTU for your uplink(s) using this guide, and subtract 46 from the result when using IPv4, 66 when using IPv6, this accounts for the overhead of using OpenVPN. If the MTU for your uplinks differ, use the lowest value.
I’m also adding a
post-up command, that’ll add a route for 192.168.0.0/24 (assuming this is your LAN subnet) on the
bond0 interface. This way we don’t need to set up NAT on the EdgeRouter, and thus avoid double-NAT’ing.
Since multiple clients are going to connect to the internet from the same public IP address, we need to set up NAT. First make sure the
iptables-persistent package is installed, then add a NAT rule that modifies the source address on all packets going out the WAN interface to your public IP.
You should also configure the firewall to secure the server and your network, but that’s outside the scope of this article.
apt install iptables-persistent
First we need to create the TAP devices. Open the
/etc/network/interfaces file again and add the contents below. For the sake of clarification, we’re setting the interface down right after it’s creation because you can’t add an active interface as a bond slave. Adding the device as a slave brings it up again, which we don’t want to happen until the OpenVPN connection is established, so we’re bringing it down again.
This is the server OpenVPN config we’re using. We need one OpenVPN instance per internet connection, so create one file in
/etc/openvpn per instance. For example
/etc/openvpn/isp_2.conf. You need to change the
local option to the IP address you’d like OpenVPN to bind to, and the
dev option to one of the TAP devices you created earlier.
# Encryption adds extra overhead, so I'm disabling it.
Now we need to create the down/up scripts for OpenVPN. These scripts simply set the TAP interface down or up, depending on whether the OpenVPN connection is up or not. This allows the bond interface to determine which interface is available for sending on.
Add this to
Add this to
Make sure the scripts are executable:
chmod +x /etc/openvpn/tap-down
And last enable the OpenVPN service, so our servers are started on system boot:
systemctl enable openvpn.service
Restart the server to make our changes active.
Setting up the client
As mentioned earlier, I’m using an EdgeRouter (EdgeRouter Lite to be specific) for this setup, so all configuration examples will only work on the EdgeRouter series of devices (and maybe Vyatta/VyOS).
First we need to create the bond interface, make sure that the subnet matches what you set on the server.
set interfaces bonding bond0 address 10.75.0.2/24
Now we need to set up the OpenVPN clients. Just to be clear, normally you would configure the client only by using
set interfaces openvpn commands. However I’ve not figured out how to disable the security features using that, so we’re doing this with
.ovpn files instead.
It doesn’t really matter where you save them, as long as it’s beneath the /config folder. Like on the server, we need one file per connection.
# Disable encryption
/config/scripts/tap-down script bringing the TAP device down on connection close.
/config/scripts/tap-up script bringing the TAP device up when the connection is established. You’ll notice that this one does more than the one we created on the server. This is because EdgeOS doesn’t support adding OpenVPN TAP interfaces to a bond. So as a workaround we’re adding the TAP interface to the bond when the connection is established, IF it isn’t already.
set interfaces openvpn vtun10 config-file /config/vpn-isp1.ovpn
Now on to a tricky part. By default all VPN connections would go over one WAN interface, which we have to avoid. The trick is using multiple routing tables, and selecting which one to use based on the source IP address of the connection. I’ve added dummy networks in this example, so make sure to replace them with whatever gateway and netmask your ISPs are using.
set protocols static table 10 description "VPN from ISP1 uplink"
If you haven’t done so already, now would be a good time to commit your changes. And preferably save them too.
Last thing to do is adding the rules for selecting which routing table to use for which traffic.
I would like to mention this question over at StackOverflow. The poster answered his own question, and it was a great help when first setting this up.