Introduction
R5.0 is right now in beta testing and this cluster feature is not generally available.
Starting at R5.0, several BQN servers can be associated into one cluster. The cluster covers management aspects: common rule configuration, common software updates and common visibility from either of the servers forming the cluster.
Logging into any of the servers in the cluster:
The home page shows the traffic throughput of each server and clicking on a BQN box will go to the familiar BQN home page of that server.
The pages can show not only information of the local server, but of any server of the cluster. For example, in Status->Interfaces->Throughput, a selector in the upper right can show throughput of a specific BQN Id or for all servers in the cluster (the icon next to BQN 0 indicates that we are logged in BQN 0):
In this version, the cluster does not include API aspects (integration via REST, RADIUS or billing systems), that each BQN server carries out on its own. It does not cover data plane traffic distribution either: each BQN server will receive its traffic according to an external traffic distribution (e.g. based on the place in the ISP network where each BQN server is deployed).
Cluster Deployment
The following is an example of a three-server cluster deployment:
Each server in the cluster has a unique identifier, in this example, identifiers 0, 1 and 2. The server with the lowest Id (0 in this case) is the one communicating with the other servers in the cluster, it is what we call the primary server. The other two are secondary servers. The cluster is resilient: if the primary server becomes unavailable, the next with the lowest Id (1 in our example) will claim the primary role.
The communications between cluster servers use the server management IP addresses. Two types of communication exist:
- Cluster control traffic: TCP, with ports in63500-63755 range. A BQN will listen in port 63500 + Id. E.g. BQN of Id 0 willuse port 63500, BQN of Id 1 port 63501, etc.
- Information transfers: SSH/SCP in port 22.
Every server in the cluster must have connectivity to the other servers using these ports.
To define a cluster, we need the list of Ids with the IP address and SSH port where they are reachable. This is straight forward if all the server management addresses are in the same subnet.
If there are NATs or port forwarding, this must be taken into account when setting up the cluster. In the following example, BQN are reachable through public IP addresses and their SSH port is port forwarded.
The next section describes the details to verify network connectivity and the configuration steps in order to build a cluster.
Cluster Configuration
A step-by-step procedure to create a cluster follows. We will use a simple example of cluster with a primary server with BQN ID 0 andonly one secondary with ID 1.
Software Installation
All BQN servers to be part of the cluster must have the BQN software installed (R5.0 or higher), following the steps described in Software Installation section.
Setting BQN Identifiers (BQN ID)
Each BQN server will have a unique identifier. The BQN with the lowest ID will act as primary server.
Our recommendation is to start from ID 0 and continue with 1, 2, etc.
By default, a BQN server has ID 0, so the one to be the primary will not need a change of ID. For the other servers, follow this procedure:
- Go to Administration->Cluster andclick on Modify BQN ID.
- Enter the new BQN ID value, between 0 and 255 (e.g. 1). The change reloads the software, so, if the BQN is processing traffic, it causes a brief interruption of service. The GUI will log you out. Log in again to continue.
OAM IP addresses connectivity
The BQN management IP addresses must be reachable from each other. So, for example, for two BQNs with IP addresses 192.168.56.111 and 192.168.56.112, we access both servers to check their mutual connectivity:
The cluster uses the TCP port range between 63500 and 63755. The port depends on the BQN identifier. For example, if only IDs 0 and 1 are used, the ports 63500 and 63501 will be used for cluster control.
To check that those ports are open, generate TCP traffic to each BQN address and port with telnet and check that it is received using tcpdump on the destination. For example, to check bqn0 can send TCP to bqn1, we need to check on port 63501 (the port a bqn with ID 1 is listening to). Start tcpdump on destination:
Generate traffic to port using telnet (it is expected to be rejected, no listening when the cluster is not yet configured):
while on destination, the traffic is received:
Likewise with port 63500 from bqn1 to bqn0.
SSH is also used. To check an ssh from bqn0 to bqn1.
And likewise from bqn1 to bqn0:
If a third server is to be added to the cluster, the connectivity between this server and the other two must be checked as previously described.
Sometimes, a BQN OAM IP address is reachable through a public IP address that is mapped to the BQN actual IP. In that case, you need to add the cluster ports to the mapping (e.g. 63500-63501). When verifying connectivity, use the public IP address to reach that BQN server.
Unify configuration across the cluster
If the cluster will be built with BQNs previously in use as standalone, you need to check their configurations to spot differences and settle on a common one. If there are differences and no unification is done, the configuration of the primary server will be used as configuration of the cluster.
The following example shows a typical procedure. We will have two servers to add to cluster.
- Go to primary server Administration->Backup->SaveConfiguration and save configuration (e.g. bqn0-v1.conf).
- Do the same in secondary server Administration->Backup->SaveConfiguration and save configuration (e.g. bqn1-v1.conf).
- Compare both files. For example, using kdiff3 tool, we find the following differences:
- NTP server 216.229.0.50 is missing in bqn0. Wewill add it.
- bqn0 has no REST API enabled. We add it.
- Interface differences are OK, they are server specific,so we take no action. Routing is also server specific (it is the same in this example, but if it were different, they would not be merged).
- In speed-tests profile, bqn1 has an extraentry, we add it to bqn0.
- In sw-updates, bqn0 has an extra entry, weadd it to bqn1.
- bqn0 has an extra rule for sw updates during peak time. We add the policy and the rule to bqn1 (the time profile was already there).
After the previous steps, we should have abqn0-v2.conf and bqn1-v2.conf with the merged configuration.
Go to primary server Administration->Backup->Load Configuration and load bqn0-v2.conf.
Do the same in the secondary server with bqn1-v2.conf.
Configuring the Cluster
We will start the creation of the cluster by configuring inthe primary server. Go to the primary server GUI and select Administration->Cluster and click on Add BQN to a cluster...
In the dialogue, enter the secondary BQN ID (1) and its BQN management IP address. In IP Address for this BQN in cluster, enter the primary address.
In this example, we use the management IP addresses 192.168.56.111 and 192.168.56.112. If the servers are accessible via public IP addresses, use them instead.
The same procedure is followed in the secondary BQN server: go to the secondary GUI and select Administration->Cluster and click on Add BQN to a cluster...
In the dialogue, enter the primary BQN ID (0) and the BQN IP address. In the IP Address for this BQN in cluster field, enter the IP address of the secondary server.
The IPs can be the management addresses, like in this example, but if any of the servers is accessible through a public IP address, use that address instead.
The cluster should show a state ready and a status complete:
You can repeat the process to add more servers to the cluster by going to that server GUI and clicking on Add BQN to a cluster.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.