Using MetalLB And Traefik for Load balancing on your Bare Metal Kubernetes Cluster – Part 1
Running a Kubernetes Cluster in your own data center on Bare Metal hardware can be lots of fun but also can be challenging.
One of the changeless are exposing your service to an external Load Balancer, Kubernetes does not offer an implementation of an external load-balancer for bare metal cluster implementations.
The Kubernetes implementations of Network Load Balancers are only available on specific Cloud Providers like AWS, GCP, Azure, etc.. and is enabled by specifying the –cloud-provider= option. leaving a gap/challenge if you create your own Bare Metal Kubernetes Cluster.
While their are many workarounds to address this issue most of them are not perfect and have their own list of challenges. while I was looking for a for a solution, most recently came across a project called MetalLB, at first I was quite sceptical if its going to work, but after testing this for a while I have to say, I am quite happy with the solution.
Below is a quote from the MetalLB website.
Kubernetes does not offer an implementation of network load-balancers (Services of type LoadBalancer) for bare metal clusters. The implementations of Network LB that Kubernetes does ship with are all glue code that calls out to various IaaS platforms (GCP, AWS, Azure…). If you’re not running on a supported IaaS platform (GCP, AWS, Azure…), LoadBalancers will remain in the “pending” state indefinitely when created.
Note: On the MetalLB project website they warn you as follows. MetalLB is a young project. You should treat it as a beta system. The project maturity page explains what that implies., however I using this for a while and it seems quite mature.
One last note, I am continuing to use Traefik for internal micro services / load balancing which works great, however MetalLB helps exposing your service for external use.
Now that we have that out of the way, lets see what it takes to install and configure MetalLB on your Kubernetes cluster.
Installing MetalLB on your Kubernetes cluster
You have two options to install MetalLB on your Kubernetes cluster.
- Use the a per-packged MetalLBHelm chart.
- Install / configure manually by applying a YAML configuration file.
Installing MetalLB with Helm
Note: To use Helm you first need to install/configure Helm/Tiller on your Kubernetes cluster. you can see a how-to step by step process here Part 1 and here Part 2.
First, lets create a configmap which will be using at helm install time, the below configmap was used with a flannel configuration.
cat values.yaml
|
1
2
3
4
5
6
|
configInline:
address–pools:
– name: default
protocol: layer2
addresses:
– 10.90.10.190–10.90.10.199
|
Note: There are a number of networking options you can specify likeprotocol: BGP, etc.. please take a look at the MetalLB website for a full list of options.
Once helm is available on your cluster, just run the below to install MetalLB (omit the -tls if your are not using a secure configuration).
Note: The below configuration installs MetalLB with helm including a configmap. (otherwise you get the errors below)
|
1
2
3
4
5
6
7
8
9
10
|
helm install <span style=”font-family:Monaco, MonacoRegular, “Courier New”, monospace;height:inherit;font-size:inherit;line-height:inherit;font-weight:inherit;color:rgb(0, 111, 224);box-sizing:border-box;”–<name=metallb <span style=”font-family:Monaco, MonacoRegular, “Courier New”, monospace;height:inherit;font-size:inherit;line-height:inherit;font-weight:inherit;color:rgb(0, 111, 224);box-sizing:border-box;”–<namespace=metallb–system –f values.yaml stable/metallb <span style=”font-family:Monaco, MonacoRegular, “Courier New”, monospace;height:inherit;font-size:inherit;line-height:inherit;font-weight:inherit;color:rgb(0, 111, 224);box-sizing:border-box;”–<tls
# If all is ok, you shuld see something like the below message at the end of the helm deploy.
MetalLB is now running in the cluster.
LoadBalancer Services in your cluster are now available on the IPs you
defined in MetalLB‘s configuration. To see IP assignments,
try `kubectl get services`.
# Without specifying a configmap.
helm install stable/metallb <span style=”font-family:Monaco, MonacoRegular, “Courier New”, monospace;height:inherit;font-size:inherit;line-height:inherit;font-weight:inherit;color:rgb(0, 111, 224);box-sizing:border-box;”–<name=metallb <span style=”font-family:Monaco, MonacoRegular, “Courier New”, monospace;height:inherit;font-size:inherit;line-height:inherit;font-weight:inherit;color:rgb(0, 111, 224);box-sizing:border-box;”–<namespace=metallb–system <span style=”font-family:Monaco, MonacoRegular, “Courier New”, monospace;height:inherit;font-size:inherit;line-height:inherit;font-weight:inherit;color:rgb(0, 111, 224);box-sizing:border-box;”–<tls
|
Errors you will see if you don’t specify / use a configmap to list your network range, etc..
|
1
2
3
4
|
MetalLB is now running in the cluster.
WARNING: you specified a ConfigMap that isn‘t managed by
Helm. LoadBalancer services will not function until you add that
ConfigMap to your cluster yourself.
|
You can also check the logs to make sure all looks good, by running the below
|
1
|
kubectl logs –l component=speaker –n metallb–system
|
If your cluster is not configured to use RBAC, you might run somthing like the below.
|
1
2
3
4
|
helm install \
<span style=”font-family:Monaco, MonacoRegular, “Courier New”, monospace;height:inherit;font-size:inherit;line-height:inherit;font-weight:inherit;color:rgb(0, 111, 224);box-sizing:border-box;”–<set rbac.create=false \
–f values.yaml \
stable/metallb <span style=”font-family:Monaco, MonacoRegular, “Courier New”, monospace;height:inherit;font-size:inherit;line-height:inherit;font-weight:inherit;color:rgb(0, 111, 224);box-sizing:border-box;”–<name=metallb <span style=”font-family:Monaco, MonacoRegular, “Courier New”, monospace;height:inherit;font-size:inherit;line-height:inherit;font-weight:inherit;color:rgb(0, 111, 224);box-sizing:border-box;”–<namespace=metallb–system <span style=”font-family:Monaco, MonacoRegular, “Courier New”, monospace;height:inherit;font-size:inherit;line-height:inherit;font-weight:inherit;color:rgb(0, 111, 224);box-sizing:border-box;”–<tls
|
An example configmap if using a BGP overlay system.
|
1
2
3
4
5
6
7
8
9
10
11
|
# cat values.yaml
configInline:
peers:
– peer–address: 10.0.0.1
peer–asn: 64512
my–asn: 64512
address–pools:
– name: default
protocol: bgp
addresses:
– 10.90.10.190–10.90.10.199
|
If all works correctly, you should now have a MetalLB working configuration.
You can also check what helm report about the installation.
|
1
2
3
|
helm list <span style=”font-family:Monaco, MonacoRegular, “Courier New”, monospace;height:inherit;font-size:inherit;line-height:inherit;font-weight:inherit;color:rgb(0, 111, 224);box-sizing:border-box;”–<tls
NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE
metallb 1 Mon Feb 11 19:52:17 2019 DEPLOYED metallb–0.8.4 0.7.3 metallb–system
|
To remove MetalLB from your cluster, just run the below.
|
1
|
helm delete <span style=”font-family:Monaco, MonacoRegular, “Courier New”, monospace;height:inherit;font-size:inherit;line-height:inherit;font-weight:inherit;color:rgb(0, 111, 224);box-sizing:border-box;”–<purge metallb <span style=”font-family:Monaco, MonacoRegular, “Courier New”, monospace;height:inherit;font-size:inherit;line-height:inherit;font-weight:inherit;color:rgb(0, 111, 224);box-sizing:border-box;”–<tls
|
Installing MetalLB manually
To install MetalLB manually, you would do so in cases ware you don’t have a helm configuration or you would like to install a version not available in the helm repository.
To MetalLB from the web just run the below.
|
1
|
kubectl apply –f https://raw.githubusercontent.com/google/metallb/master/manifests/metallb.yaml
|
You can also create a deploy file, like the one below.
|
1
2
3
4
5
6
7
8
9
10
11
12
13
|
cat deploye_metal.yaml
apiVersion: v1
kind: ConfigMap
metadata:
namespace: metallb–system
name: config
data:
config: |
address–pools:
– name: metal–lb–ip–space
protocol: layer2
addresses:
– 10.90.10.190–10.90.10.199
|
Then to install just run the below, this will already include your IP address definitions.
|
1
|
kubectl apply –f deploye_metal.yaml
|
A quick check on your pods will reveal something like the below, note the metallb-controller and metallb-speaker.
|
1
2
3
4
5
6
|
kubectl get po <span style=”font-family:Monaco, MonacoRegular, “Courier New”, monospace;height:inherit;font-size:inherit;line-height:inherit;font-weight:inherit;color:rgb(0, 111, 224);box-sizing:border-box;”–<all–namespaces
metallb–system metallb–controller–776cdd69dc–4lzcx 1/1 Running 0 9d
metallb–system metallb–speaker–4xrnm 1/1 Running 0 9d
metallb–system metallb–speaker–g59h6 1/1 Running 0 9d
metallb–system metallb–speaker–jzh4g 1/1 Running 0 9d
metallb–system metallb–speaker–k6spd 1/1 Running 0 9d
|
MetalLB in action
To test MetalLB working, you can deploy any pod(s) to you cluster, and specify the Cluster Type as LoadBalancer.
An example Nginx deployment is below.
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
|
# cat nginx_load_balance.yaml
apiVersion: apps/v1beta2
kind: Deployment
metadata:
name: nginx
spec:
selector:
matchLabels:
app: nginx
replicas: 2
template:
metadata:
labels:
app: nginx
spec:
containers:
– name: nginx
image: nginx:1
ports:
– name: http
containerPort: 80
<span style=”font-family:Monaco, MonacoRegular, “Courier New”, monospace;height:inherit;font-size:inherit;line-height:inherit;font-weight:inherit;color:rgb(0, 111, 224);box-sizing:border-box;”–<–
apiVersion: v1
kind: Service
metadata:
name: nginx
spec:
ports:
– name: http
port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
type: LoadBalancer
|
Now, just run the below to deploy to your cluster.
|
1
|
kubectl apply –f nginx_load_balance.yaml
|
Now take a look on your Kubernets Cluster services, note the EXTERNAL-IP that got assigned automatically by MetalLB.
|
1
2
3
4
5
6
|
kubectl get svc <span style=”font-family:Monaco, MonacoRegular, “Courier New”, monospace;height:inherit;font-size:inherit;line-height:inherit;font-weight:inherit;color:rgb(0, 111, 224);box-sizing:border-box;”–<all–namespaces
NAMESPACE NAME TYPE CLUSTER–IP EXTERNAL–IP PORT(S) AGE
default kubernetes ClusterIP 10.30.0.1 <none> 443/TCP 14d
kube–system tiller–deploy ClusterIP 10.30.4.154 <none> 44134/TCP 13d
default nginx LoadBalancer 10.30.2.56 10.90.10.190 80:30827/TCP 2s
...
|
In the next post Part 2 – I will show you how to combine this with Traefik for internal Micro-services Load Balancing.
I hope you enjoyed reading Using MetalLB And Traefik Load Balancing For Your Bare Metal Kubernetes Cluster, give it a thumbs up by rating the article or by just providing feedback.
You might also like – realted to Docker Kubernetes / micro-services.