Traefik as K8S Ingress Controller
It’s not secret that K8S documentation needs improvements. K8S community is great, but still, contributions to the docs side could be better. For finding some answers, it’s normal to consult official docs, but sometimes, that’s just not enough. In this blog post, I will try to write a “technically rich” procedure how to deploy a Traefik as Ingress controller and all its components.
What is an Ingress?
As you already know, Ingress is an API object that manages external access to services in the cluster, mostly over HTTP/HTTPS. To get Ingress resource working, you need an Ingress controller. If you’re using GCE, Ingress controller is already deployed to the master. However, if you bootstrapped a cluster by yourself, eg. with kops on AWS, you need to deploy Ingress controller by yourself. On minikube, you have to enable Ingress add-on.
Ingress Controllers
In this post, it will be shown how to set up Traefik and use it as Ingress Controller for every service in your cluster. However, you can use NGINX Ingress Controller, Kong, Octavia Ingress Controller, etc…
Why?
Why to use Ingress controller if you can expose your service via NodePortor LoadBalancer service types? Simply, you can have a central point for proxying your traffic and cost savings as well. With usage of Ingress Controller you will need a LoadBalancer ONLY for Traefik and nothing else. All traffic will be served with Traefik and its dedicated LoadBalancer.
Traefik components
First of all, Traefik announced Kubernetes Ingress support from version 1.4. However, recently released Traefik 1.7 has publishedService option that will update Ingress status field what wasn’t a case in previous versions. Components that are needed to get the whole setup running are:
- •Create a namespace
- •Create a service account
- •Create a TLS secret
- •Create a cluster role
- •Create a cluster role binding
- •Create a configmap
- •Create a deployment
- •Create a service for http and https
- •Create a service for Traefik dashboard
- •Create an Ingress
Namespace
Create a dedicated namespace:
kubectl create namespace traefik
TLS Secret
Create certificate first:
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ./tls.key -out ./tls.crt -subj "/CN=*.example.com"Create a TLS certificate:
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ./tls.key -out ./tls.crt -subj "/CN=*.example.com"Create a secret object:
kubectl create secret tls traefik-ui-tls-cert --key ./tls.key --cert ./tls.crt
Visit Github’s README file for better copy/paste experience.
ConfigMap
---
apiVersion: v1
kind: ConfigMap
metadata:
name: traefik-configmap
namespace: traefik
data:
traefik.toml: |
defaultEntryPoints = ["http","https"]
[entryPoints]
[entryPoints.http]
address = ":80"
[entryPoints.https]
address = ":443"
[entryPoints.https.tls]
[[entryPoints.https.tls.certificates]]
CertFile = "/ssl/tls.crt"
KeyFile = "/ssl/tls.key"
[entryPoints.traefik]
address = ":8080"
[entryPoints.traefik.auth.basic]
users = ["admin:$apr1$zjjGWKW4$W2JIcu4m26WzOzzESDF0W/"]
[kubernetes]
[kubernetes.ingressEndpoint]
publishedService = "traefik/traefik"
[ping]
entryPoint = "http"
[api]
entryPoint = "traefik"
Default entry points are ports 80 and 443.
http entry point is listening on :80 and has no additional rules
https entry point is listening on :443 and has rule to load TLS certificates
traefik entry point is listening on :8080 and requires basic authentication. Username is admin and password is admin .
Defining Kubernetes Ingress endpoint is done by defining publishService and it must contain values of namespace and name of Traefik service . In this case, that is traefik/traefik .
ping, or health check will use http endpoint.
api , or dashboard/ui will use traefik endpoint.
Note that you can define additional entry points on some custom port. That port can be used to proxy traffic to some custom port instead of using ephemeral ports and NodePort .
ClusterRole and ClusterRoleBinding
Traefik Service account needs a permission to update Ingress Status field. This is an important setting and it is not presented in Traefik’s official docs yet:
--- kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: traefik-ingress-controller rules: - apiGroups: - "" resources: - services - endpoints - secrets verbs: - get - list - watch - apiGroups: - extensions resources: - ingresses verbs: - get - list - watch - apiGroups: - extensions resources: - ingresses/status verbs: - update<br/--<- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: traefik-ingress-controller roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: traefik-ingress-controller subjects: - kind: ServiceAccount name: traefik-ingress-controller namespace: traefik
Last 6 lines are very important to get this setup working as expected.
Deployment
Deployment is mostly self-explanatory and clear. However, let’s make clear some parts of it:
volumes:
- name: traefik-ui-tls-cert
secret:
secretName: traefik-ui-tls-cert
- name: traefik-configmap
configMap:
name: traefik-configmap
Define configmap and secret as volumes and then define volumeMounts to use them properly.
volumeMounts:
- mountPath: "/ssl"
name: "traefik-ui-tls-cert"
- mountPath: "/config"
name: "traefik-configmap"
Liveness and readiness probes (ping) are peformed on port 80, just as we defined that in the config file inside configmap.
Expose all ports/entry points from config file:
ports:
- name: http
containerPort: 80
- name: https
containerPort: 443
- name: dashboard
containerPort: 8080
Again, some examples on internet/docs are out of date and then you have to take care about args needed to run Traefik:
args: - --logLevel=INFO - --configfile=/config/traefik.toml
Do not pass additional flags to args such -api , -ping , or -kubernetes because that will override the configuration in config file.
Service
First service will expose http and https entry points to LoadBalancer. If you check LoadBalancer Security Group (ingress), ports 80 and 443 are opened. K8S will automatically create a LoadBalancer and join a node(s) that is/are running Traefik pods. If you want to create internal ELB as I did, you have to define annotations.
---
kind: Service
apiVersion: v1
metadata:
name: traefik
namespace: traefik
annotations: {}
# service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0
spec:
selector:
k8s-app: traefik-ingress
ports:
- protocol: TCP
port: 80
name: http
- protocol: TCP
port: 443
name: https
type: LoadBalancer
Service (for Dashboard)
This is the cool part! I wanted to have Dashboard under https since it requires basic authentication. The Dashboard is running on port :8080 and we need to redirect it to use SSL. Service is very simple and real magic will happen in the Ingress object.
---
kind: Service
apiVersion: v1
metadata:
name: traefik-dashboard
namespace: traefik
spec:
selector:
k8s-app: traefik-ingress
ports:
- port: 8080
name: dashboard
Ingress (for Dashboard)
The magic part of a proxying secured Traefik Dashboard through Traefik itself is defined in Ingress object. Controlling Traefik ingress is possible by using Traefik annotations.
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: traefik-dashboard-ingress
namespace: traefik
annotations:
kubernetes.io/ingress.class: traefik
traefik.ingress.kubernetes.io/frontend-entry-points: http,https
traefik.ingress.kubernetes.io/redirect-entry-point: https
traefik.ingress.kubernetes.io/redirect-permanent: "true"
spec:
rules:
- host: traefik-ui.example.com
http:
paths:
- path: /
backend:
serviceName: traefik-dashboard
servicePort: 8080
We need 4 annotations and they are all self-explanatory.
Of course, to get it fully working you have to create DNS record traefik-ui.example.com that will point your ELB.
It would be cool to have that feature automagically done for you? Not a problem, I will write about creating automatic DNS records in next blog.
Links
Full deployment.yaml file is available on my Github repository.
If you have any trouble with getting it working do not hesitate to ask for help.
If you like it ->