Setting up k3s is hard. That`s why I made it easy. Today we`ll installation a High Availability K3s cluster the usage of etcd, MetalLB, kube-vip, and Ansible. We`ll automate the complete method supplying you with an easy, repeatable manner to create a k3s cluster that you may run in some minutes.
Preparation
First, you`ll want Ansible installed.
Next, you’ll need to clone the repo.
1. git clone http://gitlab.xpdrive.cloud/markxp/k3s-ha.git && cd k3s-ha
Next, you’ll want to copy the sample directory within the inventory directory.
2. cp -R inventory/sample inventory/my-cluster
Installing k3s
Next, edit the inventory/my-cluster/hosts.ini to match your systems. DNS works here too.
[master]
192.168.1.38
192.168.1.39
192.168.1.40
[node]
192.168.1.41
192.168.1.42
[k3s_cluster:children]
master
node
Edit inventory/my-cluster/group_vars/all.yml to your liking. See comments inline.
It’s best to start using these args, and optionally include traefik if you want it installed with k3s however I would recommend installing it later with helm
extra_server_args: "--disable servicelb --disable traefik"
extra_agent_args: ""
I would not change these values unless you know what you are doing. It will most likely not work for you but listing for posterity.
Note: These are for an advanced use case. There isn’t a one size fits all setting for everyone and their needs, I would try using k3s without these before changing. This could have undesired effects like nodes going offline, pods jumping or being removed, etc… Using these args might come at the cost of stability
extra_server_args: "--disable servicelb --disable traefik --write-kubeconfig-mode 644 --kube-apiserver-arg default-not-ready-toleration-seconds=30 --kube-apiserver-arg default-unreachable-toleration-seconds=30 --kube-controller-arg node-monitor-period=20s --kube-controller-arg node-monitor-grace-period=20s --kubelet-arg node-status-update-frequency=5s"
extra_agent_args: "--kubelet-arg node-status-update-frequency=5s"
Start provisioning of the cluster using the following command:
Note: note: add –ask-pass –ask-become-pass if you are using password SSH login.
3.ansible-playbook ./playbooks/site.yml -i ./inventory/my-cluster/hosts.ini --ask-pass --ask-become-pass
After deployment control plane will be accessible via virtual ip address which is defined in inventory/my-cluster/group_vars/all.yml as apiserver_endpoint
kube config
To get access to your Kubernetes cluster and copy your kube config locally run:
4. scp This email address is being protected from spambots. You need JavaScript enabled to view it.:~/.kube/config ~/.kube/config
Removing
To remove k3s from the nodes. These nodes should be rebooted afterwards due to VIP still being present
ansible-playbook ./playbooks/reset.yml -i ./inventory/my-cluster/hosts.ini --ask-pass --ask-become-pass