Microk8s Cluster in LXD

The installation of a 6-node cluster in LXD for testing services of a SOC took me more time than expected, and after resolving several (and stressful) problems, I was finally able to get a functional cluster using Microk8s. Here are the steps:

Prerequisites

• It is necessary to have LXD installed on the host machine. In my case, I used the version that comes with Snap, as it includes a web interface to facilitate container management. The installation of LXD depends on the Linux distribution being used. The guide to installing the web interface can be found here.

• At least 8 GB of RAM is required for a 3-node cluster. If more nodes are required, the system may become unstable.

Installing Rocky Linux in LXD

Once LXD is installed, the master node (master) will be installed, where all necessary configurations will be made, and subsequent nodes will be created from this container.

First, you need to download the special profile created to run Microk8s in LXD. There are two versions of this profile, depending on the file system of your Linux installation on the host. For EXT4, use this, and for ZFS, use this.

Add this profile (in this example, for EXT4) to an LXD project in the “Profiles” section. For this cluster, a project called kubeTest was created, as shown in the image:

Once the profile is added, proceed to create the container with Rocky Linux in the “Instances” section by pressing the “Create Instance” button. The following images show this process:

Following these steps, you should now have Rocky Linux installed in LXD.

Adding a non-root user, installing Open SSH, and nfs-utils in the container

It is essential to add a non-root user to manage Microk8s:

1
2
3
4
5

# adduser node
# passwd node
# usermod -aG wheel node
# dnf update && dnf upgrade -y
# dnf install openssh-server nano nfs-utils -y

Installing SNAP and Microk8s in the container

To install SNAP, follow the steps described in snapcraft.

To install Microk8s, write the following commands as root:

1
2

# setenforce 0
# sed -i 's/^SELINUX=.*/SELINUX=permissive/g' /etc/selinux/config

Install Microk8s and add the user node to the microk8s group:

1
2

# snap install microk8s --classic
# usermod -aG microk8s node

As the user node, create the .kube directory, configure permissions, add aliases to .bashrc, and create the .ssh directory:

1
2
3
4
5
6
7

# su node
$ cd /home/node
$ mkdir .kube
$ sudo chown -f -R node ~/.kube
$ echo "alias kubectl='microk8s kubectl'" >> .bashrc
$ echo "alias helm='microk8s helm3'" >> .bashrc
$ ssh-keygen

If necessary, add a public key to the .ssh/authorized_keys file.

Verify the status of Microk8s:

1

$ microk8s status

Creating a snapshot

In the Snapshots section of the master node, create a snapshot that will be used to create the other nodes of the cluster. Before creating the snapshot, the container must be shut down:

Creating cluster nodes

In a terminal of the host, copy the snapshot:

1
2
3
4
5

$ lxc copy master/basemaster node2
$ lxc copy master/basemaster node3
$ lxc copy master/basemaster node4
$ lxc copy master/basemaster node5
$ lxc copy master/basemaster node6

At this point, you can create as many nodes as you need, but at least three are required.

Adding nodes to the cluster

Up to this point, the “Instances” section should show all the running containers:

In the terminal of the master node, as the user node, write the command:

1

$ microk8s add-node

The execution of this command generates a unique key. This key must be executed on a single node. Therefore, different keys must be generated for each node.

Run the following command on the master node to verify the status of the cluster:

1

$ microk8s kubectl get no

You should have a list of all the nodes that make up the cluster:

From this point on, you can deploy services, which will be covered in future posts.