Installing Kubernetes on my Ubuntu laptop

Why?

Since the company I'm working with uses Kubernetes in production, I like to have a sandbox handy to try upgrades and just for general experimentation.

Prerequisites

I'm using Ubuntu 19.04 "Disco Dingo".

I'm going to install Kubernetes v1.15.4. The current newest version works a bit differently that makes the latest stable flannel release not to work properly. The procedure below can probably be repeated with now-current (in reader-time :) ) versions as long as they're compatible.

Docker

The docker I choose is version 18.09. Check the supported docker version with your Kubernetes version. Do not skip this step. I found this information in the release notes for v1.15: https://v1-15.docs.kubernetes.io/docs/setup/release/notes/

Flannel

I'm using flannel with the local installation. The latest stable at this time is v0.11.0. I found no information on the compatiblity of different flannel and Kubernetes versions, so I just tried the latest of both and I got lucky the first time.

If I'd start the installation today, I'd be out of luck since flannel v0.11.0 does not work with Kubernetes v1.16.0. The flannel manifest on the master branch works fine, but I'd rather use a stable release, and hence I'd go back to use Kubernetes v1.15.4.

The problem with flannel v0.11.0 and Kubernetes v1.16.0 is that it stopped serving certain deprecated APIs, namely DaemonSet has been moved from extensions/v1beta1 to apps/v1beta2 (see https://kubernetes.io/blog/2019/07/18/api-deprecations-in-1-16/ for more details).

Even after fixing the v0.11.0 flannel manifest, the CNI wouldn't work properly, and "kubectl describe node <nodename>" shows "network plugin is not ready: cni config uninitialized". This is probably an other  incompatibility between my particular flannel - Kubernetes versions.

Network setup

I'd like to run the (one-node) cluster on a laptop. This laptop may or may not have network connectivity, and both the public facing interface and IP address will change a lot.

Unfortunately Kubernetes does not like to work with loopback interfaces. I couldn't get Kubernetes 

to use the loopback address neither as the advertised API server address nor as the node IP.

So I needed an interface with a static IP regardless of Internet connectivity.

Enter the dummy interface.

A dummy interface on Linux is really just another loopback device: it does the same thing really without actually being a loopback device. This is important, since I could find lines in the Kubernetes code that were meant to actively avoid using the loopback device. Since a "dummy" is not "loopback", Kubernetes will be happy with one, and will need little persuasion to use this dummy interface with a static address.

So let's edit /etc/network/interfaces and add these lines:

auto aaa0
iface aaa0 inet static
address 172.31.0.1/16
pre-up modprobe dummy; ip link add aaa0 type dummy
post-down ip link del aaa0

I'm using a name that will make the interface appear on the top of the list provided by "ifconfig". I don't know if this has any importance in our case. You could probably use any name.

Docker setup

Let's install the appropriate docker-ce version, but first let's make sure no conflicting version is installed:

$ sudo apt remove docker docker-engine docker.io

$ sudo apt install apt-transport-https ca-certificates curl software-properties-common

$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

$ sudo apt-add-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu cosmic stable"

You can view the available version with apt-cache madison:

$ apt-cache madison docker-ce

docker-ce | 5:19.03.2~3-0~ubuntu-cosmic | https://download.docker.com/linux/ubuntu cosmic/stable amd64 Packages

docker-ce | 5:19.03.1~3-0~ubuntu-cosmic | https://download.docker.com/linux/ubuntu cosmic/stable amd64 Packages

docker-ce | 5:19.03.0~3-0~ubuntu-cosmic | https://download.docker.com/linux/ubuntu cosmic/stable amd64 Packages

docker-ce | 5:18.09.9~3-0~ubuntu-cosmic | https://download.docker.com/linux/ubuntu cosmic/stable amd64 Packages

docker-ce | 5:18.09.8~3-0~ubuntu-cosmic | https://download.docker.com/linux/ubuntu cosmic/stable amd64 Packages

docker-ce | 5:18.09.7~3-0~ubuntu-cosmic | https://download.docker.com/linux/ubuntu cosmic/stable amd64 Packages

docker-ce | 5:18.09.6~3-0~ubuntu-cosmic | https://download.docker.com/linux/ubuntu cosmic/stable amd64 Packages

docker-ce | 5:18.09.5~3-0~ubuntu-cosmic | https://download.docker.com/linux/ubuntu cosmic/stable amd64 Packages

docker-ce | 5:18.09.4~3-0~ubuntu-cosmic | https://download.docker.com/linux/ubuntu cosmic/stable amd64 Packages

docker-ce | 5:18.09.3~3-0~ubuntu-cosmic | https://download.docker.com/linux/ubuntu cosmic/stable amd64 Packages

docker-ce | 5:18.09.2~3-0~ubuntu-cosmic | https://download.docker.com/linux/ubuntu cosmic/stable amd64 Packages

docker-ce | 5:18.09.1~3-0~ubuntu-cosmic | https://download.docker.com/linux/ubuntu cosmic/stable amd64 Packages

I'd like to use version 18.09, so I'll issue the following commands:

$ sudo apt install docker-ce=5:18.09.9~3-0~ubuntu-cosmic

$ sudo apt-mark hold docker-ce

Let's create the file /etc/docker/daemon.json with the following content:

{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "10"
  },
  "storage-driver": "overlay2"
}

The most important part is the cgroup driver set to systemd. I'm going to use overlay2 as the storage driver merely because it works and does not require any further setup. Logging will take place in the JSON format, and log-opts will prevent docker from keeping too much of them.

Since this is my laptop we're talking about here, I don't like docker or Kubernetes to start automatically, I don't need them all the time, especially not when I'm using the laptop while travelling or otherwise being contrained by the battery, so I'll go ahead and prevent docker from starting automatically, and also restart it for the above configuration to take effect:

$ systemctl disable docker
$ service docker restart

Docker is ready and should be running (check it with "service docker status").

Installing Kubernetes

Let's add the appropriate public key to apt's keystore, add the repository, and install the correct version. (Hint: use apt-cache madison to check for available versions).

$ curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add

$ sudo apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main"

$ sudo apt install kubelet=1.15.4-00 kubeadm=1.15.4-00 kubectl=1.15.4-00

$ sudo apt-mark hold kubelet kubeadm kubectl
$ sudo systemctl disable kubelet

That's it, we have all the software installed on the local system for Kubernetes to work.

Setting up Kubernetes

I'm going to use kubeadm to take care most installations steps. Let's write out the default configuration first:

$ kubeadm config print init-defaults | sudo tee /etc/kubeadm.conf

Open that file in an editor, and look for the "networking" key. Add the "podSubnet" under it.

networking:
  ...
  podSubnet: 10.244.0.0/16
  ...

At the end of the file, add the following snippet:

---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
failSwapOn: false
evictionHard:
  imagefs.available: 5%
  memory.available: 100Mi
  nodefs.available: 5%
  nodefs.inodesFree: 5%

This will make sure that Kubelet will start with swap on.

The other thing I'm going to deal with is the node IP. Kubelet will still going to use the public-facing interface (eth0 or wlan0 in my case) and the IP address of it as the node address. Kubelet's documentation mentions the --node-ip option.

Reading kubelet's systemd service unit file revealed that extra options can be passed to kubelet via the KUBELET_EXTRA_ARGS variable. The variable should be defined in /etc/default/kubelet:

KUBELET_EXTRA_ARGS="--node-ip=172.31.0.1"

We can now fire up kubeadm and it will set up our single node cluster.

$ sudo kubeadm init --config=/etc/kubeadm.conf --ignore-preflight-errors=Swap

I need to use the --ignore-preflight-errors=Swap option to tell kubeadm that it is OK to proceed with swap enabled. Kubeadm and Kubelet are two independent components, each of them must be told to ignore swap being turned on.

I'm going to get warnings about the swap, and that docker is disabled, but that's OK.

To be able to use kubectl I need authentication. A configuration file with the appropriate credentials is written to /etc/kubernetes/admin.conf. Copying it to ~/.kube/config enables me to use kubectl:

$ mkdir $HOME/.kube
$ cp /etc/kubernetes/admin.conf $HOME/.kube/config

The cluster might need a minute or two to start up, but after that, I should see some activity:

$ kubectl get -A pods -o wide

There sould be several pods running, and coredns pending.

$ kubectl get nodes

reveals that the node is not ready yet. The problem is that I didn't install any CNI yet. So go ahead and let's install flannel:

$ kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/v0.11.0/Documentation/kube-flannel.yml

The flannel manifest - among other things - contains DaemonSet definitions for kube-flannel-ds. This pod should start up shortly, and coredns should follow in a minute or so.

The cluster is ready for use. Just let's not forget to start it manually when needed:

$ service docker start
$ service kubelet start

The cluster need a minute or two to start, be patient. When not needed, the cluster can be stopped:

$ service kubelet stop
$ service docker stop

The last command will stop any running containers, so do use it to completely shut the cluster down.