K3s is a lightweight Kubernetes distribution by Rancher. It uses k3d to create containerized k3s clusters. With Docker, we can spin up a multi-node k3s cluster on a single machine. K3d helps to run k3s inside a Docker container.

Installation of k3d on Linux

For this guide, I am going to be using Ubuntu 20.04 for any examples. Let’s go ahead to install Docker CE on our server.

Install Docker CE on Ubuntu 20.04

Install dependency packages with the below command

sudo apt update
sudo apt install apt-transport-https ca-certificates curl software-properties-common

Add docker GPG key and docker repository to your hosts with the following commands:

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable"

Update packages and install Docker CE

sudo apt-get update
sudo apt install docker-ce

Installing k3d on Linux

There are a number of options you can use to install k3s. See below installation steps for your particular linux operating system.

Using installation script to get the latest version:

# With wget
wget -q -O - https://raw.githubusercontent.com/rancher/k3d/main/install.sh | bash

# Or with curl
curl -s https://raw.githubusercontent.com/rancher/k3d/main/install.sh | bash

Install on macOS with Homebrew:

$ brew install k3d

For Arch Linux you can use yay:

$ yay -S rancher-k3d-bin

Confirm installed version

$ k3d version
k3d version v3.3.0 
k3s version v1.19.3-k3s3 (default)

Once you have installed, you can run k3d help to see what you can do with k3d

$ k3d help 
https://k3d.io/ 
k3d is a wrapper CLI that helps you to easily create k3s clusters inside docker. 
Nodes of a k3d cluster are docker containers running a k3s image. 
All Nodes of a k3d cluster are part of the same docker network. 

Usage: 
 k3d [flags] 
 k3d [command] 

Available Commands: 
 cluster     Manage cluster(s) 
 completion  Generate completion scripts for [bash, zsh, fish, powershell | psh] 
 help        Help about any command 
 image       Handle container images. 
 kubeconfig  Manage kubeconfig(s) 
 node        Manage node(s) 
 version     Show k3d and default k3s version 

Flags: 
 -h, --help      help for k3d 
     --trace     Enable super verbose output (trace logging) 
     --verbose   Enable verbose output (debug logging) 
     --version   Show k3d and default k3s version 

Use "k3d [command] --help" for more information about a command.

Create a Kubernetes cluster with k3d

To create a cluster named ‘newcluster’ with a single node, run the command below:

$ sudo k3d cluster create newcluster
INFO[0000] Created network 'k3d-newcluster'              
INFO[0000] Created volume 'k3d-newcluster-images'        
INFO[0001] Creating node 'k3d-newcluster-server-0'       
INFO[0004] Pulling image 'docker.io/rancher/k3s:v1.19.3-k3s3'  
INFO[0080] Creating LoadBalancer 'k3d-newcluster-serverlb'  
INFO[0084] Pulling image 'docker.io/rancher/k3d-proxy:v3.3.0'  
INFO[0108] (Optional) Trying to get IP of the docker host and inject it into the cluster as 'host.k3d.internal' for easy access  
INFO[0117] Successfully added host record to /etc/hosts in 2/2 nodes and to the CoreDNS ConfigMap  
INFO[0117] Cluster 'newcluster' created successfully!    
INFO[0119] You can now use it like this:                 
kubectl cluster-info

Now merge the default config with the new cluster connection details. The command below updates your default kubeconfig and switch the current-context to the new one.

$ sudo k3d kubeconfig merge newcluster --switch-context 
/root/.k3d/kubeconfig-newcluster.yaml

Managing k3s Cluster

List clusters with k3d

$ sudo k3d cluster list
NAME         SERVERS   AGENTS   LOADBALANCER 
newcluster   1/1       0/0      true

List nodes with k3d

$ sudo k3d node list
NAME                      ROLE           CLUSTER      STATUS 
k3d-newcluster-server-0   server         newcluster   running 
k3d-newcluster-serverlb   loadbalancer   newcluster   running

Delete a cluster with k3d

$ k3d cluster delete newcluster
INFO[0001] Deleting cluster 'newcluster'                 
INFO[0005] Deleted k3d-newcluster-serverlb               
INFO[0008] Deleted k3d-newcluster-server-0               
INFO[0008] Deleting cluster network '40d1445aaac90b4aa8c2db5c2547d3bbfa7662941808df0721373050ffea9c18'  
INFO[0009] Deleting image volume 'k3d-newcluster-images'  
INFO[0009] Removing cluster details from default kubeconfig...  
INFO[0009] Removing standalone kubeconfig file (if there is one)...  
INFO[0009] Successfully deleted cluster newcluster! 

Manage k3s Cluster with Kubectl

To be able to use Kubectl in K3s cluster, we need to install Kubectl first.

Install Kubectl on Linux

Run the command below to install the latest version.

curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl"

To install a specific version, you can run the command below:

curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.19.0/bin/linux/amd64/kubectl

Make the kubectl binary executable.

sudo chmod +x ./kubectl

Move the binary in to your PATH.

sudo mv ./kubectl /usr/local/bin/kubectl

To install Kubectl on MacOS, use the below commands

brew install kubectl 
# or
brew install kubernetes-cli

You can now run k3s cluster commands with kubectl. For example, get all pods

$ sudo kubectl get pods --all-namespaces     
NAMESPACE     NAME                                     READY   STATUS      RESTARTS   AGE 
kube-system   metrics-server-7b4f8b595-26mhr           1/1     Running     0          19m 
kube-system   coredns-66c464876b-pljsx                 1/1     Running     0          19m 
kube-system   local-path-provisioner-7ff9579c6-jj6mn   1/1     Running     0          19m 
kube-system   helm-install-traefik-zp9t4               0/1     Completed   0          19m 
kube-system   svclb-traefik-sdjzp                      2/2     Running     0          18m 
kube-system   traefik-5dd496474-htq9n                  1/1     Running     0          18m

List nodes

$ sudo kubectl get nodes    
NAME                      STATUS   ROLES    AGE   VERSION 
k3d-newcluster-server-0   Ready    master   51m   v1.19.3+k3s3

To create a k3s cluster with more than one worker nodes, run the command below:

sudo k3d cluster create  newcluster -a 2

Where -a flag specifies the number of agents you need. In this case, I have specified 2. Remember to export config.

Now list clusters

$ sudo k3d cluster list 
NAME         SERVERS   AGENTS   LOADBALANCER 
newcluster   1/1       2/2      true

List nodes

$ sudo k3d node list    
NAME                      ROLE           CLUSTER      STATUS 
k3d-newcluster-agent-0    agent          newcluster   running 
k3d-newcluster-agent-1    agent          newcluster   running 
k3d-newcluster-server-0   server         newcluster   running 
k3d-newcluster-serverlb   loadbalancer   newcluster   running

Confirm running containers

$ sudo docker ps 
CONTAINER ID        IMAGE                      COMMAND                  CREATED             STATUS              PORTS                             NAMES 
3848e36c9377        rancher/k3d-proxy:v3.3.0   "/bin/sh -c nginx-pr…"   3 minutes ago       Up 3 minutes        80/tcp, 0.0.0.0:40337->6443/tcp   k3d-newcluster-serverlb 
a81de7560371        rancher/k3s:v1.19.3-k3s3   "/bin/k3s agent"         3 minutes ago       Up 3 minutes                                          k3d-newcluster-agent-1 
ba22a8a4120c        rancher/k3s:v1.19.3-k3s3   "/bin/k3s agent"         3 minutes ago       Up 3 minutes                                          k3d-newcluster-agent-0 
42650141dfe6        rancher/k3s:v1.19.3-k3s3   "/bin/k3s server --t…"   4 minutes ago       Up 4 minutes                                          k3d-newcluster-server-0

Get cluster info

$ sudo kubectl cluster-info

Kubernetes master is running at https://0.0.0.0:43467 
CoreDNS is running at https://0.0.0.0:43467/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy 
Metrics-server is running at https://0.0.0.0:43467/api/v1/namespaces/kube-system/services/https:metrics-server:/proxy 

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

Get all services

$ sudo kubectl -n kube-system get all 
NAME                                         READY   STATUS      RESTARTS   AGE
pod/metrics-server-7b4f8b595-qdfsc           1/1     Running     0          33m
pod/helm-install-traefik-ztlj8               0/1     Completed   0          33m
pod/svclb-traefik-vmwr5                      2/2     Running     0          31m
pod/svclb-traefik-787b2                      2/2     Running     0          31m
pod/svclb-traefik-4l4q2                      2/2     Running     0          31m
pod/coredns-66c464876b-v9m94                 1/1     Running     0          33m
pod/traefik-5dd496474-sp9hl                  1/1     Running     0          31m
pod/local-path-provisioner-7ff9579c6-mnccm   1/1     Running     4          33m

NAME                         TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
service/kube-dns             ClusterIP      10.43.0.10      <none>        53/UDP,53/TCP,9153/TCP       33m
service/metrics-server       ClusterIP      10.43.237.220   <none>        443/TCP                      33m
service/traefik-prometheus   ClusterIP      10.43.116.77    <none>        9100/TCP                     31m
service/traefik              LoadBalancer   10.43.178.246   172.21.0.4    80:30172/TCP,443:30736/TCP   31m

NAME                           DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
daemonset.apps/svclb-traefik   3         3         3       3            3           <none>          31m

NAME                                     READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/metrics-server           1/1     1            1           33m
deployment.apps/coredns                  1/1     1            1           33m
deployment.apps/traefik                  1/1     1            1           31m
deployment.apps/local-path-provisioner   1/1     1            1           33m

NAME                                               DESIRED   CURRENT   READY   AGE
replicaset.apps/metrics-server-7b4f8b595           1         1         1       33m
replicaset.apps/coredns-66c464876b                 1         1         1       33m
replicaset.apps/traefik-5dd496474                  1         1         1       31m
replicaset.apps/local-path-provisioner-7ff9579c6   1         1         1       33m

NAME                             COMPLETIONS   DURATION   AGE
job.batch/helm-install-traefik   1/1           94s        33m

Enjoy working with K3s cluster in Docker containers! More interesting guides below:

Best Docker Learning Courses:

LEAVE A REPLY

Please enter your comment!
Please enter your name here