K3s is a lightweight Kubernetes distribution by Rancher. It uses k3d to create containerized k3s clusters. With Docker, we can spin up a multi-node k3s cluster on a single machine. K3d helps to run k3s inside a Docker container.
Installation of k3d on Linux
For this guide, I am going to be using Ubuntu 20.04 for any examples. Let’s go ahead to install Docker CE on our server.
Install Docker CE on Ubuntu 20.04
Install dependency packages with the below command
sudo apt update
sudo apt install apt-transport-https ca-certificates curl software-properties-commonAdd docker GPG key and docker repository to your hosts with the following commands:
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable"Update packages and install Docker CE
sudo apt-get update
sudo apt install docker-ceInstalling k3d on Linux
There are a number of options you can use to install k3s. See below installation steps for your particular linux operating system.
Using installation script to get the latest version:
# With wget
wget -q -O - https://raw.githubusercontent.com/rancher/k3d/main/install.sh | bash
# Or with curl
curl -s https://raw.githubusercontent.com/rancher/k3d/main/install.sh | bashInstall on macOS with Homebrew:
$ brew install k3dFor Arch Linux you can use yay:
$ yay -S rancher-k3d-binConfirm installed version
$ k3d version
k3d version v3.3.0
k3s version v1.19.3-k3s3 (default)Once you have installed, you can run k3d help to see what you can do with k3d
$ k3d help
https://k3d.io/
k3d is a wrapper CLI that helps you to easily create k3s clusters inside docker.
Nodes of a k3d cluster are docker containers running a k3s image.
All Nodes of a k3d cluster are part of the same docker network.
Usage:
k3d [flags]
k3d [command]
Available Commands:
cluster Manage cluster(s)
completion Generate completion scripts for [bash, zsh, fish, powershell | psh]
help Help about any command
image Handle container images.
kubeconfig Manage kubeconfig(s)
node Manage node(s)
version Show k3d and default k3s version
Flags:
-h, --help help for k3d
--trace Enable super verbose output (trace logging)
--verbose Enable verbose output (debug logging)
--version Show k3d and default k3s version
Use "k3d [command] --help" for more information about a command.Create a Kubernetes cluster with k3d
To create a cluster named ‘newcluster’ with a single node, run the command below:
$ sudo k3d cluster create newcluster
INFO[0000] Created network 'k3d-newcluster'
INFO[0000] Created volume 'k3d-newcluster-images'
INFO[0001] Creating node 'k3d-newcluster-server-0'
INFO[0004] Pulling image 'docker.io/rancher/k3s:v1.19.3-k3s3'
INFO[0080] Creating LoadBalancer 'k3d-newcluster-serverlb'
INFO[0084] Pulling image 'docker.io/rancher/k3d-proxy:v3.3.0'
INFO[0108] (Optional) Trying to get IP of the docker host and inject it into the cluster as 'host.k3d.internal' for easy access
INFO[0117] Successfully added host record to /etc/hosts in 2/2 nodes and to the CoreDNS ConfigMap
INFO[0117] Cluster 'newcluster' created successfully!
INFO[0119] You can now use it like this:
kubectl cluster-infoNow merge the default config with the new cluster connection details. The command below updates your default kubeconfig and switch the current-context to the new one.
$ sudo k3d kubeconfig merge newcluster --switch-context
/root/.k3d/kubeconfig-newcluster.yamlManaging k3s Cluster
List clusters with k3d
$ sudo k3d cluster list
NAME SERVERS AGENTS LOADBALANCER
newcluster 1/1 0/0 trueList nodes with k3d
$ sudo k3d node list
NAME ROLE CLUSTER STATUS
k3d-newcluster-server-0 server newcluster running
k3d-newcluster-serverlb loadbalancer newcluster runningDelete a cluster with k3d
$ k3d cluster delete newcluster
INFO[0001] Deleting cluster 'newcluster'
INFO[0005] Deleted k3d-newcluster-serverlb
INFO[0008] Deleted k3d-newcluster-server-0
INFO[0008] Deleting cluster network '40d1445aaac90b4aa8c2db5c2547d3bbfa7662941808df0721373050ffea9c18'
INFO[0009] Deleting image volume 'k3d-newcluster-images'
INFO[0009] Removing cluster details from default kubeconfig...
INFO[0009] Removing standalone kubeconfig file (if there is one)...
INFO[0009] Successfully deleted cluster newcluster! Manage k3s Cluster with Kubectl
To be able to use Kubectl in K3s cluster, we need to install Kubectl first.
Install Kubectl on Linux
Run the command below to install the latest version.
curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl"To install a specific version, you can run the command below:
curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.19.0/bin/linux/amd64/kubectlMake the kubectl binary executable.
sudo chmod +x ./kubectlMove the binary in to your PATH.
sudo mv ./kubectl /usr/local/bin/kubectlTo install Kubectl on MacOS, use the below commands
brew install kubectl
# or
brew install kubernetes-cliYou can now run k3s cluster commands with kubectl. For example, get all pods
$ sudo kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system metrics-server-7b4f8b595-26mhr 1/1 Running 0 19m
kube-system coredns-66c464876b-pljsx 1/1 Running 0 19m
kube-system local-path-provisioner-7ff9579c6-jj6mn 1/1 Running 0 19m
kube-system helm-install-traefik-zp9t4 0/1 Completed 0 19m
kube-system svclb-traefik-sdjzp 2/2 Running 0 18m
kube-system traefik-5dd496474-htq9n 1/1 Running 0 18mList nodes
$ sudo kubectl get nodes
NAME STATUS ROLES AGE VERSION
k3d-newcluster-server-0 Ready master 51m v1.19.3+k3s3To create a k3s cluster with more than one worker nodes, run the command below:
sudo k3d cluster create newcluster -a 2Where -a flag specifies the number of agents you need. In this case, I have specified 2. Remember to export config.
Now list clusters
$ sudo k3d cluster list
NAME SERVERS AGENTS LOADBALANCER
newcluster 1/1 2/2 trueList nodes
$ sudo k3d node list
NAME ROLE CLUSTER STATUS
k3d-newcluster-agent-0 agent newcluster running
k3d-newcluster-agent-1 agent newcluster running
k3d-newcluster-server-0 server newcluster running
k3d-newcluster-serverlb loadbalancer newcluster runningConfirm running containers
$ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3848e36c9377 rancher/k3d-proxy:v3.3.0 "/bin/sh -c nginx-pr…" 3 minutes ago Up 3 minutes 80/tcp, 0.0.0.0:40337->6443/tcp k3d-newcluster-serverlb
a81de7560371 rancher/k3s:v1.19.3-k3s3 "/bin/k3s agent" 3 minutes ago Up 3 minutes k3d-newcluster-agent-1
ba22a8a4120c rancher/k3s:v1.19.3-k3s3 "/bin/k3s agent" 3 minutes ago Up 3 minutes k3d-newcluster-agent-0
42650141dfe6 rancher/k3s:v1.19.3-k3s3 "/bin/k3s server --t…" 4 minutes ago Up 4 minutes k3d-newcluster-server-0Get cluster info
$ sudo kubectl cluster-info
Kubernetes master is running at https://0.0.0.0:43467
CoreDNS is running at https://0.0.0.0:43467/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
Metrics-server is running at https://0.0.0.0:43467/api/v1/namespaces/kube-system/services/https:metrics-server:/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.Get all services
$ sudo kubectl -n kube-system get all
NAME READY STATUS RESTARTS AGE
pod/metrics-server-7b4f8b595-qdfsc 1/1 Running 0 33m
pod/helm-install-traefik-ztlj8 0/1 Completed 0 33m
pod/svclb-traefik-vmwr5 2/2 Running 0 31m
pod/svclb-traefik-787b2 2/2 Running 0 31m
pod/svclb-traefik-4l4q2 2/2 Running 0 31m
pod/coredns-66c464876b-v9m94 1/1 Running 0 33m
pod/traefik-5dd496474-sp9hl 1/1 Running 0 31m
pod/local-path-provisioner-7ff9579c6-mnccm 1/1 Running 4 33m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kube-dns ClusterIP 10.43.0.10 <none> 53/UDP,53/TCP,9153/TCP 33m
service/metrics-server ClusterIP 10.43.237.220 <none> 443/TCP 33m
service/traefik-prometheus ClusterIP 10.43.116.77 <none> 9100/TCP 31m
service/traefik LoadBalancer 10.43.178.246 172.21.0.4 80:30172/TCP,443:30736/TCP 31m
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/svclb-traefik 3 3 3 3 3 <none> 31m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/metrics-server 1/1 1 1 33m
deployment.apps/coredns 1/1 1 1 33m
deployment.apps/traefik 1/1 1 1 31m
deployment.apps/local-path-provisioner 1/1 1 1 33m
NAME DESIRED CURRENT READY AGE
replicaset.apps/metrics-server-7b4f8b595 1 1 1 33m
replicaset.apps/coredns-66c464876b 1 1 1 33m
replicaset.apps/traefik-5dd496474 1 1 1 31m
replicaset.apps/local-path-provisioner-7ff9579c6 1 1 1 33m
NAME COMPLETIONS DURATION AGE
job.batch/helm-install-traefik 1/1 94s 33m
Enjoy working with K3s cluster in Docker containers! More interesting guides below:
