Skip to main content

Configure Kubernetes Control Plane Cluster

1. 🧭 Scope Legend

Use these scope markers throughout this page and the wider Kubernetes CI/CD documentation.

1.COMMON · NO CHANGE NEEDEDConfigure once and reuse across all GitHub organizations and private repositories.
2.CHANGE PER GITHUB ORGRepeat or recreate for every GitHub organization or product.
3.CHANGE PER REPOSITORYRepeat for every private repository inside the GitHub organization.
Shared-cluster rule: this control plane is configured once and reused by Shelvera, FP, and future GitHub organizations. Adding another organization or private repository does not require another Kubernetes cluster, control plane, load balancer, Harbor VM, or cluster IP range.

2. 🧩 Control Plane Inputs

COMMON · NO CHANGE NEEDEDThese are shared cluster-infrastructure values. They are not GitHub organization or repository inputs and may persist using the versioned common storage keys on this page.
Paste the full control plane join command here only after executing kubeadm init. This value must never persist.
Kubernetes APT Repository:
https://pkgs.k8s.io/core:/stable:/v1.35/deb/
kubeadm init Command:
sudo kubeadm init \
  --control-plane-endpoint "ac-cicd-api.aspireclan.com:443" \
  --upload-certs \
  --pod-network-cidr=10.244.0.0/16 \
  --service-cidr=10.96.0.0/12 \
  --v=5

3. 🖥️ VM Specs

COMMON · NO CHANGE NEEDEDUse the same control-plane VM sizing baseline for the shared Kubernetes cluster.
CPU: 2 vCPU
RAM: 8 GB
Disk: 100 GB

Clone the base Ubuntu 24 minimal template.

4. 🧱 Prepare the Base Template VM

COMMON · NO CHANGE NEEDEDPrepare one reusable base template for the shared control-plane nodes.

Login to the VM and complete the base OS preparation.

lsblk
sudo growpart /dev/sda 3
sudo pvresize /dev/sda3
sudo lvextend -l +100%FREE /dev/ubuntu-vg/ubuntu-lv
sudo resize2fs /dev/ubuntu-vg/ubuntu-lv
df -h

HDD resize reference: Resize Ubuntu VM HDD

sudo apt update && sudo apt upgrade -y
sudo apt autoremove
sudo apt update && sudo apt upgrade -y
swapon --show
sudo modprobe overlay
sudo modprobe br_netfilter
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF
sudo sysctl --system
sudo apt install -y containerd
sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml >/dev/null
sudo nano /etc/containerd/config.toml

Only change these kinds of values:

Nano tip: press Ctrl + W to search.
SystemdCgroup:
true
sandbox_image:
registry.k8s.io/pause:3.10.1
sudo systemctl daemon-reload
sudo systemctl enable containerd
sudo systemctl restart containerd
sudo ufw disable
sudo systemctl disable ufw
sudo mkdir -p /etc/apt/keyrings

curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.35/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.35/deb/ /" | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt update
sudo apt install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
sudo systemctl enable kubelet
sudo kubeadm config images pull
sudo systemctl status containerd --no-pager -l
sudo systemctl status kubelet --no-pager -l
swapon --show
cat <<EOF | sudo tee /etc/crictl.yaml
runtime-endpoint: unix:///run/containerd/containerd.sock
image-endpoint: unix:///run/containerd/containerd.sock
timeout: 10
debug: false
EOF
sudo crictl info
sudo swapoff -a
sudo sed -i.bak '/ swap / s/^/#/' /etc/fstab
swapon --show

5. 🧹 Clean the VM Before Templating

COMMON · NO CHANGE NEEDEDClean the shared base image before converting it into the reusable control-plane template.
sudo apt autoremove -y
sudo apt clean
sudo journalctl --vacuum-time=3d
sudo truncate -s 0 /etc/machine-id
sudo rm -f /var/lib/dbus/machine-id
sudo ln -s /etc/machine-id /var/lib/dbus/machine-id

Optional cleanup:

sudo rm -rf /tmp/*
sudo rm -rf /var/tmp/*

Then shut it down:

sudo shutdown -h now

6. 📀 Create Template

COMMON · NO CHANGE NEEDEDCreate one reusable control-plane template for the shared cluster.
Template Name:
tmpl-ac-cicd-cp-00

Create the template only after all Kubernetes base packages and cleanup steps are complete.

7. 🧬 Clone Template for Production Control Plane VM

COMMON · NO CHANGE NEEDEDClone the shared template to create the first control-plane VM; this is not repeated per GitHub organization or repository.
sudo hostnamectl set-hostname ac-cicd-cp-01
hostnamectl

8. 🚀 Bootstrap Primary Control Plane (ac-cicd-cp-01)

COMMON · NO CHANGE NEEDEDBootstrap the primary node once for the shared highly available Kubernetes control plane.

Clone the template and configure the primary control plane VM.

Hostname:
ac-cicd-cp-01
IP:
192.168.8.62

Complete static IP and VM network configuration using: Configure VMs

sudo reboot
swapon --show

It should show nothing. If it shows anything, run:

sudo swapoff -a
sudo sed -i.bak '/ swap / s/^/#/' /etc/fstab
swapon --show
sudo kubeadm init \
  --control-plane-endpoint "ac-cicd-api.aspireclan.com:443" \
  --upload-certs \
  --pod-network-cidr=10.244.0.0/16 \
  --service-cidr=10.96.0.0/12 \
  --v=5

Securely store the output of kubeadm init.

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
curl -LO https://raw.githubusercontent.com/projectcalico/calico/v3.28.2/manifests/calico.yaml
sudo nano calico.yaml

Update the Calico manifest to align with the pod CIDR.

Nano tip: press Ctrl + W to search for CALICO_IPV4POOL_IPIP.
- name: CALICO_IPV4POOL_IPIP
  value: "Always"
- name: CALICO_IPV4POOL_VXLAN
  value: "Never"
- name: CALICO_IPV4POOL_CIDR
  value: "10.244.0.0/16"
kubectl apply -f calico.yaml

kubectl get nodes -o wide
kubectl get pods -n kube-system -o wide
kubectl get ippool default-ipv4-ippool -o yaml
kubectl -n kube-system get cm kubeadm-config -o yaml
kubectl get nodes
kubectl -n kube-system edit configmap coredns

Add this block above the existing .:53 block in the Corefile:

aspireclan.com:53 {
    forward . 192.168.8.4
    cache 30
}
kubectl rollout restart deployment coredns -n kube-system
kubectl rollout status deployment coredns -n kube-system
cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: net-test
  namespace: default
spec:
  restartPolicy: Never
  tolerations:
    - key: "node-role.kubernetes.io/control-plane"
      operator: "Exists"
      effect: "NoSchedule"
  containers:
    - name: net-test
      image: curlimages/curl:8.10.1
      command: ["sh", "-c"]
      args:
        - |
          nslookup harbor.aspireclan.com || true
          echo
          curl -vk --connect-timeout 10 https://harbor.aspireclan.com/v2/ || true
EOF
kubectl get pod net-test -n default -o wide
kubectl logs net-test -n default
kubectl delete pod net-test -n default --ignore-not-found

The Harbor check should return unauthorized 401.

9. 🧯 Enable Firewall on ac-cicd-cp-01 (Post-Bootstrap)

COMMON · NO CHANGE NEEDEDApply the control-plane firewall baseline to the shared primary node after bootstrap.
sudo ufw allow 6443/tcp
sudo ufw allow 2379:2380/tcp
sudo ufw allow 10250/tcp
sudo ufw allow 30000:32767/tcp
sudo ufw allow 179/tcp
sudo ufw allow proto 4 from any to any
sudo ufw allow from 192.168.8.0/22
sudo ufw enable
sudo ufw status verbose

10. 🧬 Join Additional Control Planes

COMMON · NO CHANGE NEEDEDJoin the remaining nodes once to complete the shared highly available control plane.

Clone the Template and create CP-02 & CP-03 control plane VMs.

Repeat the following on the remaining control plane VMs.

CP-02 Hostname:
ac-cicd-cp-02
CP-02 IP:
192.168.8.63
CP-03 Hostname:
ac-cicd-cp-03
CP-03 IP:
192.168.8.64

Complete static IP and VM network configuration using: Configure VMs

sudo reboot
swapon --show

It should show nothing. If it shows anything, run:

sudo swapoff -a
sudo sed -i.bak '/ swap / s/^/#/' /etc/fstab
swapon --show

Run the control plane join command below on each additional control plane node:

sudo kubeadm join ac-cicd-api.aspireclan.com:443 --token aaa.bbbbbbbbbbbbbbbb --discovery-token-ca-cert-hash sha256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx --control-plane --certificate-key yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Verify the cluster after each join:

kubectl get nodes
kubectl get nodes -o wide
kubectl get pods -n kube-system -o wide
kubectl get pods -A -o wide
kubectl get endpoints kubernetes -n default

11. 🧯 Enable Firewall on Additional Control Planes

COMMON · NO CHANGE NEEDEDApply the same shared control-plane firewall baseline to CP-02 and CP-03.
sudo ufw allow 6443/tcp
sudo ufw allow 2379:2380/tcp
sudo ufw allow 10250/tcp
sudo ufw allow 30000:32767/tcp
sudo ufw allow 179/tcp
sudo ufw allow proto 4 from any to any
sudo ufw allow from 192.168.8.0/22
sudo ufw enable
sudo ufw status verbose