Configure Load Balancer

1. 🧭 Scope Legend

Use these scope markers throughout this page and the wider Kubernetes CI/CD documentation.

1.COMMON · NO CHANGE NEEDEDConfigure once and reuse across all GitHub organizations and private repositories.

2.CHANGE PER GITHUB ORGRepeat or recreate for every GitHub organization or product.

3.CHANGE PER REPOSITORYRepeat for every private repository inside the GitHub organization.

Shared-cluster rule: this HAProxy load balancer and Kubernetes API VIP are configured once and reused by every GitHub organization and private repository. Adding a product or repository does not require another load balancer, API VIP, control plane, Harbor VM, or cluster IP range.

2. 🧩 Generic Organization and Repository Inputs

CHANGE PER GITHUB ORGCHANGE PER REPOSITORYUse one reusable, blank input set. App Short Form and GitHub Organization are organization-scoped; Service / App Name and Target Environment participate in repository-specific derived names. Placeholders are examples only.

Service / App Name

App Short Form

GitHub Organization

Target Environment

Generic inputs are incomplete. Placeholder examples are never treated as entered values. The load-balancer instructions remain usable because the load balancer is shared, while organization and repository references remain symbolic until all four fields are entered.

3. 🧮 Derived Multi-Organization Reference Values

CHANGE PER GITHUB ORGCHANGE PER REPOSITORYThese generated names keep GitHub organization and private-repository resources isolated even though the Kubernetes API load balancer remains shared.

Runner Namespace:

arc-runners-<<APP_SHORT_FORM>>-<<ENVIRONMENT>>

GitHub App Secret:

<<APP_SHORT_FORM>>-arc-ghapp-secret

Harbor Project:

<<APP_SHORT_FORM>>-ci-cd

Harbor Pull Secret:

<<APP_SHORT_FORM>>-harbor-regcred

Harbor CI Credentials Secret:

<<APP_SHORT_FORM>>-harbor-credentials

Organization Working Folder:

~/arc/<<APP_SHORT_FORM>>

Runner Scale Set / Helm Release:

<<SERVICE_OR_APP_NAME>>-<<ENVIRONMENT>>-arc

Runner Values File:

~/arc/<<APP_SHORT_FORM>>/<<SERVICE_OR_APP_NAME>>-<<ENVIRONMENT>>-values.yaml

GitHub Repository URL:

https://github.com/<<GITHUB_ORGANIZATION>>/<<SERVICE_OR_APP_NAME>>

Harbor Image Repository:

harbor.aspireclan.com/<<APP_SHORT_FORM>>-ci-cd/<<SERVICE_OR_APP_NAME>>

CI Robot — Pull + Push:

robot$<<APP_SHORT_FORM>>-ci-cd+<<APP_SHORT_FORM>>-github-ci-cd-robot-01

Runtime Robot — Pull Only:

robot$<<APP_SHORT_FORM>>-ci-cd+<<APP_SHORT_FORM>>-runtime-pull-robot-01

Organization-isolated derived values reviewed

4. 🧾 Shared Load Balancer Inputs

COMMON · NO CHANGE NEEDEDThese are shared Kubernetes cluster infrastructure values and may persist using versioned common localStorage keys.

Base Ubuntu Template

Load Balancer Template

Production LB Hostname

Load Balancer IP

Kubernetes API Hostname

API Frontend Port

HAProxy Stats Port

Timezone

Trusted Subnet

DNS Zone Name

DNS Zone File

Harbor Hostname

CP-01 Hostname

CP-01 IP

CP-02 Hostname

CP-02 IP

CP-03 Hostname

CP-03 IP

Kubernetes API Endpoint:

ac-cicd-api.aspireclan.com:443

HAProxy Stats URL:

http://192.168.8.61:8404/stats

5. 🖥️ VM Specs

COMMON · NO CHANGE NEEDEDUse this VM sizing baseline for the shared Kubernetes API load balancer.

CPU: 1 vCPU
RAM: 2 GB
Disk: 32 GB

Load balancer VM specs reviewed

6. 🧬 Clone the Base Ubuntu Template

COMMON · NO CHANGE NEEDEDClone the common Ubuntu 24 minimal template once to prepare a dedicated reusable load-balancer template.

Source Template:

tmplt-ub-24-min

Temporary VM / Future Template Name:

tmpl-ac-cicd-lb-00

Clone tmplt-ub-24-min, start the VM, and complete the base configuration below.

Base Ubuntu template cloned for load balancer preparation

7. 💽 Resize Disk and Update the Base OS

COMMON · NO CHANGE NEEDEDPrepare the reusable load-balancer template disk and operating system.

lsblk

sudo growpart /dev/sda 3
sudo pvresize /dev/sda3
sudo lvextend -l +100%FREE /dev/ubuntu-vg/ubuntu-lv
sudo resize2fs /dev/ubuntu-vg/ubuntu-lv
df -h

HDD resize reference: Resize Ubuntu VM HDD

sudo apt update && sudo apt upgrade -y

Load balancer template disk resized and OS updated

8. 📦 Install Load Balancer Utilities

COMMON · NO CHANGE NEEDEDInstall HAProxy and the common troubleshooting utilities in the reusable template.

sudo apt update
sudo apt install -y haproxy curl vim net-tools dnsutils tcpdump jq ca-certificates

sudo apt install -y qemu-guest-agent
sudo systemctl enable --now qemu-guest-agent
sudo systemctl status qemu-guest-agent --no-pager

HAProxy and troubleshooting utilities installed

9. 🕒 Configure Timezone

COMMON · NO CHANGE NEEDEDApply the shared infrastructure timezone baseline to the load-balancer template.

sudo timedatectl set-timezone America/New_York
timedatectl

Load balancer timezone configured

10. 🔐 Configure SSH Authorized Keys

COMMON · NO CHANGE NEEDEDConfigure administrative SSH access in the reusable template without storing private keys or PEM contents in this page.

mkdir -p ~/.ssh
chmod 700 ~/.ssh
nano ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

Sensitive-data rule: never paste or persist a private key, PEM content, password, token, or kubeadm join command in browser localStorage. Only public SSH keys belong in authorized_keys.

SSH authorized keys configured

11. ⚙️ Install and Enable HAProxy in the Template

COMMON · NO CHANGE NEEDEDInstall and enable the shared HAProxy service before creating the reusable VM template.

sudo apt update
sudo apt install -y haproxy

Execute only if automatic package cleanup is required:

sudo apt autoremove

sudo systemctl enable haproxy

sudo systemctl status haproxy

HAProxy installed and enabled in the template

12. 🧹 Clean the VM Before Templating

COMMON · NO CHANGE NEEDEDClean machine-specific state before converting the prepared VM into the reusable load-balancer template.

sudo apt autoremove -y
sudo apt clean
sudo journalctl --vacuum-time=3d
sudo truncate -s 0 /etc/machine-id
sudo rm -f /var/lib/dbus/machine-id
sudo ln -s /etc/machine-id /var/lib/dbus/machine-id

Optional cleanup:

sudo rm -rf /tmp/*
sudo rm -rf /var/tmp/*

Then shut down the VM:

sudo shutdown -h now

Load balancer VM cleaned and shut down

13. 📀 Create the Load Balancer VM Template

COMMON · NO CHANGE NEEDEDCreate one reusable HAProxy load-balancer template for the shared Kubernetes platform.

Template Name:

tmpl-ac-cicd-lb-00

Convert the prepared VM to a Proxmox/VM template only after the base packages, SSH access, HAProxy installation, and cleanup steps are complete.

Load balancer template created

14. 🧬 Clone the Template for the Production Load Balancer

COMMON · NO CHANGE NEEDEDClone the shared load-balancer template once to create the production Kubernetes API load balancer.

Template:

tmpl-ac-cicd-lb-00

Production VM:

ac-cicd-lb-01

Production IP:

192.168.8.61

Production load balancer VM cloned

15. 🌐 Configure Production Hostname and Static IP

COMMON · NO CHANGE NEEDEDConfigure the shared production load-balancer identity and network settings.

sudo hostnamectl set-hostname ac-cicd-lb-01
hostnamectl

Complete static IP and VM network configuration using: Configure VMs

Hostname:

ac-cicd-lb-01

Static IP:

192.168.8.61

sudo reboot

Production hostname and static IP configured

16. 🧯 Configure the Load Balancer Firewall

COMMON · NO CHANGE NEEDEDAllow the shared Kubernetes API frontend and HAProxy statistics endpoint. Restrict the stats endpoint to trusted administrative networks whenever practical.

sudo ufw allow 443/tcp
sudo ufw reload
sudo ufw status

sudo ufw allow from 192.168.8.0/22 to any port 8404 proto tcp
sudo ufw reload
sudo ufw status

Load balancer firewall configured

17. 🗂️ Back Up the HAProxy Configuration

COMMON · NO CHANGE NEEDEDPreserve the default HAProxy configuration before replacing it with the Kubernetes API load-balancing configuration.

sudo cp /etc/haproxy/haproxy.cfg /etc/haproxy/haproxy.cfg.bak

sudo nano /etc/haproxy/haproxy.cfg

HAProxy configuration backed up

18. ⚖️ Configure the Kubernetes API HAProxy Frontend and Backend

COMMON · NO CHANGE NEEDEDConfigure one shared TCP frontend on port 443 and distribute Kubernetes API traffic across the three control-plane nodes.

global
    log /dev/log local0
    log /dev/log local1 notice
    daemon
    user haproxy
    group haproxy
    maxconn 2000

defaults
    log global
    mode tcp
    option tcplog
    timeout connect 10s
    timeout client  1m
    timeout server  1m
    retries 3

frontend k8s_api_frontend
    bind 192.168.8.61:443
    default_backend k8s_api_backend

backend k8s_api_backend
    balance roundrobin
    option tcp-check
    default-server inter 3s fall 3 rise 2
    server ac-cicd-cp-01 192.168.8.62:6443 check
    server ac-cicd-cp-02 192.168.8.63:6443 check
    server ac-cicd-cp-03 192.168.8.64:6443 check

listen stats
    bind 192.168.8.61:8404
    mode http
    stats enable
    stats uri /stats
    stats refresh 10s

HAProxy Kubernetes API configuration written

19. ✅ Validate the HAProxy Configuration

COMMON · NO CHANGE NEEDEDValidate HAProxy syntax before restarting the shared load-balancer service.

sudo haproxy -c -f /etc/haproxy/haproxy.cfg

The expected result is:

Configuration file is valid

HAProxy configuration validation passed

20. 🚀 Restart and Enable HAProxy

COMMON · NO CHANGE NEEDEDStart the shared HAProxy service and ensure it starts automatically after a VM reboot.

sudo systemctl restart haproxy

sudo haproxy -c -f /etc/haproxy/haproxy.cfg

sudo systemctl enable haproxy
sudo systemctl restart haproxy
sudo systemctl status haproxy --no-pager

HAProxy enabled and running

21. 🔎 Verify the API Listener, Backends, and Statistics Endpoint

COMMON · NO CHANGE NEEDEDVerify that HAProxy is listening on the shared Kubernetes API VIP and that each control-plane backend is reachable.

sudo ss -tulpn | grep :443

curl -vk --connect-timeout 10 https://ac-cicd-api.aspireclan.com/version || true

for BACKEND in 192.168.8.62 192.168.8.63 192.168.8.64; do
  echo "Testing ${BACKEND}:6443"
  nc -vz -w 5 "${BACKEND}" 6443
done

Browse to the HAProxy statistics page from a trusted administrative network:

HAProxy Stats URL:

http://192.168.8.61:8404/stats

curl -I http://192.168.8.61:8404/stats

API listener, control-plane backends, and stats endpoint verified

22. 📚 Add the Shared Production DNS Records

COMMON · NO CHANGE NEEDEDAdd the load balancer, Kubernetes API, control-plane, and worker records once to the shared production DNS zone.

Add the following records to /etc/bind/zones/db.aspireclan.com for zone aspireclan.com:

; Load Balancer / API
ac-cicd-api     IN  A       192.168.8.61
ac-cicd-lb-01   IN  A       192.168.8.61

; Control Planes
ac-cicd-cp-01   IN  A       192.168.8.62
ac-cicd-cp-02   IN  A       192.168.8.63
ac-cicd-cp-03   IN  A       192.168.8.64

; Workers
ac-cicd-prod-wk-01 IN A     192.168.8.71
ac-cicd-prod-wk-02 IN A     192.168.8.72
ac-cicd-qa-wk-01   IN A     192.168.8.81
ac-cicd-dev-wk-01  IN A     192.168.8.91

Shared Kubernetes CI/CD DNS records added

23. 🧪 Validate DNS and Verify Client Resolution

COMMON · NO CHANGE NEEDEDValidate and reload the shared DNS zone, then verify the Kubernetes API hostname from a Windows client.

sudo named-checkconf
sudo named-checkzone aspireclan.com /etc/bind/zones/db.aspireclan.com

sudo systemctl restart bind9
sudo systemctl restart named

From a Windows machine Command Prompt:

nslookup ac-cicd-api.aspireclan.com

Test-NetConnection ac-cicd-api.aspireclan.com -Port 443

DNS validation and client resolution completed

24. 🔐 GitHub Free Repository Configuration and Final Verification

COMMON · NO CHANGE NEEDEDCHANGE PER GITHUB ORGCHANGE PER REPOSITORYThe load balancer remains shared. For each GitHub organization, keep one Harbor project and separate CI and runtime robots. For every private repository on GitHub Free, configure Harbor secrets and variables at repository scope.

Harbor isolation reference: use one Harbor project per product/GitHub organization: <<APP_SHORT_FORM>>-ci-cd. The CI robot has Pull + Push; the runtime robot has Pull only. Deployed workloads must never receive the CI push credential.

Private GitHub Repositories

Enter one private repository per line, or separate names with commas or spaces.

Repository Secret:

HARBOR_USERNAME

Repository Secret:

HARBOR_PASSWORD

Repository Variable:

HARBOR_REGISTRY

Repository Variable:

HARBOR_PROJECT

gh auth status

read -s -p "Harbor CI robot secret: " HARBOR_CI_PASSWORD
echo

HARBOR_CI_USERNAME='robot$<<APP_SHORT_FORM>>-ci-cd+<<APP_SHORT_FORM>>-github-ci-cd-robot-01'
HARBOR_REGISTRY='harbor.aspireclan.com'
HARBOR_PROJECT='<<APP_SHORT_FORM>>-ci-cd'

REPOSITORIES=(
  "<<ENTER_PRIVATE_REPOSITORY_NAME>>"
)

for REPOSITORY in "${REPOSITORIES[@]}"; do
  FULL_REPOSITORY='<<GITHUB_ORGANIZATION>>/'"${REPOSITORY}"

  echo "Configuring ${FULL_REPOSITORY}..."

  printf '%s' "${HARBOR_CI_USERNAME}" |     gh secret set HARBOR_USERNAME --repo "${FULL_REPOSITORY}"

  printf '%s' "${HARBOR_CI_PASSWORD}" |     gh secret set HARBOR_PASSWORD --repo "${FULL_REPOSITORY}"

  gh variable set HARBOR_REGISTRY     --repo "${FULL_REPOSITORY}"     --body "${HARBOR_REGISTRY}"

  gh variable set HARBOR_PROJECT     --repo "${FULL_REPOSITORY}"     --body "${HARBOR_PROJECT}"

done

unset HARBOR_CI_PASSWORD
unset HARBOR_CI_USERNAME
unset HARBOR_REGISTRY
unset HARBOR_PROJECT

Repository-level Harbor secrets and variables configured for private-repository

sudo systemctl status haproxy --no-pager
sudo ss -tulpn | grep -E ':443|:8404'
nslookup ac-cicd-api.aspireclan.com
curl -I http://192.168.8.61:8404/stats

The HAProxy load balancer is shared across all GitHub organizations and repositories.
The Kubernetes API hostname resolves to 192.168.8.61.
HAProxy reports all three control-plane API backends as healthy.
Each GitHub organization uses its own Harbor project and robot accounts.
The runtime robot remains pull-only; only the CI robot can push images.
Every private repository uses repository-level GitHub Actions secrets and variables.
No password, token, private key, PEM content, or join command is persisted by this page.

Load balancer and multi-organization reference verification completed