Skip to main content

Bootstrap DNS Server VM

1. 🧭 Scope Legend

Use these markers to distinguish fixed instructions from values that identify this environment and workflow.

1.COMMON · NO CHANGE NEEDEDRun exactly as documented for the prod-dns-01 bootstrap, validation, health gate, and recovery workflow.
2.CHANGE FOR PROD DNS BUILDReview values that identify prod-dns-01, its network, BIND source, tests, inventory, and playbook paths.

2. 🎯 Purpose and Dependency Rule

COMMON · NO CHANGE NEEDEDBuild prod-dns-01 first from the generic Ubuntu template and establish the mandatory DNS health gate before any DNS-dependent infrastructure is configured.
DOCUMENT REVISION
  Version: 2026-06-16.2
  Purpose: Build prod-dns-01 first from the generic Ubuntu template and establish the mandatory DNS health gate.

DEPENDENCY RULE
  The base template keeps DHCP-provided DNS.
  prod-dns-01 is bootstrapped before all DNS-dependent infrastructure.
  Non-DNS clones are not pinned to 192.168.8.4 until this page reports PASS.

This page deliberately restores the current Git-managed zone declarations and zone files instead of inventing DNS records during disaster recovery.

Status and consistency update — June 16, 2026 · revision 2026-06-16.2: this page now uses the same component placement, scope legend, scoped section headings, full-width content grid, 260px right anchor panel, mobile behavior, input styling, and browser-persisted values as the working tmplt-ub-26-min-base page.

3. 🧱 Bootstrap Inputs

CHANGE FOR PROD DNS BUILDReview the DNS VM identity, trusted network, BIND forwarders, validation records, and repository paths. These inputs are stored locally in this browser.

4. 📚 Source-of-Truth Requirement

COMMON · NO CHANGE NEEDEDCHANGE FOR PROD DNS BUILDRestore the exact Git-managed BIND zone declarations and zone files; do not reconstruct production DNS records during disaster recovery.
envs/prod/dns/ansible/files/prod-dns-01/etc-bind/
├── named.conf.local
└── zones/
    ├── <all current forward-zone files>
    └── <all current reverse-zone files>

Do not invent or reconstruct production records during DR.
The Git-managed files must be captured and reviewed before the old DNS VM is destroyed.

The source bundle must contain the exact working named.conf.local and every forward and reverse zone file. The generated options template controls recursion, forwarding, listeners, and access boundaries.

5. 🖥️ Create and Verify the DNS VM

COMMON · NO CHANGE NEEDEDCHANGE FOR PROD DNS BUILDCreate prod-dns-01 from the approved base template while retaining DHCP-provided DNS for the initial bootstrap.
Approved VM identity
  Template: tmplt-ub-26-min-base
  Hostname: prod-dns-01
  IP:       192.168.8.4
  VM ID:    3156004

Create the VM through the existing dedicated Terraform DNS root. Keep DHCP-provided DNS during this initial bootstrap. Confirm SSH, passwordless sudo, the permanent hostname, the reserved address, and the fail-closed UFW baseline before continuing.

[prod_dns]
prod-dns-01 ansible_host=192.168.8.4 ansible_user=acllc ansible_ssh_private_key_file=~/.ssh/id_ed25519_ansible ansible_python_interpreter=/usr/bin/python3 ansible_ssh_common_args='-o IdentitiesOnly=yes'

6. 📥 Capture the Working BIND Source

COMMON · NO CHANGE NEEDEDCHANGE FOR PROD DNS BUILDCapture the exact working custom BIND files before destroying the existing DNS server, unless the same files are already committed and verified in Git.

Run this on prod-terraform-deploy-02 before destroying the currently working DNS server. Skip only when the same files are already committed and verified in Git.

set -euo pipefail

DNS_HOST="acllc@192.168.8.4"
SOURCE_DIR="envs/prod/dns/ansible/files/prod-dns-01/etc-bind"
WORK_DIR="$(mktemp -d)"
trap 'rm -rf "${WORK_DIR}"' EXIT

mkdir -p "${SOURCE_DIR}/zones"

ssh \
  -i "${HOME}/.ssh/id_ed25519_ansible" \
  -o IdentitiesOnly=yes \
  "${DNS_HOST}" \
  "sudo tar -C /etc/bind -czf - named.conf.local zones" \
  > "${WORK_DIR}/prod-dns-01-bind-custom.tgz"

tar \
  -xzf "${WORK_DIR}/prod-dns-01-bind-custom.tgz" \
  -C "${SOURCE_DIR}"

find "${SOURCE_DIR}" -type d -exec chmod 0755 {} +
find "${SOURCE_DIR}" -type f -exec chmod 0644 {} +


find "${SOURCE_DIR}" -maxdepth 3 -type f -print | sort
sha256sum "${WORK_DIR}/prod-dns-01-bind-custom.tgz"

# Review the captured files, then commit only the DNS configuration and zone data.
git status --short -- "${SOURCE_DIR}"

7. 🧩 Create the Ansible Drop-In Files

COMMON · NO CHANGE NEEDEDCHANGE FOR PROD DNS BUILDCreate the BIND options template, the local-resolver Netplan template, and the bootstrap playbook using the reviewed input values.

7.1 templates/named.conf.options.j2

acl "trusted-lan" {
    127.0.0.1;
    192.168.8.0/22;
};

options {
    directory "/var/cache/bind";

    listen-on port 53 {
        127.0.0.1;
        192.168.8.4;
    };
    listen-on-v6 { none; };

    recursion yes;
    allow-query { trusted-lan; };
    allow-recursion { trusted-lan; };
    allow-query-cache { trusted-lan; };
    allow-transfer { none; };

    forwarders {
        1.1.1.1;
        8.8.8.8;
    };
    forward only;

    dnssec-validation auto;
    auth-nxdomain no;
    minimal-responses yes;
};

7.2 templates/99-bind-local-resolver.yaml.j2

network:
  version: 2
  renderer: networkd
  ethernets:
    ens18:
      dhcp4: true
      dhcp4-overrides:
        use-dns: false
        use-domains: false
      nameservers:
        addresses:
          - 127.0.0.1

7.3 envs/prod/dns/ansible/bootstrap-prod-dns-01.yml

---
- name: Bootstrap and validate prod-dns-01
  hosts: prod_dns
  become: true
  gather_facts: true

  vars:
    dns_hostname: "prod-dns-01"
    dns_ip: "192.168.8.4"
    dns_lan_cidr: "192.168.8.0/22"
    dns_primary_interface: "ens18"
    dns_internal_test_name: "harbor.aspireclan.com"
    dns_internal_test_expected_ipv4: "192.168.8.5"
    dns_public_test_name: "ubuntu.com"
    bind_source_directory: "{{ playbook_dir }}/files/prod-dns-01/etc-bind"

  pre_tasks:
    - name: Assert the play targets only prod-dns-01
      ansible.builtin.assert:
        that:
          - inventory_hostname == dns_hostname
          - ansible_host == dns_ip
        fail_msg: >-
          Refusing to bootstrap DNS because the inventory host or address does
          not match the approved prod-dns-01 identity.

    - name: Verify the VM still has initial bootstrap DNS and a default route
      ansible.builtin.shell: |
        set -euo pipefail
        ip -4 route show default | grep -q '^default '
        getent ahostsv4 ubuntu.com >/dev/null
      args:
        executable: /bin/bash
      changed_when: false

    - name: Set the permanent hostname
      ansible.builtin.hostname:
        name: "{{ dns_hostname }}"

  tasks:
    - name: Install BIND and DNS troubleshooting packages
      ansible.builtin.apt:
        name:
          - bind9
          - bind9-utils
          - dnsutils
        state: present
        update_cache: true

    - name: Verify the Git-managed custom BIND source exists on the controller
      ansible.builtin.assert:
        that:
          - lookup('ansible.builtin.fileglob', bind_source_directory + '/named.conf.local', wantlist=True) | length == 1
          - lookup('ansible.builtin.fileglob', bind_source_directory + '/zones/*', wantlist=True) | length > 0
        fail_msg: >-
          The Git-managed named.conf.local or zone files are missing. Restore
          and review the working DNS source before continuing.
      delegate_to: localhost
      become: false

    - name: Back up the package-created BIND directory before replacement
      ansible.builtin.command:
        argv:
          - tar
          - -C
          - /etc
          - -czf
          - "/root/bind-before-bootstrap.tgz"
          - bind
      args:
        creates: "/root/bind-before-bootstrap.tgz"

    - name: Install the generated BIND options
      ansible.builtin.template:
        src: named.conf.options.j2
        dest: /etc/bind/named.conf.options
        owner: root
        group: bind
        mode: "0644"
      notify: Restart BIND

    - name: Install the Git-managed zone declarations
      ansible.builtin.copy:
        src: "{{ bind_source_directory }}/named.conf.local"
        dest: /etc/bind/named.conf.local
        owner: root
        group: bind
        mode: "0644"
      notify: Restart BIND

    - name: Install all Git-managed forward and reverse zone files
      ansible.builtin.copy:
        src: "{{ bind_source_directory }}/zones/"
        dest: /etc/bind/zones/
        owner: root
        group: bind
        directory_mode: "0755"
        mode: "0644"
      notify: Restart BIND

    - name: Validate the complete BIND configuration and load every primary zone
      ansible.builtin.command: named-checkconf -z /etc/bind/named.conf
      changed_when: false

    - name: Apply any pending BIND restart before health checks
      ansible.builtin.meta: flush_handlers

    - name: Enable and start BIND
      ansible.builtin.systemd_service:
        name: bind9.service
        enabled: true
        state: started

    - name: Allow UDP DNS from the trusted LAN through UFW
      ansible.builtin.command: >-
        ufw allow from {{ dns_lan_cidr }} to any port 53 proto udp
        comment 'Allow LAN DNS UDP'
      register: ufw_dns_udp
      changed_when: "'Rule added' in ufw_dns_udp.stdout"

    - name: Allow TCP DNS from the trusted LAN through UFW
      ansible.builtin.command: >-
        ufw allow from {{ dns_lan_cidr }} to any port 53 proto tcp
        comment 'Allow LAN DNS TCP'
      register: ufw_dns_tcp
      changed_when: "'Rule added' in ufw_dns_tcp.stdout"

    - name: Prove BIND answers the required internal record directly
      ansible.builtin.command:
        argv:
          - dig
          - "@{{ dns_ip }}"
          - "{{ dns_internal_test_name }}"
          - A
          - +short
      register: internal_query
      changed_when: false
      failed_when: dns_internal_test_expected_ipv4 not in internal_query.stdout_lines

    - name: Prove BIND forwards a public query directly
      ansible.builtin.command:
        argv:
          - dig
          - "@{{ dns_ip }}"
          - "{{ dns_public_test_name }}"
          - A
          - +short
      register: public_query
      changed_when: false
      failed_when: public_query.stdout_lines | select('match', '^[0-9]+(\\.[0-9]+){3}$') | list | length == 0

    - name: Configure prod-dns-01 to use its local BIND service
      ansible.builtin.template:
        src: 99-bind-local-resolver.yaml.j2
        dest: /etc/netplan/99-bind-local-resolver.yaml
        owner: root
        group: root
        mode: "0600"
      register: local_resolver_netplan

    - name: Validate Netplan before applying the local resolver
      ansible.builtin.command: netplan generate
      changed_when: false

    - name: Apply the local resolver Netplan configuration
      ansible.builtin.command: netplan apply
      when: local_resolver_netplan.changed

    - name: Flush systemd-resolved caches
      ansible.builtin.command: resolvectl flush-caches
      changed_when: false

    - name: Verify prod-dns-01 uses only its local BIND listener
      ansible.builtin.shell: |
        set -euo pipefail
        actual="$(resolvectl dns '{{ dns_primary_interface }}' | sed -E 's/^Link [0-9]+ \([^)]*\):[[:space:]]*//' | xargs)"
        test "${actual}" = "127.0.0.1"
        getent ahostsv4 "{{ dns_internal_test_name }}" >/dev/null
        getent ahostsv4 "{{ dns_public_test_name }}" >/dev/null
      args:
        executable: /bin/bash
      changed_when: false

    - name: Verify the final listening sockets and firewall state
      ansible.builtin.shell: |
        set -euo pipefail
        ss -lntup | grep -E '(:53[[:space:]])'
        ufw status | grep -q '^Status: active'
        systemctl is-active --quiet bind9.service
      args:
        executable: /bin/bash
      changed_when: false

  handlers:
    - name: Restart BIND
      ansible.builtin.systemd_service:
        name: bind9.service
        state: restarted

8. ▶️ Run the Bootstrap

COMMON · NO CHANGE NEEDEDCHANGE FOR PROD DNS BUILDRun the bootstrap playbook only against the approved prod-dns-01 inventory identity.
set -euo pipefail

ansible-playbook \
  -i "envs/prod/dns/ansible/inventory.ini" \
  "envs/prod/dns/ansible/bootstrap-prod-dns-01.yml" \
  --limit "prod-dns-01" \
  --diff

9. 🧪 Mandatory DNS Health Gate

COMMON · NO CHANGE NEEDEDCHANGE FOR PROD DNS BUILDProve authoritative internal resolution, public forwarding, BIND configuration validity, service health, and the local resolver before continuing.

Run from prod-terraform-deploy-02. Do not create or configure DNS-dependent VMs until this block prints PROD DNS HEALTH GATE: PASS.

set -euo pipefail

DNS_IP="192.168.8.4"
INTERNAL_NAME="harbor.aspireclan.com"
INTERNAL_IP="192.168.8.5"
PUBLIC_NAME="ubuntu.com"

internal_answer="$(dig "@${DNS_IP}" "${INTERNAL_NAME}" A +short)"
printf '%s\n' "${internal_answer}"
printf '%s\n' "${internal_answer}" | grep -Fx "${INTERNAL_IP}"

dig "@${DNS_IP}" "${PUBLIC_NAME}" A +short | grep -Eq '^[0-9]+(\.[0-9]+){3}$'
dig "@${DNS_IP}" aspireclan.com SOA +short | grep -q .

ssh \
  -i "${HOME}/.ssh/id_ed25519_ansible" \
  -o IdentitiesOnly=yes \
  "acllc@${DNS_IP}" \
  "set -e; hostname; systemctl is-active bind9.service; resolvectl dns ens18; sudo named-checkconf -z /etc/bind/named.conf"

echo 'PROD DNS HEALTH GATE: PASS'

10. 💾 Back Up the Git-Managed DNS Source

COMMON · NO CHANGE NEEDEDCHANGE FOR PROD DNS BUILDCreate and verify an archive of the Git-managed DNS source, then copy it and its checksum to approved off-host storage.
set -euo pipefail

SOURCE_DIR="envs/prod/dns/ansible/files/prod-dns-01/etc-bind"
BACKUP_DIR="${HOME}/secure-infrastructure-backups/prod-dns-01"
STAMP="$(date -u +%Y%m%dT%H%M%SZ)"
ARCHIVE="${BACKUP_DIR}/prod-dns-01-bind-source-${STAMP}.tgz"

mkdir -p "${BACKUP_DIR}"
tar -C "$(dirname "${SOURCE_DIR}")" -czf "${ARCHIVE}" "$(basename "${SOURCE_DIR}")"
sha256sum "${ARCHIVE}" | tee "${ARCHIVE}.sha256"

test -s "${ARCHIVE}"
tar -tzf "${ARCHIVE}" | grep -q 'named.conf.local'
tar -tzf "${ARCHIVE}" | grep -q '/zones/'

echo "Verified DNS source backup: ${ARCHIVE}"

Copy the verified archive and checksum to approved off-host storage. A backup that exists only on prod-dns-01 is not a disaster-recovery backup.

11. 🏁 Finished State

COMMON · NO CHANGE NEEDEDCHANGE FOR PROD DNS BUILDStop here only after prod-dns-01 is healthy and the non-DNS clone resolver handoff is safe to begin.
ACCEPTANCE CHECKPOINT
  VM identity:                 prod-dns-01 at 192.168.8.4
  Generic template source:     tmplt-ub-26-min-base
  BIND service:                active and enabled
  Authoritative zones:         loaded by named-checkconf -z
  Internal health record:      harbor.aspireclan.com -> 192.168.8.5
  Public forwarding:           ubuntu.com resolves through BIND
  LAN DNS firewall:            TCP/UDP 53 allowed from 192.168.8.0/22 only
  DNS server host resolver:    127.0.0.1 only
  Client resolver handoff:     NOT applied here
  Next page:                   configure-internal-dns-resolver-on-cloned-vms.mdx

Official References