feat: deploy k8s vms with Terraform and Ansible(#8)

Reviewed-on: #8

ansible/kubernetes/README.md

@@ -0,0 +1,97 @@
+# Ansible – Kubernetes nodes
+
+Prepares the Kubernetes VM nodes provisioned by the
+[`kubernetes_vms` Terraform project](../../terraform/projects/kubernetes_vms/).
+
+## What it does
+
+The `k8s_node` playbook configures each node for `kubeadm`/`kubelet`:
+
+1. **System** – disables swap (runtime + fstab), loads `overlay` and `br_netfilter`,
+   persists kernel modules, applies the Kubernetes `sysctl` settings.
+2. **Packages** – installs prerequisites, adds the official Kubernetes apt repo/key,
+   installs and holds `kubeadm` and `kubelet`.
+3. **containerd** – installs it, generates the default config, enables the
+   `SystemdCgroup` driver, starts and enables the service.
+4. **Firewall** – configures UFW per role: SSH + common Kubernetes ports from the admin
+   subnet, control-plane ports (`2379:2380`, `10257`, `10259`) on masters, worker ports
+   (`10256`, `30000:32767`) on workers, default deny incoming.
+
+Group membership (who is a `master` / `worker`) is resolved from the Terraform inventory,
+so the role-specific firewall rules apply automatically.
+
+> Cluster bootstrap (`kubeadm init` / `kubeadm join`) is **not** part of this playbook
+> and is done manually per the migration roadmap.
+
+## Inventory
+
+The inventory is generated from Terraform state via the `cloud.terraform` inventory
+plugin:
+
+```yaml
+# inventories/tf_k8s_vms.yml
+---
+plugin: cloud.terraform.terraform_provider
+project_path:
+  - /tmp/semaphore/project_1/repository_1_template_7/terraform/projects/kubernetes_vms
+```
+
+> The `project_path` is a Semaphore runtime checkout path. If you run locally, point it
+> at your own `terraform/projects/kubernetes_vms` directory. The same `ansible` provider
+> that writes the inventory is used by Terraform, so hosts/groups are always in sync with
+> the applied VMs.
+
+## Requirements
+
+Install the collections referenced by `collections/requirements.yml`:
+
+```bash
+ansible-galaxy collection install -r collections/requirements.yml
+```
+
+- `cloud.terraform` – Terraform inventory plugin
+- `community.general` – `ufw` module
+
+## Usage
+
+```bash
+# Connectivity check
+ansible-playbook -i inventories/tf_k8s_vms.yml ping.yml
+
+# Prepare all Kubernetes nodes
+ansible-playbook -i inventories/tf_k8s_vms.yml k8s_node.yml
+```
+
+Both playbooks target `all`; the inventory plugin runs from the Terraform project path.
+
+## Role variables
+
+Defined in `roles/k8s_node/defaults/main.yml` (override as needed):
+
+| Variable                        | Default                      | Description                            |
+| ------------------------------- | ---------------------------- | -------------------------------------- |
+| `k8s_version`                   | `1.37`                       | Kubernetes minor version               |
+| `k8s_repo`                      | `pkgs.k8s.io` core v<ver>    | Kubernetes apt repository              |
+| `k8s_repo_key`                  | matching `Release.key` URL   | Repository signing key                 |
+| `k8s_packages`                  | `kubelet`, `kubeadm`         | Packages installed and held            |
+| `k8s_prerequisite_packages`     | `apt-transport-https`, `ca-certificates`, `curl`, `gnupg` | Build deps for the repo   |
+| `k8s_firewall_admin_subnet`     | `192.168.66.0/24`            | Subnet allowed through UFW             |
+| `k8s_firewall_ssh_port`         | `22`                         | SSH port                               |
+| `k8s_firewall_common_ports`     | `6443`, `10250`              | Ports open on all nodes                |
+| `k8s_firewall_master_ports`     | `2379:2380`, `10257`, `10259` | Ports open on masters                |
+| `k8s_firewall_worker_ports`     | `10256`, `30000:32767`       | Ports open on workers                  |
+
+## File reference
+
+| Path                                     | Purpose                              |
+| ---------------------------------------- | ------------------------------------ |
+| `k8s_node.yml`                           | Main playbook (role `k8s_node`)      |
+| `ping.yml`                               | Connectivity smoke-test playbook     |
+| `inventories/tf_k8s_vms.yml`             | Terraform-backed inventory           |
+| `collections/requirements.yml`           | Required Ansible collections         |
+| `roles/k8s_node/tasks/main.yml`          | Task dispatcher                      |
+| `roles/k8s_node/tasks/system.yml`        | Swap, kernel modules, sysctl         |
+| `roles/k8s_node/tasks/packages.yml`      | Kubernetes repo + kubeadm/kubelet    |
+| `roles/k8s_node/tasks/containerd.yml`    | containerd install/config            |
+| `roles/k8s_node/tasks/firewall.yml`      | Per-role UFW rules                   |
+| `roles/k8s_node/defaults/main.yml`       | Role defaults                        |

ansible/kubernetes/collections/requirements.yml

@@ -0,0 +1,4 @@
+---
+collections:
+  - name: cloud.terraform
+  - name: community.general

ansible/kubernetes/inventories/tf_k8s_vms.yml

@@ -0,0 +1,4 @@
+---
+plugin: cloud.terraform.terraform_provider
+project_path:
+  - /tmp/semaphore/project_1/repository_1_template_7/terraform/projects/kubernetes_vms

ansible/kubernetes/k8s_node.yml

@@ -0,0 +1,6 @@
+---
+- name: Prepare Kubernetes nodes
+  hosts: all
+
+  roles:
+    - role: k8s_node

ansible/kubernetes/ping.yml

@@ -0,0 +1,5 @@
+---
+- name: Prepare Kubernetes nodes
+  hosts: all
+  tasks:
+    - ansible.builtin.ping:

ansible/kubernetes/roles/k8s_node/defaults/main.yml

@@ -0,0 +1,32 @@
+---
+k8s_version: "1.37"
+
+k8s_repo: "https://pkgs.k8s.io/core:/stable:/v{{ k8s_version }}/deb"
+k8s_repo_key: "https://pkgs.k8s.io/core:/stable:/v{{ k8s_version }}/deb/Release.key"
+
+k8s_packages:
+  - kubelet
+  - kubeadm
+
+k8s_prerequisite_packages:
+  - apt-transport-https
+  - ca-certificates
+  - curl
+  - gnupg
+
+k8s_firewall_admin_subnet: "192.168.88.0/24"
+k8s_firewall_ssh_port: "22"
+k8s_firewall_common_ports:
+  - "6443"
+  - "10250"
+k8s_firewall_master_ports:
+  - "2379:2380"
+  - "10257"
+  - "10259"
+k8s_firewall_worker_ports:
+  - "10256"
+  - "30000:32767"
+k8s_firewall_cilium_udp_ports:
+  - "8472"
+k8s_firewall_cilium_tcp_ports:
+  - "4240"

ansible/kubernetes/roles/k8s_node/tasks/containerd.yml

@@ -0,0 +1,34 @@
+---
+- name: Install containerd
+  ansible.builtin.apt:
+    name: containerd
+    state: present
+    update_cache: true
+  become: yes
+
+- name: Create containerd configuration directory
+  ansible.builtin.file:
+    path: /etc/containerd
+    state: directory
+    mode: "0755"
+  become: yes
+
+- name: Generate default containerd configuration
+  ansible.builtin.shell:
+    cmd: containerd config default > /etc/containerd/config.toml
+    creates: /etc/containerd/config.toml
+  become: yes
+
+- name: Enable systemd cgroup driver in containerd
+  ansible.builtin.replace:
+    path: /etc/containerd/config.toml
+    regexp: '^(\s*)SystemdCgroup(\s*=\s*)false'
+    replace: '\1SystemdCgroup\2true'
+  become: yes
+
+- name: Enable and start containerd
+  ansible.builtin.systemd:
+    name: containerd
+    state: restarted
+    enabled: true
+  become: yes

ansible/kubernetes/roles/k8s_node/tasks/firewall.yml

@@ -0,0 +1,62 @@
+---
+- name: Allow SSH from admin subnet
+  community.general.ufw:
+    rule: allow
+    port: "{{ k8s_firewall_ssh_port }}"
+    proto: tcp
+    from_ip: "{{ k8s_firewall_admin_subnet }}"
+  become: yes
+
+- name: Allow common Kubernetes ports from admin subnet
+  community.general.ufw:
+    rule: allow
+    port: "{{ item }}"
+    proto: tcp
+    from_ip: "{{ k8s_firewall_admin_subnet }}"
+  become: yes
+  loop: "{{ k8s_firewall_common_ports }}"
+
+- name: Allow etcd and control plane ports on control-plane nodes
+  community.general.ufw:
+    rule: allow
+    port: "{{ item }}"
+    proto: tcp
+    from_ip: "{{ k8s_firewall_admin_subnet }}"
+  become: yes
+  loop: "{{ k8s_firewall_master_ports }}"
+  when: inventory_hostname in groups['master']
+
+- name: Allow kube-proxy and NodePort ports on worker nodes
+  community.general.ufw:
+    rule: allow
+    port: "{{ item }}"
+    proto: tcp
+    from_ip: "{{ k8s_firewall_admin_subnet }}"
+  become: yes
+  loop: "{{ k8s_firewall_worker_ports }}"
+  when: inventory_hostname in groups['worker']
+
+- name: Allow Cilium VXLAN and health check UDP ports from admin subnet
+  community.general.ufw:
+    rule: allow
+    port: "{{ item }}"
+    proto: udp
+    from_ip: "{{ k8s_firewall_admin_subnet }}"
+  become: yes
+  loop: "{{ k8s_firewall_cilium_udp_ports }}"
+
+- name: Allow Cilium health check TCP ports from admin subnet
+  community.general.ufw:
+    rule: allow
+    port: "{{ item }}"
+    proto: tcp
+    from_ip: "{{ k8s_firewall_admin_subnet }}"
+  become: yes
+  loop: "{{ k8s_firewall_cilium_tcp_ports }}"
+
+- name: Enable ufw and deny incoming by default
+  community.general.ufw:
+    state: enabled
+    policy: deny
+    direction: incoming
+  become: yes

ansible/kubernetes/roles/k8s_node/tasks/main.yml

@@ -0,0 +1,5 @@
+---
+- ansible.builtin.import_tasks: system.yml
+- ansible.builtin.import_tasks: packages.yml
+- ansible.builtin.import_tasks: containerd.yml
+- ansible.builtin.import_tasks: firewall.yml

ansible/kubernetes/roles/k8s_node/tasks/packages.yml

@@ -0,0 +1,36 @@
+---
+- name: Install prerequisite packages
+  ansible.builtin.apt:
+    name: "{{ k8s_prerequisite_packages }}"
+    state: present
+    update_cache: true
+  become: yes
+
+- name: Add Kubernetes apt key
+  ansible.builtin.apt_key:
+    url: "{{ k8s_repo_key }}"
+    keyring: /etc/apt/keyrings/kubernetes-apt-keyring.gpg
+    state: present
+  become: yes
+
+- name: Add Kubernetes apt repository
+  ansible.builtin.apt_repository:
+    repo: "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] {{ k8s_repo }} /"
+    state: present
+    filename: kubernetes
+  become: yes
+
+- name: Install kubelet and kubeadm
+  ansible.builtin.apt:
+    name: "{{ k8s_packages }}"
+    state: present
+    update_cache: true
+  become: yes
+
+- name: Hold kubelet and kubeadm packages
+  ansible.builtin.command:
+    cmd: apt-mark hold {{ item }}
+  loop: "{{ k8s_packages }}"
+  register: _kube_hold
+  become: yes
+  changed_when: "'already set on hold' not in _kube_hold.stdout"

ansible/kubernetes/roles/k8s_node/tasks/system.yml

@@ -0,0 +1,51 @@
+---
+- name: Disable swap
+  ansible.builtin.shell:
+    cmd: swapoff -a
+  become: yes
+  changed_when: false
+
+- name: Comment out swap entries in fstab
+  ansible.builtin.replace:
+    path: /etc/fstab
+    regexp: '^(\s*[^#\s][^\s]*\s+swap\s+.*)$'
+    replace: '# \1'
+  become: yes
+
+- name: Load overlay kernel module
+  ansible.builtin.shell:
+    cmd: modprobe overlay
+  become: yes
+  changed_when: false
+  failed_when: false
+
+- name: Load br_netfilter kernel module
+  ansible.builtin.shell:
+    cmd: modprobe br_netfilter
+  become: yes
+  changed_when: false
+
+- name: Persist kernel modules
+  ansible.builtin.copy:
+    dest: /etc/modules-load.d/k8s.conf
+    content: |
+      overlay
+      br_netfilter
+    mode: "0644"
+  become: yes
+
+- name: Configure kernel network settings
+  ansible.builtin.copy:
+    dest: /etc/sysctl.d/k8s.conf
+    content: |
+      net.bridge.bridge-nf-call-iptables  = 1
+      net.bridge.bridge-nf-call-ip6tables = 1
+      net.ipv4.ip_forward                 = 1
+    mode: "0644"
+  become: yes
+
+- name: Apply kernel network settings
+  ansible.builtin.command:
+    cmd: sysctl --system
+  become: yes
+  changed_when: false