Vezpi/Homelab
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs: create README.me

Browse Source
This commit is contained in:
Vezpi
2026-09-12 20:39:00 +00:00
parent d6ae654a80
commit 57c42b94ea
2 changed files with 190 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ansible/kubernetes/README.md
+97
View File
@@ -0,0 +1,97 @@
# Ansible – Kubernetes nodes
Prepares the Kubernetes VM nodes provisioned by the
[`kubernetes_vms` Terraform project](../../terraform/projects/kubernetes_vms/).
## What it does
The `k8s_node` playbook configures each node for `kubeadm`/`kubelet`:
1. **System** – disables swap (runtime + fstab), loads `overlay` and `br_netfilter`,
persists kernel modules, applies the Kubernetes `sysctl` settings.
2. **Packages** – installs prerequisites, adds the official Kubernetes apt repo/key,
installs and holds `kubeadm` and `kubelet`.
3. **containerd** – installs it, generates the default config, enables the
`SystemdCgroup` driver, starts and enables the service.
4. **Firewall** – configures UFW per role: SSH + common Kubernetes ports from the admin
subnet, control-plane ports (`2379:2380`, `10257`, `10259`) on masters, worker ports
(`10256`, `30000:32767`) on workers, default deny incoming.
Group membership (who is a `master` / `worker`) is resolved from the Terraform inventory,
so the role-specific firewall rules apply automatically.
> Cluster bootstrap (`kubeadm init` / `kubeadm join`) is **not** part of this playbook
> and is done manually per the migration roadmap.
## Inventory
The inventory is generated from Terraform state via the `cloud.terraform` inventory
plugin:
```yaml
# inventories/tf_k8s_vms.yml
---
plugin: cloud.terraform.terraform_provider
project_path:
- /tmp/semaphore/project_1/repository_1_template_7/terraform/projects/kubernetes_vms
```
> The `project_path` is a Semaphore runtime checkout path. If you run locally, point it
> at your own `terraform/projects/kubernetes_vms` directory. The same `ansible` provider
> that writes the inventory is used by Terraform, so hosts/groups are always in sync with
> the applied VMs.
## Requirements
Install the collections referenced by `collections/requirements.yml`:
```bash
ansible-galaxy collection install -r collections/requirements.yml
```
- `cloud.terraform` – Terraform inventory plugin
- `community.general` – `ufw` module
## Usage
```bash
# Connectivity check
ansible-playbook -i inventories/tf_k8s_vms.yml ping.yml
# Prepare all Kubernetes nodes
ansible-playbook -i inventories/tf_k8s_vms.yml k8s_node.yml
```
Both playbooks target `all`; the inventory plugin runs from the Terraform project path.
## Role variables
Defined in `roles/k8s_node/defaults/main.yml` (override as needed):
| Variable | Default | Description |
| ------------------------------- | ---------------------------- | -------------------------------------- |
| `k8s_version` | `1.37` | Kubernetes minor version |
| `k8s_repo` | `pkgs.k8s.io` core v<ver> | Kubernetes apt repository |
| `k8s_repo_key` | matching `Release.key` URL | Repository signing key |
| `k8s_packages` | `kubelet`, `kubeadm` | Packages installed and held |
| `k8s_prerequisite_packages` | `apt-transport-https`, `ca-certificates`, `curl`, `gnupg` | Build deps for the repo |
| `k8s_firewall_admin_subnet` | `192.168.66.0/24` | Subnet allowed through UFW |
| `k8s_firewall_ssh_port` | `22` | SSH port |
| `k8s_firewall_common_ports` | `6443`, `10250` | Ports open on all nodes |
| `k8s_firewall_master_ports` | `2379:2380`, `10257`, `10259` | Ports open on masters |
| `k8s_firewall_worker_ports` | `10256`, `30000:32767` | Ports open on workers |
## File reference
| Path | Purpose |
| ---------------------------------------- | ------------------------------------ |
| `k8s_node.yml` | Main playbook (role `k8s_node`) |
| `ping.yml` | Connectivity smoke-test playbook |
| `inventories/tf_k8s_vms.yml` | Terraform-backed inventory |
| `collections/requirements.yml` | Required Ansible collections |
| `roles/k8s_node/tasks/main.yml` | Task dispatcher |
| `roles/k8s_node/tasks/system.yml` | Swap, kernel modules, sysctl |
| `roles/k8s_node/tasks/packages.yml` | Kubernetes repo + kubeadm/kubelet |
| `roles/k8s_node/tasks/containerd.yml` | containerd install/config |
| `roles/k8s_node/tasks/firewall.yml` | Per-role UFW rules |
| `roles/k8s_node/defaults/main.yml` | Role defaults |
terraform/projects/kubernetes_vms/README.md
+93
View File
@@ -0,0 +1,93 @@
# Terraform – Kubernetes VMs
Provisions the Kubernetes node VMs on Proxmox for the homelab Kubernetes cluster.
## Overview
This Terraform project clones the `ubuntu-noble-cloud` template into virtual machines
on the proxmox nodes, following a capacity-aware placement algorithm:
1. **Control plane (master)** VMs are spread one per node, on the nodes with the most
free memory that still fit the role's `ram`.
2. **Pack roles (worker)** are water-filled over the remaining capacity, most-available
node first, without a per-node limit.
Each VM is tagged with the environment (`vm_env`) and registered in a Terraform-backed
Ansible inventory (via the `ansible` provider), so the `k8s_node` playbook can target
them by role group.
## Placement details
- Online Proxmox nodes are ordered by available memory (most available first, node name
as tie-break).
- The control-plane role must fit entirely on distinct nodes with `free_mib >= ram`.
- Workers are packed into residual capacity (`free_mib - ram_per_cp_vm_on_node`),
again most-available first.
- Naming: `kub-<env>-m<NN>` for masters and `kub-<env>-w<NN>` for workers, e.g.
`kub-prod-m01`, `kub-prod-w01`.
Deployment is blocked by `check` blocks when the fleet cannot be placed:
- no online node is available,
- not enough nodes have the free memory required for the control-plane role,
- residual capacity cannot fit all worker VMs.
## Prerequisites
- Proxmox with:
- a cloneable template named `ubuntu-noble-cloud` (from the `pve_vm` module),
- a `ceph-workload` datastore,
- VLAN bridge `vlan<id>` (default `vlan66`),
- an API token and an SSH user/password with rights to create VMs on the nodes.
- Terraform with the `bpg/proxmox` and `ansible/ansible` providers (init downloads them).
## Input variables
| Variable | Type | Required | Default | Description |
| ---------------------------- | -------- | -------- | ---------------------------- | -------------------------------------------------- |
| `proxmox_endpoint` | string | yes | – | Proxmox API URL |
| `proxmox_api_token` | string | yes | – | Proxmox API token (secret) |
| `proxmox_ssh_username` | string | yes | – | Proxmox SSH user for placement (secret) |
| `proxmox_ssh_password` | string | yes | – | Proxmox SSH password (secret) |
| `vm_env` | string | no | `"prod"` | Environment, part of the VM name and tags |
| `master_count` | number | no | `null` → `vm_attr` default | Number of master VMs |
| `master_cpu` | number | no | `null` → `vm_attr` default | vCPU cores for masters |
| `master_ram` | number | no | `null` → `vm_attr` default | Master RAM in **GiB** (converted to MiB internally)|
| `master_vlan` | number | no | `null` → `vm_attr` default | VLAN for masters |
| `worker_count` | number | no | `null` → `vm_attr` default | Number of worker VMs |
| `worker_cpu` | number | no | `null` → `vm_attr` default | vCPU cores for workers |
| `worker_ram` | number | no | `null` → `vm_attr` default | Worker RAM in **GiB** |
| `worker_vlan` | number | no | `null` → `vm_attr` default | VLAN for workers |
| `vm_attr` | map(… ) | no | see below | Full per-role VM defaults |
The `vm_attr` map holds the fallback attributes used when the corresponding
`master_*`/`worker_*` override is `null`:
```hcl
vm_attr = {
master = { ram = 2048, cpu = 2, vlan = 66, count = 3 }
worker = { ram = 4096, cpu = 2, vlan = 66, count = 3 }
}
```
> `ram` values inside `vm_attr` are MiB; the `master_ram` / `worker_ram` overrides are
> set in GiB and converted internally (`× 1024`). The placement math always works in MiB.
## Usage
```bash
terraform init
terraform plan
terraform apply
```
The `vm_ip` output maps VM name to assigned IP and feeds the Ansible inventory
(`inventories/`) of the `k8s_node` playbook.
## File reference
| File | Purpose |
| ------------ | --------------------------------------------------------- |
| `main.tf` | Placement algorithm, `pve_vm` module wiring, checks |
| `variables.tf` | Clusters, Proxmox credentials, per-role defaults/overrides |
| `provider.tf` | `proxmox` and `ansible` provider configuration |