grant/fog

Fork 0

Grant Hunter bbd382c667 update install instructions, update talos install version

2025-09-22 20:01:59 -06:00

6.9 KiB

Raw Blame History

Fog

Like the cloud but local.

Required tools

talosctl (https://www.talos.dev/v1.9/talos-guides/install/talosctl/ )
kubectl (https://kubernetes.io/docs/tasks/tools/#kubectl )
helm (https://helm.sh/ )
helmfile (https://helmfile.readthedocs.io/en/latest/ )
bitwarded secrets CLI (https://bitwarden.com/help/secrets-manager-cli/ )
just (https://just.systems/man/en/ )

Talos Cluster

Hardware

HP ProDesk 400 G3 (x3)
Seagate Iron Wolf 2Tb (x2)
Western Digital Red Pro 2Tb
Sabrent HDD Docking Station (3x)
Switch
Ecoflow River 3 Plus
Ecoflow River 3
Router
Modem

HP ProDesk check list

Using refurbished hardware is an adventure in configuration, make sure some things are standard before starting.

Secure boot is off
BIOS password disabled
Enable restore after power outage

Generate config

bws secret get <talos-secret-id> -o json | jq .value --raw-output > talos/secrets.yaml
talosctl gen config fog  https://192.168.1.43:6443 \
         --with-secrets talos/secrets.yaml \
         --config-patch @talos/machine_patch.yaml \
         --config-patch-control-plane @talos/cluster_patch.yaml \
         --config-patch-control-plane @talos/remove_node_label_patch.yaml \
         --config-patch-worker @talos/framework_patch.yaml

Add node to talos cluster

Create an ISO USB

sudo dd if=metal-amd64.iso of=/dev/sdc bs=4M

Start the machine with the USB. Once it has booted REMOVE the USB so that it doesn't get selected as an install location.
Apply config to install the OS to disk

talosctl apply-config [--insecure] -n <ips> controlplane.yaml

Update context

talosctl config node 192.168.1.38 192.168.1.43 192.168.1.39

Kubeconfig

talosctl kubeconfig --nodes 192.168.1.43

Applications

metrics

kubectl apply -f https://raw.githubusercontent.com/alex1989hu/kubelet-serving-cert-approver/main/deploy/standalone-install.yaml
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

Apply and upgrade

check for updates:

bws run 'helmfile deps'

apply changes:

bws run 'helmfile apply'

Extras

Ceph

kubectl label namespace rook-ceph pod-security.kubernetes.io/enforce=privileged

k8up

This requires some CRDs which can be installed via:

kubectl apply -f https://github.com/k8up-io/k8up/releases/download/k8up-4.8.4/k8up-crd.yaml --server-side

Traefik

kubectl create secret generic digitalocean-api-key --from-literal=token=$DIGITAL_OCEAN_API_TOKEN

Postgres

synapse

kubectl create secret generic goatchatca-signingkey --from-literal=signing.key=$GOATCHAT_SYNAPSE_SIGNING_KEY

copy secret from datastore over to goatchat namespace

kubectl get secrets -n datastore postgres-pguser-synapse -o json | jq 'del(.metadata.resourceVersion,.metadata.uid,.metadata.ownerReferences) | .metadata.creationTimestamp=null,.metadata.namespace="goatchat"' | kubectl apply -f -

delete synapse db and recreate with correct locale

PRIMARY_POD=$(kubectl -n datastore get pods --selector='postgres-operator.crunchydata.com/cluster=postgres,postgres-operator.crunchydata.com/role=master' -o jsonpath='{.items[*].metadata.labels.statefulset\.kubernetes\.io/pod-name}')
PGPASSWORD=$(kubectl -n datastore get secrets  "postgres-pguser-grant" -o go-template='{{.data.password | base64decode}}')

kubectl -n datastore exec -it "$PRIMARY_POD" -- psql -c 'DROP DATABASE synapse;'
kubectl -n datastore exec -it "$PRIMARY_POD" -- createdb --encoding=UTF8 --locale=C --template=template0 --owner=synapse synapse

gitea

copy secret from datastore over to gitea namespace

kubectl get secrets -n datastore postgres-pguser-gitea -o json | jq 'del(.metadata.resourceVersion,.metadata.uid,.metadata.ownerReferences) | .metadata.creationTimestamp=null,.metadata.namespace="gitea"' | kubectl apply -f -

delete gitea db and recreate with correct locale

PRIMARY_POD=$(kubectl -n datastore get pods --selector='postgres-operator.crunchydata.com/cluster=postgres,postgres-operator.crunchydata.com/role=master' -o jsonpath='{.items[*].metadata.labels.statefulset\.kubernetes\.io/pod-name}')
PGPASSWORD=$(kubectl -n datastore get secrets  "postgres-pguser-grant" -o go-template='{{.data.password | base64decode}}')

kubectl -n datastore exec -it "$PRIMARY_POD" -- psql -c 'DROP DATABASE gitea;'
kubectl -n datastore exec -it "$PRIMARY_POD" -- createdb --encoding=UTF8 --locale=C --template=template0 --owner=gitea gitea

create runner token

kubectl exec -it <pod> -- gitea --config /data/gitea/conf/app.ini actions generate-runner-token
kubectl create secret generic gitea-runner-token --from-literal=token='<token>'

allow priviledge pods

kubectl label namespace gitea pod-security.kubernetes.io/enforce=privileged

Immich

copy secret from datastore over to immich namespace

kubectl get secrets -n datastore postgres-pguser-immich -o json | jq 'del(.metadata.resourceVersion,.metadata.uid,.metadata.ownerReferences) | .metadata.creationTimestamp=null,.metadata.namespace="immich"' | kubectl apply -f -

Ghost Blogs

I got tired of fighting the docker container so I manually overrode the config.production.json which looks like

{
  "url": <url>,
  "server": {
    "port": 2368,
    "host": "::"
  },
  "mail": {
    "transport": "SMTP",
      "options": {
          "host": "smtp.sendgrid.net",
          "service": "SendGrid",
          "port": 465,
          "secure": true,
          "tls": {
              "requireTLS": true
          }
      }
  },
  "logging": {
    "transports": [
      "file",
      "stdout"
    ]
  },
  "process": "local",
  "paths": {
    "contentPath": "/opt/bitnami/ghost/content"
  },
  "database": {
    "client": "mysql",
    "connection": {
      "host": "mariadb.datastore.svc.cluster.local",
      "port": 3306,
      "user": <user>,
      "password": <password>,
      "database": <database>
    }
  }
}

kngot Create db and user

CREATE DATABASE kgnot_ghost;
grant all privileges on kgnot_ghost.* to kgnot_ghost@'10.%.%.%' identified by '$KGNOT_MYSQL_PASSWORD';

Install app

kubectl create secret generic ghost-kgnot-user-secret --from-literal=ghost-password=$KGNOT_GHOST_USER_PASSWORD
kubectl create secret generic ghost-kgnot-db-secret --from-literal=mysql-password=$KGNOT_MYSQL_PASSWORD
kubectl create secret generic kgnot-smtp-password --from-literal=smtp-password=$KNGOT_SMTP_PASSWORD

53ll Create db and user

CREATE DATABASE 53ll_ghost;
grant all privileges on 53ll_ghost.* to 53ll_ghost@'10.%.%.%' identified by '$GHOST_53LL_MYSQL_PASSWORD';