Homelab
Homelab⌗
First, some explaination! Homelab is what I usually call my experimentation lab, where I try out new technologies, host some fun sites, and test things. I learned about virtual machines, networking, containerization, kubernetes, gitops, and more.
History⌗
Minecraft servers are what got me into running and maintaining virtual machines, and they’ve stayed an important goal with any setup I have. Originally, I used Azure to learn about the cloud, using it to host Minecraft servers and a few other game servers as well.
Eventually I got a used HP ProLiant DL360P on Amazon. It has 64GB of Memory, a raid array with 8TB of storage, and 2 CPUs. This server was a really fun project to work on, since I got to use my novice woodworking skills to build a server cabinet. I used a set of computer fans and wired the power so that air will flow in from the bottom, and out the top.
I started out just writing bash scripts, running modded and vanilla Minecraft servers, and then I started messing around with Docker.
This was my first real interaction with Docker, and I ended up setting up a pretty nice Docker compose project.
I used Traefik as a reverse proxy to make TLS and subdomains work nicely, since everything was hosted on the same IP.
This way, I could use outline.reeve.dev for my notes, wiki.reeve.dev for Bookstack (an older notes site I used) and a few more.
Eventually, the heat it produced was too much for my room. I moved it out to the garage, but it was still using a lot of power. After this I considered moving to EKS since I had experience with that by this time at work, but that was going to be expensive to run. I ended up learning that Oracle has a cloud platform with quite a generous free tier, and since then I’ve set up a pretty nice cluster with Traefik (again).
Oracle Cloud⌗
In my cluster, I use the app of apps pattern. This means, once ArgoCD is set up, I have a root app that tracks itself and a others. The other apps includes ArgoCD, meaning it will manage and update itself.
Below you can see the root application:
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: root
namespace: argocd
spec:
project: default
source:
path: apps/
repoURL: https://gitlab.com/reeve567-k8s-homelab/io-argocd-environment.git
targetRevision: HEAD
destination:
server: https://kubernetes.default.svc
namespace: argocd
syncPolicy:
automated:
prune: true
selfHeal: true
ArgoCD:
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: argo-cd
namespace: argocd
spec:
destination:
namespace: argocd
server: https://kubernetes.default.svc
project: default
source:
path: charts/argo-cd
repoURL: https://gitlab.com/reeve567-k8s-homelab/io-argocd-environment.git
targetRevision: HEAD
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
And the main ApplicationSet for everything else:
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: root-appset
namespace: argocd
spec:
goTemplate: true
generators:
- git:
repoURL: https://gitlab.com/reeve567-k8s-homelab/io-argocd-environment.git
revision: HEAD
directories:
- path: charts/*
- path: charts/argo-cd
exclude: true
template:
metadata:
name: "{{ .path.basename }}"
namespace: argocd
annotations:
argocd-image-updater.argoproj.io/image-list: registry.gitlab.com/reeve567/zaylith,registry.gitlab.com/reeve567/strapi,registry.gitlab.com/reeve567/portfolio-hugo
argocd-image-updater.argoproj.io/write-back-method: git
argocd-image-updater.argoproj.io/git-branch: master
argocd-image-updater.argoproj.io/registry.gitlab.com.pullsecret: argocd/gitlab-secret
spec:
project: default
source:
repoURL: https://gitlab.com/reeve567-k8s-homelab/io-argocd-environment.git
targetRevision: HEAD
path: "{{ .path.path }}"
destination:
server: https://kubernetes.default.svc
namespace: "{{ .path.basename }}"
syncPolicy:
syncOptions:
- CreateNamespace=true
automated:
prune: true
selfHeal: true
Project Structure⌗
If you look closely here, you’ll notice that these are all pointing to the same repo, which is just how I’ve set it up. You could easily have each Application in it’s own repo, but that didn’t really make sense for my setup.
My folder structure is like this:
apps/
kustomization.yaml
resources/
argo-cd.yaml
root-set.yaml
root.yaml
charts/
.coder/
.couchdb/
.kubernetes-dashboard/
.outline/
...
argo-cd/
cert-manager/
cnpg/
...
portfolio/
Where anything that starts with a . is ignored by the root-set. These are effectively my disabled applications.
What’s also neat, is I’m using a mix of kustomization and Helm. For my Applications and ApplicationSets, kustomization, and then all the actual Applications are just Helm.
So for example my portfolio:
/charts
...
portfolio/
templates/
deployment.yaml
ingress.yaml
service.yaml
Chart.yaml
values.yaml
Using Traefik and cert-manager⌗
In order to avoid my cluster costing anything, I need to use the a specific kind of Loadbalancer on Oracle that doesn’t cost anything. This Loadbalancer basically just sends the traffic to my cluster rather than terminating it and acting as a middle man. This means that Oracle won’t deal with TLS, or directing traffic for me. That’s where these two come in. The initial setup isn’t too difficult, it just means making sure you have all the right settings for ports in Traefik, and making sure those services get tied to the right kind of Loadbalancer.
Continuing my portfolio project for example, after the initial setup, I just have to use the CustomResources that Traefik and cert-manager provide.
And then I just have some settings for the ingress.yaml that I use in the values.yaml. Here’s a little show of that since it’s a little different.
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: {{ .Chart.Name }}-domain-tls
spec:
dnsNames:
- reeve.dev
secretName: {{ .Chart.Name }}-domain-tls
issuerRef:
name: letsencrypt-http
kind: ClusterIssuer
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: {{ .Chart.Name }}-domain-https
spec:
entryPoints:
- websecure
routes:
- match: Host(`reeve.dev`)
kind: Rule
services:
- name: {{ .Values.service.name }}
port: {{ .Values.service.port }}
tls:
secretName: {{ .Chart.Name }}-domain-tls