Things in monospace

Roundup for week ending May 26th

Authored on 2024-05-26

Kubernetes + AWS NLB

Diving into AWS NLB and how it works with Kubernetes and omfg I'm so happy I only have to touch this occasionally. What a fucking mess it is. What it boils down to in our case is the intersection of two things:

The two working together means we have spikes in 5xx class of errors whenever our (very spot instance oriented) cluster has to drain a node that is being interrupted by AWS. Which depending on traffic shape and the phase of the moon, can eat into substantial amount of our error budget.

Anyway, my team doesn't have control over the NLB or cluster-wide modifications, so we're just trying to get the Kubernetes team to notice our wails and give us the tools to fix it ourselves, since they clearly indicated neither switching NLB config nor ingress will happen within observable future i.e. before Kubernetes is outdated and abandoned in favour of Some Future Thing.

Terraform and uptime

Ok so this one is pretty obvious, but you should avoid running a terraform apply on a VPS instance if the plan is going to recreate said instance. So, don't be me and don't bring down your own website for 12 hours without noticing (because of course I don't have monitoring, I am running this website for myself, not you).

This website is hosted on Hetzner and majority of the infra management is done with Terraform through a private Gitlab repo. I also have a number of self-hosted devices that all run gitlab-runner, so it was a matter of time until an event like the one described would occur and yet I was still surprised. The way I found out was seeing a notification from Gitlab informing me of a failed pipeline run in main. It does make me think it's time to setup some kind of monitoring for my infra, but that is Yet Another Project on my List Of Things I Never Have Time For so I think it's ok as is.

One-liners

I'm continuing my journey with mise and slowly getting to the point where things just work. Which is pleasant.

Running Playwright on my late 2015 MBP-turned-Ubuntu laptop turned out to be harder than initially imagined, since Playwright expects only Ubuntu LTS versions. Cue sad trombone noises.

Wrote my first CloudFormation template in ages, hated it. At least it's not exclusively JSON nowadays, so yay?

Discovered nice and clean .zshrc leveraging zinit. Got me thinking...