k3s custom resolv.conf

Setting up a local k3s just taught me a valuable lesson in networking.

It’s always DNS

And my own networking knowledge gaps. So I wrongly assumed that k3s was using my /etc/resolv.conf - wrong. It uses core-dns defaults (AFAIK) which means my pfSense DNS server which has host overrides for my local cluster was not being respected.

This resulted in my CI (drone and woodpecker, in testing) not being able to communicate with my git server. This makes sense, it cannot resolve the address.

To rectify, I used this process. It is probably a bandaid when I should be using something else. But, it works for now.

echo "nameserver 192.168.1.1" | sudo tee /etc/k3s-resolv.conf

echo 'kubelet-arg:' | sudo tee -a /etc/rancher/k3s/config.yaml
echo '- "resolv-conf=/etc/k3s-resolv.conf"' | sudo tee -a /etc/rancher/k3s/config.yaml

sudo systemctl restart k3s

Then kill the all the coredns pods so that they reload with the new resolv.conf.

kubectl get pod -n kube-system -l k8s-app=kube-dns --no-headers | awk '{print $1}' | xargs -I{} kubectl delete pod -n kube-system {}

Tags:

#kubernetes #coredns #dns