Install Ingress Nginx as an Ingress Controller in K3s and Retrieve the Source IP Address as a Request Header
2025-03-09 15:43 (2 days ago)
ytyng
This blog post explains how to launch K3s as a single node using Ingress Nginx as the ingress controller to obtain the source IP address in the request headers (x-real-ip, x-forwarded-for). It is a suitable method for setting up a K8s environment on a Raspberry Pi or a similar single-board computer.
Challenges
The challenges addressed here include: - Using Nginx instead of Traefik. - Retrieving the source IP address as headers.
K3s installs Traefik as the default ingress controller, which might not behave as expected if you are used to Nginx due to different configuration settings. By installing Ingress Nginx and configuring it to receive connections from the K3s load balancer (by default, Service LB), it becomes possible to handle the source IP address as request headers with Nginx. Consequently, apps cannot log the source IP address or use it for access control.
Related issue: Getting Real Client IP with k3s · k3s-io/k3s · Discussion #2997
Approach
This practice involves launching Ingress Nginx as a DaemonSet instead of Deployment + Service, binding ports to HostPort 80 and 443, so packets to the host machine are directly forwarded to Ingress Nginx. This approach is similar to the default Ingress Nginx of MicroK8s, resulting in similar usage. Note that this setup cannot make Ingress Nginx itself redundant and has permission-related issues. It is also unsuitable for multi-node operations. Ultimately, a single-node cluster reminiscent of a more compact version of MicroK8s is achieved.
Checking Machine State
$ cat /proc/cgroupsIf the enabled for memory is not 1, adjustments are needed. When starting a Raspberry Pi with Raspbian, it should default to 0. In that case, add cgroup_memory=1 cgroup_enable=memory to the end of /boot/firmware/cmdline.txt and reboot the machine.
Installing k3s
Reference: Quick Start Guide | K3s
$ curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--disable=traefik --disable=servicelb" sh -s - --write-kubeconfig-mode 644Disabling traefik and servicelb, install K3s.
Verifying Startup Status
$ k3s kubectl get all --all-namespacesWithin about a minute, three pods will start up.
Installing Ingress Nginx
Checking the Latest Version of Ingress Nginx
Visit https://github.com/kubernetes/ingress-nginx/releases to check the latest version.
Installation
Use the manifest of the version number checked earlier:
$ k3s kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.12.0/deploy/static/provider/baremetal/deploy.yamlVerifying Startup Status
$ k3s kubectl get all --all-namespacesThe Service and Deployment for ingress-nginx will be created. Verify with:
$ curl https://127.0.0.1:32508 --insecureIt should return a 404 response, confirming the setup.
Deleting Ingress Nginx Deployment and Service
$ k3s kubectl delete service -n ingress-nginx ingress-nginx-controller
$ k3s kubectl delete deployment -n ingress-nginx ingress-nginx-controllerCreating DaemonSet
Modify the manifest to create a DaemonSet and save it.
Changes Made
- Change
kindto DaemonSet. - Remove
spec.strategy. - Add
spec.template.spec.hostNetwork: true. - Add
hostPorttospec.template.spec.containers.ports.
Apply the Manifest
$ sudo k3s kubectl apply -f ingress-nginx-daemonset.yamlConfirm
$ k3s kubectl get all --all-namespacesVerify:
$ curl 'https://127.0.0.1' --insecureIt should now listen on :443.
This concludes the main topic. Testing with a simple echo server and sending an HTTP request will confirm that x-real-ip and x-forwarded-for contain the remote client IP address as expected.
Additional Information
Obtaining kubeconfig
$ cat /etc/rancher/k3s/k3s.yamlIf using remotely, change server: https://127.0.0.1:6443 to the external IP address.
Uninstalling K3s
$ /usr/local/bin/k3s-uninstall.shBe cautious as it removes everything without confirmation.
We look forward to discussing your development needs.
Comments