zvolsky.org > blog > do-lb-ingress-nginx-ip

Log client ip with ingress-nginx behind DO load balancer

2020-07-19

SSL-terminating LB

First we need to know whether our load balancer terminates TLS traffic, or whether traffic just passes through encrypted. This is important because if the LB terminates TLS traffic, it automatically adds the original, pre-proxy source IP to the request's http headers.

I'd like to focus on the SSL-passthrough case, so without any testing, here's the configuration that should suffice to log real IP when using an SSL-terminating LB.

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
data:
  log-format-upstream: '$http_x_forwarded_for $remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" $request_length $request_time [$proxy_upstream_name] [$proxy_alternative_upstream_name] $upstream_addr $upstream_response_length $upstream_response_time $upstream_status $req_id'`
}

SSL-passthrough LB

The SSL-passthrough is different because the load balancer can't add headers to the request. To preserve the original IP, we need to use the PROXY Protocol. To minimize downtime we'll turn it on simultaneously in the load balancer and the ingress controller.

The load balancer's Proxy Protocol flag is in its settings at https://cloud.digitalocean.com/networking/load_balancers/<UUID>/settings. It can also be set via an annotation.

Ingress-nginx exposes the use-proxy-protocol flag.

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
data:
  use-proxy-protocol: "true"`
}

The resource name nginx-configuration is not random; it is passed as an argument to the ingress controller, see ingress-nginx-configuration.

When you apply the above configuration, ingress-nginx automatically configures the nginx realip module, so changing the log format to include a custom header isn't necessary.

References

  • SSH into nginx-ingress-controller to view current configuration
$ kubectl exec -itn ingress-nginx deployment/nginx-ingress-controller -- /bin/bash
bash-5.0$ cat /etc/nginx/nginx.conf | less