Nginx reverse-proxy configuration and access logs formatting
Let’s say we have an online store where we have different sections mobiles, watches, shoes etc, and we want to distribute the load to display the content from different servers.
In a containerised environment we can run different services on the containers and can configure reverse proxy to serve from the relevant containers.
For the details on topology, docker bridge networking used in this example you can refer my another post. Here we are going to much focus on reverse proxy.
Reverse proxy has several benefits -
SSL Encryption — The expensive resource of the origin server is not being wasted in the encryption and decryption (SSL and TLS), this is being offloaded to the reverse proxy can do that all.
No Redundant — we can maintain servers for specific services and the data for a service is not spread across different servers, which enables the easy upgrade/enhancements of the specific service without disturbing other services.
Attack Protection — The incoming traffic has no direct interaction with the actual servers, those could be in a private network as well. The attackers could only be able to target the reverse proxy server.
Global Server Load Balancing — using reverse proxy, the traffic could be diverted to a regional server on the basis of the location of the client.
Below is my configuration to setup the above, reverse proxy setup
Proxy Server configurations, server 20.20.41.200.
/etc/nginx/nginx.conf , have commented the default configuration by commenting /etc/nginx/modules-enabled/*.conf;
user www-data;
worker_processes auto;
pid /run/nginx.pid;
#include /etc/nginx/modules-enabled/*.conf;events {
worker_connections 768;
# multi_accept on;
}http { ##
# Basic Settings
## sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
# server_tokens off; # server_names_hash_bucket_size 64;
# server_name_in_redirect off; include /etc/nginx/mime.types;
default_type application/octet-stream; ##
# SSL Settings
## ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
ssl_prefer_server_ciphers on; ##
# Logging Settings
## access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log; ##
# Gzip Settings
##
gzip on; # gzip_vary on;
# gzip_proxied any;
# gzip_comp_level 6;
# gzip_buffers 16 8k;
# gzip_http_version 1.1;
# gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript; ##
# Virtual Host Configs
## include /etc/nginx/conf.d/*.conf;
#include /etc/nginx/sites-enabled/*;
}
/etc/nginx/conf.d/default.conf
server { listen 80 default_server;
server_name localhost; #charset koi8-r;
#access_log /var/log/nginx/host.access.log main; location / {
root /usr/share/nginx/html;
index index.html index.htm;
} location /shoe {
proxy_pass http://192.168.2.1;
}
location /watch {
proxy_pass http://192.168.2.2;
} location /mobile {
proxy_pass http://192.168.2.3;
} location /home {
#proxy_set_header HOST $proxy_host;
#proxy_set_header HOST $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://20.20.38.183;
}
}
Configuration for 20.20.41.200/shoe which is basically the container 192.168.2.1.
The server section should be like blow in /etc/nginx/conf.d/default.conf file.
server {
listen 80;
server_name localhost; #charset koi8-r;
#access_log /var/log/nginx/host.access.log main; location /shoe {
root /usr/share/nginx/html;
index index.html;
} #error_page 404 /404.html; # redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
The /etc/nginx/nginx.conf should be used as default unless some custom configurations are not required in the behaviour of the service.
As in above /etc/nginx/conf.d/default.conf we have defined the index file, should be present in the /usr/share/nginx/html/shoe directory, which is index in the above example. because in the redirection in the proxy-server we have not given any pattern so the request to the upstream(192.168.2.1) would be http://192.168.2.1/shoe`
location /shoe {
proxy_pass http://192.168.2.1;
}Now lets say if we have below in the proxy-server
location /shoe {
proxy_pass http://192.168.2.1/brand/model;
}Then the request to the upstream would be http://192.168.2.1/brand/model and the lookup will be for the index.html file in the container 192.168.2.1 at location /use/share/nginx/html/brand/model .
Configuration for 20.20.41.200/home, Now lets look at the final section in the
/etc/nginx/conf.d/default.confin the proxy-server configuration, as we have few additional parameters defined there.
location /home {
#proxy_set_header HOST $proxy_host;
#proxy_set_header HOST $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://20.20.38.183;
}Headers, another important topic to discuss when working on proxies — in the proxy configuration there are some automatic adjustments are doing when it proxies the request. Here we will discuss and important header proxy_set_header and different information provided under this -
HOST
Below are lines from the logs /var/logs/nginx/access.log , respective of the proxy definition.
location /home {
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://20.20.38.183;
}localhost host — 20.20.38.183 http 192.10.18.53, 20.20.41.200 20.20.41.200 [03/May/2020:13:09:29 -0600]”GET /host/ HTTP/1.0" 304 0 “-” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:75.0) Gecko/20100101 Firefox/75.0”location /home {
proxy_set_header HOST $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://20.20.38.183;
}localhost host — 20.20.41.200 http 192.10.18.53, 20.20.41.200 10.149.41.200 [03/May/2020:13:15:05 -0600]”GET /host/ HTTP/1.0" 304 0 “-” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:75.0) Gecko/20100101 Firefox/75.0”
X-Real-IP
The server on which the proxying is happening, is considered as the client ip for the proxying system.
location /home {
proxy_set_header HOST $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://20.20.38.183;
}
20.20.41.200 - - [03/May/2020:13:32:38 -0600] "GET /host/ HTTP/1.0" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:75.0) Gecko/20100101 Firefox/75.0"X-Forward-For
This the ip of the client from where the request is generated, the ip of the proxy server. In the below example the bold section is the ip of the client where i am trying to get the url http://20.20.41.200/home.
location /home {
proxy_set_header HOST $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://20.20.38.183;
}localhost host — 20.20.41.200 http 192.10.18.53, 20.20.41.200 20.20.41.200 [03/May/2020:13:15:05 -0600]”GET /host/ HTTP/1.0" 304 0 “-” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:75.0) Gecko/20100101 Firefox/75.0”
X-Forword-Proto
The protocol which is http in above example.
More information on directive is present at
Below is my sample log format, i used at the server where my /home is present(20.20.38.183).
log_format upstreamlog '$server_name '
'host - $host '
'$scheme $proxy_add_x_forwarded_for '
'$remote_addr [$time_local]'
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent"';References
https://docs.nginx.com/nginx/admin-guide/web-server/reverse-proxy
