One Hat Cyber Team

Edit File: Running-behind-nginx.md

# Running Netdata behind Nginx

## Intro

[Nginx](https://nginx.org/en/) is an HTTP and reverse proxy server, a mail proxy server, and a generic TCP/UDP proxy server used to host websites and applications of all sizes.

The software is known for its low impact on memory resources, high scalability, and its modular, event-driven architecture which can offer secure, predictable performance.

## Why Nginx

-   By default, Nginx is fast and lightweight out of the box.

-   Nginx is used and useful in cases when you want to access different instances of Netdata from a single server.

-   Password-protect access to Netdata, until distributed authentication is implemented via the Netdata cloud Sign In mechanism.

-   A proxy was necessary to encrypt the communication to Netdata, until v1.16.0, which provided TLS (HTTPS) support.

## Nginx configuration file

All Nginx configurations can be found in the `/etc/nginx/` directory. The main configuration file is `/etc/nginx/nginx.conf`. Website or app-specific configurations can be found in the `/etc/nginx/site-available/` directory.

Configuration options in Nginx are known as directives. Directives are organized into groups known as blocks or contexts. The two terms can be used interchangeably.

Depending on your installation source, you’ll find an example configuration file at `/etc/nginx/conf.d/default.conf` or `etc/nginx/sites-enabled/default`, in some cases you may have to manually create the `sites-available` and `sites-enabled` directories.

You can edit the Nginx configuration file with Nano, Vim or any other text editors you are comfortable with.

After making changes to the configuration files:

-   Test Nginx configuration with `nginx -t`.

-   Restart Nginx to effect the change with `/etc/init.d/nginx restart` or `service nginx restart`.

## Ways to access Netdata via Nginx

### As a virtual host

With this method instead of `SERVER_IP_ADDRESS:19999`, the Netdata dashboard can be accessed via a human-readable URL such as `netdata.example.com` used in the configuration below.

```conf
upstream backend {
    # the Netdata server
    server 127.0.0.1:19999;
    keepalive 1024;
}

server {
    # nginx listens to this
    listen 80;
    # uncomment the line if you want nginx to listen on IPv6 address
    #listen [::]:80;

# the virtual host name of this
    server_name netdata.example.com;

location / {
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Server $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://backend;
        proxy_http_version 1.1;
        proxy_pass_request_headers on;
        proxy_set_header Connection "keep-alive";
        proxy_store off;
    }
}
```

### As a subfolder to an existing virtual host

This method is recommended when Netdata is to be served from a subfolder (or directory). 
In this case, the virtual host `netdata.example.com` already exists and Netdata has to be accessed via `netdata.example.com/netdata/`.

```conf
upstream netdata {
    server 127.0.0.1:19999;
    keepalive 64;
}

server {
    listen 80;
    # uncomment the line if you want nginx to listen on IPv6 address
    #listen [::]:80;

# the virtual host name of this subfolder should be exposed
    #server_name netdata.example.com;

location = /netdata {
        return 301 /netdata/;
    }

location ~ /netdata/(?<ndpath>.*) {
        proxy_redirect off;
        proxy_set_header Host $host;

proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Server $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_http_version 1.1;
        proxy_pass_request_headers on;
        proxy_set_header Connection "keep-alive";
        proxy_store off;
        proxy_pass http://netdata/$ndpath$is_args$args;

gzip on;
        gzip_proxied any;
        gzip_types *;
    }
}
```

### As a subfolder for multiple Netdata servers, via one Nginx

This is the recommended configuration when one Nginx will be used to manage multiple Netdata servers via subfolders.

```conf
upstream backend-server1 {
    server 10.1.1.103:19999;
    keepalive 64;
}
upstream backend-server2 {
    server 10.1.1.104:19999;
    keepalive 64;
}

server {
    listen 80;
    # uncomment the line if you want nginx to listen on IPv6 address
    #listen [::]:80;

# the virtual host name of this subfolder should be exposed
    #server_name netdata.example.com;

location ~ /netdata/(?<behost>.*?)/(?<ndpath>.*) {
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Server $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_http_version 1.1;
        proxy_pass_request_headers on;
        proxy_set_header Connection "keep-alive";
        proxy_store off;
        proxy_pass http://backend-$behost/$ndpath$is_args$args;

gzip on;
        gzip_proxied any;
        gzip_types *;
    }

# make sure there is a trailing slash at the browser
    # or the URLs will be wrong
    location ~ /netdata/(?<behost>.*) {
        return 301 /netdata/$behost/;
    }
}
```

Of course you can add as many backend servers as you like.

Using the above, you access Netdata on the backend servers, like this:

-   `http://netdata.example.com/netdata/server1/` to reach `backend-server1`
-   `http://netdata.example.com/netdata/server2/` to reach `backend-server2`

### Encrypt the communication between Nginx and Netdata

In case Netdata's web server has been [configured to use TLS](https://github.com/netdata/netdata/blob/master/web/server/README.md#enabling-tls-support), it is
necessary to specify inside the Nginx configuration that the final destination is using TLS. To do this, please, append
the following parameters in your `nginx.conf`

```conf
proxy_set_header X-Forwarded-Proto https;
proxy_pass https://localhost:19999;
```

Optionally it is also possible to [enable TLS/SSL on Nginx](http://nginx.org/en/docs/http/configuring_https_servers.html), this way the user will encrypt not only the communication between Nginx and Netdata but also between the user and Nginx.

If Nginx is not configured as described here, you will probably receive the error `SSL_ERROR_RX_RECORD_TOO_LONG`.

### Enable authentication

Create an authentication file to enable basic authentication via Nginx, this secures your Netdata dashboard.

If you don't have an authentication file, you can use the following command:

```sh
printf "yourusername:$(openssl passwd -apr1)" > /etc/nginx/passwords
```

And then enable the authentication inside your server directive:

```conf
server {
    # ...
    auth_basic "Protected";
    auth_basic_user_file passwords;
    # ...
}
```

## Limit direct access to Netdata

If your Nginx is on `localhost`, you can use this to protect your Netdata:

```
[web]
    bind to = 127.0.0.1 ::1
```

You can also use a unix domain socket. This will also provide a faster route between Nginx and Netdata:

```
[web]
    bind to = unix:/var/run/netdata/netdata.sock
```

*note: Netdata v1.8+ support unix domain sockets*

At the Nginx side, use something like this to use the same unix domain socket:

```conf
upstream backend {
    server unix:/var/run/netdata/netdata.sock;
    keepalive 64;
}
```

If your Nginx server is not on localhost, you can set:

```
[web]
    bind to = *
    allow connections from = IP_OF_NGINX_SERVER
```

*note: Netdata v1.9+ support `allow connections from`*

`allow connections from` accepts [Netdata simple patterns](https://github.com/netdata/netdata/blob/master/src/libnetdata/simple_pattern/README.md) to match against the
connection IP address.

## Prevent the double access.log

Nginx logs accesses and Netdata logs them too. You can prevent Netdata from generating its access log, by setting this in `/etc/netdata/netdata.conf`:

```
[global]
      access log = none
```

## Use gzip compression

By default, netdata compresses its responses. You can have nginx do that instead, with the following options in the `location /` block:

```conf
  location / {
		...
		gzip on;
		gzip_proxied any;
		gzip_types *;
	}
```

To disable Netdata's gzip compression, open `netdata.conf` and in the `[web]` section put:

```conf
[web]
    enable gzip compression = no
```

## SELinux

If you get an 502 Bad Gateway error you might check your Nginx error log:

```sh
# cat /var/log/nginx/error.log:
2016/09/09 12:34:05 [crit] 5731#5731: *1 connect() to 127.0.0.1:19999 failed (13: Permission denied) while connecting to upstream, client: 1.2.3.4, server: netdata.example.com, request: "GET / HTTP/2.0", upstream: "http://127.0.0.1:19999/", host: "netdata.example.com"
```

If you see something like the above, chances are high that SELinux prevents nginx from connecting to the backend server. To fix that, just use this policy: `setsebool -P httpd_can_network_connect true`.