SNI Proxy

1. What is a SNI Proxy

revproxy is like a man-in-the-middle between the client and the backend webserver. It gets a request from the client (web browser), makes a new request to the web server, gets a response from the server, and forwards it as a response to the original request from the client. The connection between the client and revproxy is secured by an SSL certificate, and revproxy manages SSL certificates for all the domains that it represents. On the other hand, the connection between revproxy and the backend server doesn’t have to be secure, so no SSL certificate is requested from the backend server.

note

Because it stands in the middle, it is able to read all the traffic between client and the server, and even modify a bit the forwarded request and response, if needed (for example by adding or modifying headers).

In some cases it is required that the HTTPS connection is established between the client and the backend server, and the SSL certificate is managed by the backend server, not by the proxy. The proxy just forwards the HTTPS requests and responses transparently (at the TCP level) between the client and the backend server, without even being able to peek at the traffic (because it is encrypted).

This kind of proxy is usually called a SNI proxy, because it uses the Server Name Indication (SNI) in order to figure out to which backend server it should forward the HTTPS connection.

note

Because the HTTPS connection between the client and the server is encrypted, and the proxy cannot peek into it, it cannot read the header Host of the request. So, it does not know the domain to which the request is sent, and does not know where to forward the request.

Here is where SNI comes to the rescue. It is an extension of the HTTPS protocol, which allows the client to specify the domain name of the request during the TLS/SSL handshake, before the HTTPS connection is established. This allows the proxy to forward the request to the correct backend.

2. Install `sniproxy`

There are many applications that can serve as a SNI proxy, like: haproxy, apache2, nginx, traefic, varnish, etc. We will use sniproxy in a docker-scripts container. Installing it is very easy:

ds pull sniproxy
ds init sniproxy @sniproxy
ds @sniproxy make

We have a revproxy container running already, which is listening on the ports 80 and 443. We have to stop it first, because these ports are needed by sniproxy:

ds @revproxy stop
ds @sniproxy make

docker ps
ls
ls etc/
nano etc/sniproxy.conf

3. Test with simple containers

We are going to install a couple of containers, as shown in the diagram, to which sniproxy is going to forward the HTTPS requests.

3.1 Install simple containers

Let's install a couple of containers that are based on the test app and use the test domains:

ds init sniproxy/test/app1 @test/site1
cd /var/ds/test/site1/
nano settings.sh
ds make

Make sure that settings.sh looks like this, before running ds make:

APP=sniproxy/test/app1
IMAGE=sniproxy-test-app1
CONTAINER=site1
DOMAIN_NAMES="site1.user1.fs.al alias1.user1.fs.al"

Let's install another container for the domain site2.user1.fs.al:

ds init sniproxy/test/app2 @test/site2
cd /var/ds/test/site2/
nano settings.sh
ds make

In this case make sure that settings.sh looks like this:

APP=sniproxy/test/app2
IMAGE=sniproxy-test-app2
CONTAINER=site2
DOMAIN_NAMES=site2.user1.fs.al

3.2 Configure sniproxy

Let's make a minimal configuration for sniproxy:

cd /var/ds/sniproxy/

cat <<EOF > etc/sniproxy.conf
user daemon
pidfile /var/run/sniproxy.pid

listen 0.0.0.0:80 {
    proto http
}

listen 0.0.0.0:443 {
    proto tls
}

table {
    # container: site1
    site1.user1.fs.al  site1
    alias1.user1.fs.al site1

    # container: site2
    site2.user1.fs.al  site2
}
EOF

ds restart

Note that site1 and site2 are the names of the docker containers, but inside the docker network they can be resolved to the IPs of those containers. Since sniproxy is also running in a container, on the same docker network, it is possible to reach these containers by their names.

3.3 Get SSL certificates

cd /var/ds/test/site1/
ds inject get_ssl_cert.sh

cd /var/ds/test/site2/
ds inject get_ssl_cert.sh

cd /opt/docker-scripts/sniproxy/test/
tree
nano app1/inject/get_ssl_cert.sh
nano app2/inject/get_ssl_cert.sh

3.4 Access the test sites

We can access these sites with curl (or by opening them in browser):

curl https://site1.user1.fs.al
curl https://alias1.user1.fs.al
curl https://site2.user1.fs.al

We are again using the option -k, --insecure because the domains that we are using for testing are fake ones. But if you are using some real domains/subdomain that you own, you can get SSL certificates for them like this:

3.5 Log messages

By default sniproxy logs errors and warnings to syslog. We can customize logging by adding these lines to the configuration file:

cd /var/ds/sniproxy/

cat <<EOF >> etc/sniproxy.conf
error_log {
    filename /var/log/sniproxy/error.log
    priority notice
}

access_log {
    filename /var/log/sniproxy/access.log
    priority notice
}
EOF

ds restart

Let's make some more requests and then check the logs:

curl https://site1.user1.fs.al
curl https://alias1.user1.fs.al
curl https://site2.user1.fs.al

ds exec tail /var/log/sniproxy/access.log

3.5 Enable proxy_protocol

If we check the logs on site1 and site2 we will notice that the requests are logged as coming from the IP of sniproxy. That's correct because the TCP requests are forwarded from sniproxy to these containers.

cd /var/ds/test/site1/
ds exec tail /var/log/nginx/access.log

cd /var/ds/test/site2/
ds exec tail /var/log/apache2/access.log

Is it possible to get and log instead the IP of the client that is making the request?

In the case of a Reverse Proxy, it is possible to send the IP of the client in a header (for example X-Forwarded-For) and the container can get and use this IP as the real IP of the client. However, in the case of a SNI Proxy this is not possible, because the proxy does not make a new connection to the backend, it just forwards the coming connection to it. Because this connection is encrypted, the proxy is not able to peek to the headers or the content of the HTTPS traffic between the client and the server, and neither can modify them.

For this reason, the "Proxy Protocol" was invented, as a kind of addition in front of the HTTPS protocol. It allows the proxy to pass to the backend server information about the real IP of the client, etc.

It has to be enabled both on the proxy and on the backend servers. If it is activated only on the proxy, the backend server will get confused, because it is expecting plain HTTPS and it gets something else instead. And vice-versa.

To enable the Proxy Protocol on sniproxy, add proxy_protocol on etc/sniproxy.conf, like this:

table {
    # container: site1
    site1.user1.fs.al  site1  proxy_protocol
    alias1.user1.fs.al site1  proxy_protocol

    # container: site2
    site2.user1.fs.al  site2  proxy_protocol
}

Then restart it: ds restart

On the containers, enable proxy_protocol like this:

cd /var/ds/test/site1/
ds inject enable_proxy_protocol.sh

cd /var/ds/test/site2/
ds inject enable_proxy_protocol.sh

cd /opt/docker-scripts/sniproxy/test/
nano app1/inject/enable_proxy_protocol.sh
nano app2/inject/enable_proxy_protocol.sh

Let's check again the logs:

curl https://site1.user1.fs.al
curl https://site2.user1.fs.al

cd /var/ds/test/site1/
ds exec tail /var/log/nginx/access.log

cd /var/ds/test/site2/
ds exec tail /var/log/apache2/access.log

4. Chaining proxies

It is also possible to place a revproxy behind a sniproxy, as shown in these diagrams:

4.1 Configure sniproxy

Edit the configuration of sniproxy and append these lines to the table:

table {
    # . . . . .
    
    # container: revproxy
    site3.user1.fs.al  revproxy  proxy_protocol
    site4.user1.fs.al  revproxy  proxy_protocol
}

Then restart it: ds restart

4.2 Start `revproxy`

We stopped revproxy before starting sniproxy, because they cannot use the ports 80 and 443 at the same time. Now let's start revproxy again, but before starting it we have to disable these ports in its configuration:

cd /var/ds/revproxy/
nano settings.sh

Comment out PORTS and uncomment ENABLE_PROXY_PROTOCOL=true:

#PORTS="80:80 443:443"

ENABLE_PROXY_PROTOCOL=true

Then rebuild the container with: `ds make

4.3 Install site3 and site4

Install site3:

ds init revproxy/test/app1 @test/site3
cd /var/ds/test/site3/
nano settings.sh

Set DOMAIN=site3.user1.fs.al.

ds make
ds inject set_real_ip.sh

Install site4:

ds init revproxy/test/app2 @test/site4
cd /var/ds/test/site4/
nano settings.sh

Set DOMAIN=site4.user1.fs.al.

ds make
ds inject set_real_ip.sh

4.4 Check them

Access them with curl:

curl https://site3.user1.fs.al
curl https://site4.user1.fs.al

Check the logs:

cd /var/ds/sniproxy/
ds exec tail /var/log/sniproxy/access.log

cd /var/ds/revproxy/
ds exec tail \
   /var/log/nginx/site3.user1.fs.al-access.log
ds exec tail \
   /var/log/nginx/site4.user1.fs.al-access.log

cd /var/ds/test/site3/
ds exec tail /var/log/nginx/access.log

cd /var/ds/test/site4/
ds exec tail /var/log/apache2/access.log

4.5 Make `revproxy` the default backend

We can now access the sites site3 and site4, which are behind revproxy, but what about the other sites/domains that are behind revproxy? For example NextCloud, WordPress, Moodle, etc. If we try to access them right now, we will notice that they are not accessible.

We can add a matching rule for each of them, on the configuration of sniproxy, similar to the rules for site3 and site4. However this is not very convenient. A better approach, in our case, is to add a default rule for each of them.

On the configuration of sniproxy replace the rules for site3 and site4 with a rule like this .* revproxy proxy_protocol:

cd /var/ds/sniproxy/
nano etc/sniproxy.conf
ds restart

The first part of the rule is a regular expression, and the pattern .* will match any domain name that has not been matched by the other rules.

Check that now we can access all the test sites (https://site1.user1.fs.al, https://site2.user1.fs.al, https://site3.user1.fs.al, https://site4.user1.fs.al) and all the other sites as well (https://cloud.user1.fs.al, https://vclab.user1.fs.al, https://edu.user1.fs.al).

5. Clean up

Let's remove the test sites:

for site in site{1..4}; do
    cd /var/ds/test/$site/
    ds remove
done

cd /var/ds/
rm -rf test/

cd revproxy/
ds domains-rm site3.user1.fs.al
ds domains-rm site4.user1.fs.al
ds del-ssl-cert site3.user1.fs.al
ds del-ssl-cert site4.user1.fs.al

We should also remove site1 and site2 from the configuration of sniproxy:

cd /var/ds/sniproxy/
nano etc/sniproxy.conf
ds restart

note

By the way, previously we had used the domains site1.user1.fs.al and site2.user1.fs.al for WordPress sites. Let's check that they work again by trying to open them in browser.

1. What is a SNI Proxy​

2. Install sniproxy​

3. Test with simple containers​

3.1 Install simple containers​

3.2 Configure sniproxy​

3.3 Get SSL certificates​

3.4 Access the test sites​

3.5 Log messages​

3.5 Enable proxy_protocol​

4. Chaining proxies​

4.1 Configure sniproxy​

4.2 Start revproxy​

4.3 Install site3 and site4​

4.4 Check them​

4.5 Make revproxy the default backend​

5. Clean up​

1. What is a SNI Proxy

2. Install `sniproxy`

3. Test with simple containers

3.1 Install simple containers

3.2 Configure sniproxy

3.3 Get SSL certificates

3.4 Access the test sites

3.5 Log messages

3.5 Enable proxy_protocol

4. Chaining proxies

4.1 Configure sniproxy

4.2 Start `revproxy`

4.3 Install site3 and site4

4.4 Check them

4.5 Make `revproxy` the default backend

5. Clean up