HTTPS is used for almost everything on the internet. Most web pages have moved or started to offer an HTTPS interface as standard. In order for the Smoothwall web filter to see the content of HTTPS connections, the Smoothwall supports HTTPS inspection policies. These policies use a method which is called “Man-in-the-middle attack”, which decrypts and re-encrypts the content on the fly, before the traffic gets to the client that requested it. This is done so the Smoothwall can read the content, and block and report on any type of content that is cause for concern.
To understand how this is done, and to troubleshoot any issues that the HTTPS decryption might cause, you need to understand how the certificates are used by encrypted connections, for example, HTTPS traffic between web servers and clients. This article explains, what certificates authorities and certificate are and how they're used by the Smoothwall to see the contents of traffic streams, which otherwise, would be encrypted and unreadable by the Smoothwall web filter.
Certificates authorities and certificate
Certificates are like the ID-cards of devices and services on a network. An ID card serves to confirm an identity, which is what certificates are used for on web servers. When a HTTPS web server is accessed, the certificate serves as the ID card to assure the visitor that they are visiting the right website and not some fake bank website.
The problem with ID cards is that sometimes they can be faked. Therefore, we need a way to verify that the ID card is valid and has been given to the correct entity. This is where certificate authorities comes in. You could easily put a picture of yourself on a piece of paper and fill it in with all the correct user information but it would not be suitable as a valid means of identification. We need a third party to verify the authenticity of the ID card. Again, that is the purpose of a certificate authority.
When a company wants a certificate to use on their public facing web services, they would contact a certificate authority and buy a certificate from them. The certificate authority validates that the person wanting the certificate is actually the correct person from the right company, and then issue them a certificate, handing it over in a secure manner.
All certificates and certificate authorities contain, separate from the identity and organisational information, a public and a private key. For a certificate authority, these keys are used to issue new certificates and validate that a certificate has been issued by that certificate authority. For a certificate, the public key is used to encrypt the data and the private key is used to decrypt the data. This means that you can hand out the public key to anyone that wants to send you secure data because only the private key that you have is able to decrypt it. As long as the private key is kept secret and safe, which is why it's called the “private” key. You can trust that the data hasn't been seen by anyone else but the intended recipient. This is how data is secured between a user and a HTTPS website; by encrypting it with the public key. Only the server can read this data so your transactions to Amazon or your online bank is safe from prying eyes.
The keys in the certificate are also used to find the certificate authority that created them. This is used to check if the certificate authority is trusted by your browser or device. Browsers have a place where the public keys of certificate authorities are stored, so that the browser can check if the certificates used by the websites that the browser visits can be trusted. If the browser doesn't know the certificate authority that generated the certificates, the browser displays you a warning, telling you that the certificate authority is unknown, distrusted or invalid.
A different type of warning might display if the identity information in the certificate doesn't match with what the browser expects. For example, if we access the Barclays bank website using the address https://bank.barclays.co.uk we should have no issues at all. There's a nice green padlock in the browser bar.
But if we substitute the address with the IP address of the Barclays banking site, rather than the name, we get a browser warning:
This is because the IP address is not listed in the certificate as a valid name for the site.
Those are the two reasons why you would see warnings in your browser.
- The name(s) used as ID in the certificates don't match the name of the service you are accessing.
- The device you are using doesn't trust the certificate authority that created the certificate used by the service you are accessing.
In both cases, your browser might permit you to continue, despite the errors.
There are cases where the browser might not permit you to continue. Some browsers and applications can perform certificate pinning or even have certificates embedded in them and if these clients try to access a service, where the certificate does not match what the client is expecting, then you get an error with no option to continue. You can search for HSTS and certificate pinning on the internet if you would like to know more about these methods.
Smoothwall HTTPS decrypt and inspect
Now we can take another look at the diagram shown at the start of this article and hopefully understand it a little bit better with what this process means when applied to the Smoothwall HTTPS inspection policies.
When the client requests an HTTPS page, the Smoothwall retrieves this page and decrypts it, just like your browser does. With the page decrypted, the Smoothwall web filter can then check the contents. After it has checked the contents, the Smoothwall then re-encrypts the page and sends this newly encrypted page to the end user. In order for the Smoothwall to do this, it needs to create a new certificate for the encryption. We cannot create certificates without a certificate authority, so the Smoothwall generate a self-signed certificate authority at install time. That's the certificate authority that is used to generate the new certificate used to encrypt the page that the user requested. Here is a new diagram detailing a bit more of what certificates and authorities are used during this process.
The client device receives a page that's encrypted with a new certificate created by the Smoothwall self-signed certificate authority. So, to avoid any certificate errors, you need to install the Smoothwall self-signed certificate authority on the client device. While this isn't required for the inspection to work, it's recommended, because the client, otherwise, would get certificate warnings on all websites being inspected.
On a domain joined Windows operating system, this certificate authority can be pushed out using group policies but on non-domain machines and other devices that aren't managed, users often have to install this certificate authority manually. There's a page on the Smoothwall itself which allows the users to download the certificate authority and show instructions on how to install it depending on the device and browser used.
The generic address for this page is:
The page can be downloaded and the instructions can be published on a organisation home page or emailed to users to give them as many options as possible for preparing and installing the HTTPS inspection certificate authority.