11. HOWTOs

11.1. HTTPS-Inspection HOWTO

11.1.1. How TLS Certificates for websites work

The website owner generates a (private) TLS key and a Certificate Signing Request (CSR).

The CSR is then sent to a Certification Authority (CA), which verifies that the owner is the actual owner of the website. This can be done by proving that the owner has technical write access to the site or the site's DNS, or by verifying the identity of the organization running the site using telephone and public databases.

If the verification is successful, the CA signs the CSR and creates a certificate that certifies that the private TLS key actually belongs to the website name and/or organization that owns the domain.

This TLS certificate is then added to the web server configuration, and when a browser accesses the website, it verifies that the TLS certificate presented to the browser is valid for that domain.

To do this, each browser has the certificates of multiple CAs in its trust store. The browser will only accept the certificate if the CA that signed it is in its trust store, otherwise it will warn that the certificate is not valid.

If this check passes, the browser sends a random number encrypted with the server's public key to the server, and both compute a shared secret using the Diffie-Hellman key exchange algorithm. Now server and browser can communicate, but no one else can break that communication because it's encrypted between them.

11.1.2. How HTTPS inspection works

When we try to inspect HTTPS traffic, we have to break the TLS encryption between browser and web server without being the browser or the web server. This is exactly what TLS is designed to prevent, because it's a man-in-the-middle attack.

To do this, Privoxy uses its own (private) CA (let's call it "Privoxy CA"), which needs to be added to the trust store of every single browser that you want to use with Privoxy and HTTPS inspection.

Privoxy then breaks the connection between browser and webserver by acting as a browser/client when talking to the webserver (including checking the webserver's TLS certificate against it's own trust store). Now Privoxy can read and modify the traffic from the webserver.

On the other hand, Privoxy itself encrypts the traffic it sends to the browser using an on the fly self-created TLS server certificate that is signed by Privoxy CA.

11.1.3. What happens, if the original certificate is invalid?

If Privoxy detects that a TLS certificate is invalid, because it's expired, doesn't match the hostname, is self-signed, or similar, Privoxy will block the requests and return an error message explaining the problem to prevent the user/browser from communicating over an insecure channel.

To test this behavior, just go to https://badssl.com/

11.1.4. HTTPS inspection prerequisites

HTTPS inspection in Privoxy can only be used, if Privoxy is built with FEATURE_HTTPS_INSPECTION. You can check if this feature is enabled at http://config.privoxy.org/show-status in the "Conditional #defines" section.

If the feature is not enabled, you may need to build Privoxy from source to enable it. FEATURE_HTTPS_INSPECTION relies on a TLS library. You can choose between LibreSSL, MbedTLS, OpenSSL and wolfSSL. Which TLS library is the best choice depends on various factors including operating system, cpu and license preference so there isn't any recommendation here.

After installing the development libraries for the TLS library you want to use, you can run ./configure with the --with-openssl for LibreSSL and OpenSSL or --with-mbedtls or --with-wolfssl option.

Check the output of ./configure, it must contain one of these the following two lines, otherwise HTTPS inspection will not work:

  configure: Detected OpenSSL. Enabling https inspection.
  configure: Detected mbedTLS. Enabling https inspection.
  configure: Detected wolfSSL. Enabling https inspection.

If you do not find any of these lines, the output of ./configure will tell you what went wrong.

You should then proceed with the source install. Finally, check the FEATURE_HTTPS_INSPECTION status in http://config.privoxy.org/show-status again.

11.1.5. Configuring HTTPS inspection in Privoxy

First, you need to create the private key and certificate for the "Privoxy CA". This can be done using openssl with the following command:

  openssl req -new -x509 -extensions v3_ca -keyout privoxy.pem -out privoxy.crt -days 3650

In this example, a CA validity of 10 years (3650 days) is defined. You should set the appropriate validity period based on your needs. A shorter validity makes your system more secure (it doesn't hurt that long if the key gets lost to an attacker), but if the certificate expires before you have replaced it with a new one in Privoxy and in all browsers, the communication will fail.

During key generation you will be asked to provide a "PEM pass phrase". This passphrase will appear in the Privoxy config CGI, so don't reuse it elsewhere!

Then you will be asked for Country Name, State/Province, Locality, Orginzation Name, Common Name, and Email Address. You should fill in some useful data here, because these entries will be shown by the browser as "Issuer Name" when you inspect a certificate from an https-inspection site. Especially the "Common Name" will be shown as the name of your CA, so it's good if you (and other users of your Privoxy instance) are able to identify this CA.

Copy the private key (privoxy.pem) and the CA certificate (privoxy.crt) into the ca-directory (defined in config).

Make sure that the private key (privoxy.pem in the example above) is only accessible to the user running Privoxy (usually named "privoxy"):

  chmod 600 privoxy.pem
  chown privoxy privoxy.pem

Now customize your Privoxy configuration:

  ca-directory /etc/privoxy/CA # read-only
  ca-cert-file privoxy.crt     # in ca-directory
  ca-key-file privoxy.pem      # in ca-directory
  ca-password passphrasefromabove
  certificate-directory /var/lib/privoxy/certs
  trusted-cas-file /etc/ssl/certs/ca-certificates.crt

certificate-directory contains the (on the fly) created webserver keys and certificates. It should only be readable by the privoxy user only:

  chown privoxy /var/lib/privoxy/certs
  chmod 700 /var/lib/privoxy/certs

trusted-cas-file is the trust store containing the certificates of all CAs that should be accepted. Each browser comes with it's own trust store. Most Unix systems also ship with a truststore. Debian ships it's truststore in /etc/ssl/certs/ca-certificates.crt, which is installed by the ca-certificates package and can be updated using update-ca-certificates(8). Alternatively, such a file (extracted from Mozilla) can be downloaded from https://curl.se/docs/caextract.html.

11.1.6. Browser configuration

As mentioned earlier, each browser you use must now trust the newly created Privoxy CA certificate (privoxy.crt).

In Firefox you can do this by opening the preferences "Edit" -> "Settings" -> "Privacy & Security" or by typing about:preferences#privacy in the URL. Then go down to the "Certificates" section and click on "View Certificates". Click on the "Authorities" tab and "Import..." your privoxy.crt. In the "CA certificate trust settings" select "This certificate can identify websites".

In Chrome based browsers, go to the settings and select "Privacy and security" (chrome://settings/privacy). Click on "Security" and on the opened sub-page on "Manage certificates". Now go to the "Authorities" tab and import privoxy.crt and configure it to trust the certificate for website identification.

11.1.7. Enabeling HTTPS inspection

Currently no pages use HTTPS inspection, you need to enable this for some (or all) domains first using user.action (either by editing the file by hand or via the CGI (this requires enable-edit-actions to be enabled in config) at http://config.privoxy.org/show-status (click on user.action Edit button).

Here you can enable HTTPS inspection for individual sites:

  {+https-inspection}
  .badssl.com
  clienttest.ssllabs.com

You can add more individual sites or wildcards (one per line).

Alternatively, you can use a client-tag to dynamically enable/disable this feature via the browser, as described in the next chapter.

11.2. Client Tags HOWTO

Client Tags are a mechanism to dynamically or temporarily enable and disable features in Privoxy for each browser instance.

In our example, we use this for the following two use cases:

To use this feature, you must first define a tag name and a tag description for each client-tag in config, like this:

  client-specific-tag tor Use Tor anonymous proxy
  client-specific-tag https-inspection Enable https-inspection

Now you can open http://config.privoxy.org/client-tags or http://p.p/client-tags and enable or disable the tag there (you may want to bookmark this page for quick access, though it is also available via a link at http://p.p).

You can also temporarily enable a tag, which by default means 3 minutes (180 seconds) (and can be changed using the client-tag-lifetime option in config).

Before this takes effect, you must reference the client tag in your user.action like this:

  {+forward-override{forward-socks5t 127.0.0.1:9050 .} }
  CLIENT-TAG:^tor$

This means that if the "tor" client tag is enabled, all traffic will be forwarded by Privoxy through SOCKS5T to a locally installed tor proxy listening on port 9050.

Similarly, you can specify to use the https-inspection client tag to enable https-inspection:

  {+https-inspection}
  CLIENT-TAG:^https-inspection$

The tag will be set for all requests coming from clients that have requested it to be set. Note that "clients" are distinguished by their IP address. If the IP address changes, the tag must be requested again.