Rebuild docs with updated copyright

[privoxy.git] / ChangeLog

diff --git a/ChangeLog b/ChangeLog
index b729bfa..a1964be 100644 (file)
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,7 +1,157 @@
 --------------------------------------------------------------------
 ChangeLog for Privoxy
 --------------------------------------------------------------------
 --------------------------------------------------------------------
 ChangeLog for Privoxy
 --------------------------------------------------------------------

+*** Version 3.0.34 stable ***
+
+- Bug fixes:
+  - Improve the handling of chunk-encoded responses by buffering the data
+    even if filters are disabled and properly keeping track of where the
+    various chunks are supposed to start and end. Previously Privoxy would
+    merely check the last bytes received to see if they looked like the
+    last-chunk. This failed to work if the last-chunk wasn't received in one
+    read and could also result in actual data being misdetected
+    as last-chunk.
+    Should fix: SF support request #1739.
+    Reported by: withoutname.
+  - remove_chunked_transfer_coding(): Refuse to de-chunk invalid data
+    Previously the data could get corrupted even further.
+    Now we simply pass the unmodified data to the client.
+  - gif_deanimate(): Tolerate multiple image extensions in a row.
+    This allows to deanimate all the gifs on:
+    https://commons.wikimedia.org/wiki/Category:Animated_smilies
+    Fixes SF bug #795 reported by Celejar.
+  - OpenSSL generate_host_certificate(): Use X509_get_subject_name()
+    instead of X509_get_issuer_name() to get the issuer for generated
+    website certificates so there are no warnings in the browser when using
+    an intermediate CA certificate instead of a self-signed root certificate.
+    Problem reported and patch submitted by Chakib Benziane.
+  - can_filter_request_body(): Fix a log message that contained a spurious u.
+  - handle_established_connection(): Check for pending TLS data from the client
+    before checking if data is available on the connection.
+    The TLS library may have already consumed all the data from the client
+    response in which case poll() and select() will not detect that data is
+    available to be read.
+    Sponsored by: Robert Klemme.
+  - ssl_send_certificate_error(): Don't crash if there's no certificate
+    information available. This is only relevant when Privoxy is built with
+    wolfSSL 5.0.0 or later (code not yet published). Earlier wolfSSL versions
+    or the other TLS backends don't seem to trigger the crash.
+  - socks5_connect(): Add support for target hosts specified as IPv4 address
+    Previously the IP address was sent as domain.
+
+- General improvements:
+  - Add a client-body-tagger action which creates tags based on
+    the content of the request body.
+    Sponsored by: Robert Klemme.
+  - When client-body filters are enabled, buffer the whole request
+    before opening a connection to the server.
+    Makes it less likely that the server connection times out
+    and we don't open a connection if the buffering fails anyway.
+    Sponsored by: Robert Klemme.
+  - Add periods to a couple of log messages.
+  - accept_connection(): Add missing space to a log message.
+  - Initialize ca-related defaults with strdup_or_die() so errors
+    aren't silently ignored.
+  - make_path: Use malloc_or_die() in cases where allocation errors
+    were already fatal anyway.
+  - handle_established_connection(): Improve an error message slightly.
+  - receive_client_request(): Reject https URLs without CONNECT request.
+  - Include all requests in the statistics if mutexes are available.
+    Previously in case of reused connections only the last request got
+    counted. The statistics still aren't perfect but it's an improvement.
+  - Add read_socks_reply() and start using it in socks5_connect()
+    to apply the socket timeout more consistently.
+  - socks5_connect(): Deal with domain names in the socks reply
+  - Add a filter for bundeswehr.de that hides the cookie and
+    privacy info banner.
+
+- Action file improvements:
+  - Disable filter{banners-by-size} for .freiheitsfoo.de/.
+  - Disable filter{banners-by-size} for freebsdfoundation.org/.
+  - Disable fast-redirects for consent.youtube.com/.
+  - Block requests to ups.xplosion.de/.
+  - Block requests for elsa.memoinsights.com/t.
+  - Fix a typo in a test.
+  - Disable fast-redirects for launchpad.net/.
+  - Unblock .eff.org/.
+  - Stop unblocking .org/.*(image|banner) which appears to be too generous
+    It let requests like:
+    https://stats.noblogs.org/piwik.php?action_name=anti%20gentrifizierungs%20fest&idsite=10175&rec=1&r=220192&h=17&m=7&s=44&url=https%3A%2F%2Fmuellemcalling.noblogs.org%2F&urlref=https%3A%2F%2Fmuellemcalling.noblogs.org%2Finfostande%2F&_id=&_idn=1&_refts=0&send_image=0&cookie=1&res=1366x768&pv_id=eqr7jX&pf_net=7&pf_srv=3&pf_tfr=2281&pf_dm1=156
+    pass.
+    The example URL http://www.gnu.org/graphics/gnu-head-banner.png is
+    already unblocked due to .gnu.org being unblocked.
+  - Unblock adfd.org/.
+  - Disable filter{banners-by-link} for .eff.org/.
+  - Block requests to odb.outbrain.com/.
+  - Disable fast-redirects for .gandi.net/.
+  - Disable fast-redirects{} for .onion/.*/status/.
+  - Disable fast-redirects{} for twitter.com/.*/status/.
+  - Unblock pinkstinks.de/.
+  - Disable fast-redirects for .hagalil.com/.
+
+- Privoxy-Log-Parser:
+  - Bump version to 0.9.5.
+  - Highlight more log messages.
+  - Highlight the Crunch reason only once. Previously the "crunch reason"
+    could also be highlighted when the URL contained a matching string.
+    The real crunch reason only occurs once per line, so there's no need
+    to continue looking for it after it has been found once.
+    While at it, add a comment with an example log line.
+
+- uagen:
+  - Bump version to 1.2.4.
+  - Update BROWSER_VERSION and BROWSER_REVISION to 102.0
+    to match the User-Agent of the current Firefox ESR.
+  - Explicitly document that changing the 'Gecko token' is suspicious.
+  - Consistently use a lower-case 'c' as copyright symbol.
+  - Bump copyright.
+  - Add 'aarch64' as Linux architecture.
+  - Add OpenBSD architecture 'arm64'.
+  - Stop using sparc64 as FreeBSD architecture.
+    It hasn't been supported for a while now.
+
+- Build system:
+  - Makefile: Add a 'dok' target that depends on the 'error' target
+    to show the "You are not using GNU make or did nor run configure"
+    message.
+  - configure: Fix --with-msan option.
+    Also (probably) reported by Andrew Savchenko.
+
+- macOS build system:
+  - Enable HTTPS inspection when building the macOS binary
+    (using OpenSSL as TLS library).
+
+- Documentation:
+  - Add OpenSSL to the list of libraries that may be licensed under the
+    Apache 2.0 license in which case the linked Privoxy binary has to be
+    distributed under the GPLv3 or later.
+  - config: Fix the documented ca-directory default value.
+    Reported by avoidr.
+  - Rebuild developer-manual and tidy with 'HTML Tidy for FreeBSD version 5.8.0'.
+  - Update developer manual with new macOS packaging instructions.
+  - Note that the FreeBSD installation instructions work for
+    ElectroBSD as well.
+  - Note that FreeBSD/ElectroBSD users can try to install Privoxy
+    as binary package using 'pkg'.
+

 *** Version 3.0.33 stable ***
 *** Version 3.0.33 stable ***

+- Security/Reliability:
+  - cgi_error_no_template(): Encode the template name to prevent
+    XSS (cross-site scripting) when Privoxy is configured to servce
+    the user-manual itself.
+    Commit 0e668e9409c. OVE-20211102-0001. CVE-2021-44543.
+    Reported by: Artem Ivanov
+  - get_url_spec_param(): Free memory of compiled pattern spec
+    before bailing.
+    Reported by Joshua Rogers (Opera) who also provided the fix.
+    Commit 652b4b7cb0. OVE-20211201-0003. CVE-2021-44540.
+  - process_encrypted_request_headers(): Free header memory when
+    failing to get the request destination.
+    Reported by Joshua Rogers (Opera) who also provided the fix.
+    Commit 0509c58045. OVE-20211201-0002. CVE-2021-44541.
+  - send_http_request(): Prevent memory leaks when handling errors
+    Reported by Joshua Rogers (Opera) who also provided the fix.
+    Commit c48d1d6d08. OVE-20211201-0001. CVE-2021-44542.

 
 - Bug fixes:
   - handle_established_connection(): Skip the poll()/select() calls
 
 - Bug fixes:
   - handle_established_connection(): Skip the poll()/select() calls


@@ -18,79 +168,81 @@ ChangeLog for Privoxy
     when unloading an action file with a TAG pattern while
     Privoxy has been compiled without FEATURE_PCRE_HOST_PATTERNS.
     Closes: SF patch request #147. Patch by Maxim Antonov.
     when unloading an action file with a TAG pattern while
     Privoxy has been compiled without FEATURE_PCRE_HOST_PATTERNS.
     Closes: SF patch request #147. Patch by Maxim Antonov.

-  - Establish the TLS connection with the client earlier and decide
-    how to route the request afterwards. This allows to change the
-    forwarding settings based on information from the https-inspected
-    request, for example the path.

   - Adjust build_request_line() to create a CONNECT request line when
     https-inspecting and forwarding to a HTTP proxy.
     Fixes SF bug #925 reported by Wen Yue.
   - load_config(): Add a space that was missing in a log message.
   - Adjust build_request_line() to create a CONNECT request line when
     https-inspecting and forwarding to a HTTP proxy.
     Fixes SF bug #925 reported by Wen Yue.
   - load_config(): Add a space that was missing in a log message.

+  - read_http_request_body(): Fix two error messages that used an
+    incorrect variable.
+  - If the the response is chunk-encoded, ignore the Content-Length
+    header sent by the server.
+    Allows to load https://redmine.lighttpd.net/ with filtering enabled.

 
 - General improvements:
 
 - General improvements:

+  - Allow to edit the add-header action through the CGI editor by
+    generalizing the code that got added with the suppress-tag action.
+    Closes SF patch request #146. Patch by Maxim Antonov.
+  - Add a CGI handler for /wpad.dat that returns a
+    Proxy Auto-Configuration (PAC) file.
+    Among other things, it can be used to instruct clients
+    through DHCP to use Privoxy as proxy.
+    For example with the dnsmasq option:
+    dhcp-option=252,http://config.privoxy.org/wpad.dat
+    Initial patch by Richard Schneidt.
+  - Don't log the applied actions in process_encrypted_request()
+    Log them in continue_https_chat() instead to mirror chat().
+    Prevents the applied actions from getting logged twice
+    for the first request on an https-inspected connection.
+  - OpenSSL generate_host_certificate(): Use config.privoxy.org as Common Name
+    Org and Org Unit if the real host name is too long to get accepted by OpenSSL.
+    Clients should only care about the Subject Alternative Name
+    anyway and we can continue to use the real host name for it.
+    Reported by Miles Wen on privoxy-users@.
+  - Establish the TLS connection with the client earlier and decide
+    how to route the request afterwards. This allows to change the
+    forwarding settings based on information from the https-inspected
+    request, for example the path.
+  - listen_loop(): When shutting down gracefully, close listening ports
+    before waiting for the threads to exit. Allows to start a second
+    Privoxy with the same config file while the first Privoxy is still
+    running.

   - serve(): Close the client socket as well if the server socket
     for an inspected connection has been closed. Privoxy currently
     can't establish a new server connection when the client socket
     is reused and would drop the connection in continue_https_chat()
     anyway.
   - serve(): Close the client socket as well if the server socket
     for an inspected connection has been closed. Privoxy currently
     can't establish a new server connection when the client socket
     is reused and would drop the connection in continue_https_chat()
     anyway.

-  - Don't disable redirect checkers in redirect_url()
+  - Don't disable redirect checkers in redirect_url().

     Disable them in handle_established_connection() instead.
     Doing it in redirect_url() prevented the +redirect{} and
     +fast-redirects{} actions from being logged with LOG_LEVEL_ACTIONS.
     Disable them in handle_established_connection() instead.
     Doing it in redirect_url() prevented the +redirect{} and
     +fast-redirects{} actions from being logged with LOG_LEVEL_ACTIONS.

-  - handle_established_connection(): Slightly improve a comment
-  - handle_established_connection(): Fix a comment
+  - handle_established_connection(): Slightly improve a comment.
+  - handle_established_connection(): Fix a comment.

   - socks5_connect(): Fix indentation.
   - socks5_connect(): Fix indentation.

-  - handle_established_connection(): Improve an error message
-  - create_pattern_spec(): Fix ifdef indentation
-  - Fix comment typos
-  - Add a CGI handler for /wpad.dat that returns a
-    Proxy Auto-Configuration (PAC) file.
-    Among other things, it can be used to instruct clients
-    through DHCP to use Privoxy as proxy.
-    For example with the dnsmasq option:
-    dhcp-option=252,http://config.privoxy.org/wpad.dat
-    Initial patch by Richard Schneidt.
-  - listen_loop(): When shutting down gracefully, close listening ports
-    before waiting for the threads to exit.
-    Allows to start a second Privoxy with the same config file
-    while the first Privoxy is still running.
-  - Allow to edit the add-header action through the CGI editor by
-    generalizing the code that got added with the suppress-tag action.
-    Closes SF patch request #146. Patch by Maxim Antonov.
-  - process_encrypted_request(): Improve a log message
+  - handle_established_connection(): Improve an error message.
+  - create_pattern_spec(): Fix ifdef indentation.
+  - Fix comment typos.
+  - process_encrypted_request(): Improve a log message.

     The function only processes request headers and there
     may still be unread request body data left to process.
     The function only processes request headers and there
     may still be unread request body data left to process.

-  - read_http_request_body(): Fix two error messages that used an incorrect variable.

   - chat(): Log the applied actions before deciding how to forward the request.
   - parse_time_header(): Silence a coverity complaint when building without assertions.
   - chat(): Log the applied actions before deciding how to forward the request.
   - parse_time_header(): Silence a coverity complaint when building without assertions.

-  - receive_encrypted_request_headers(): Improve a log message
+  - receive_encrypted_request_headers(): Improve a log message.

   - mbedTLS get_ciphersuites_from_string(): Use strlcpy() instead of strncpy().
     Previously the terminating NUL wasn't copied which resulted
     in a compiler warning. This didn't cause actual problems as
     the target buffer was initialized by zalloc_or_die() so the
     last byte of the target buffer was NUL already.
     Actually copying the terminating NUL seems clearer, though.
   - mbedTLS get_ciphersuites_from_string(): Use strlcpy() instead of strncpy().
     Previously the terminating NUL wasn't copied which resulted
     in a compiler warning. This didn't cause actual problems as
     the target buffer was initialized by zalloc_or_die() so the
     last byte of the target buffer was NUL already.
     Actually copying the terminating NUL seems clearer, though.

-  - Remove compiler warnings. "log_error(LOG_LEVEL_FATAL, ..." doesn't return
-    but apparently the compiler doesn't know that.
-    Get rid of several "this statement may fall through [-Wimplicit-fallthrough=]" warnings.
-  - If the the response is chunk-encoded, ignore the Content-Length
-    header sent by the server.
-    Allows to load https://redmine.lighttpd.net/ with filtering enabled.
+  - Remove compiler warnings. "log_error(LOG_LEVEL_FATAL, ..."
+    doesn't return but apparently the compiler doesn't know that.
+    Get rid of several "this statement may fall through
+    [-Wimplicit-fallthrough=]" warnings.

   - Store the PEM certificate in a dynamically allocated buffer
     when https-inspecting. Should prevent errors like:
     2021-03-16 22:36:19.148 7f47bbfff700 Error: X509 PEM cert len 16694 is larger than buffer len 16383
     As a bonus it should slightly reduce the memory usage as most
     certificates are smaller than the previously used fixed buffer.
     Reported by: Wen Yue
   - Store the PEM certificate in a dynamically allocated buffer
     when https-inspecting. Should prevent errors like:
     2021-03-16 22:36:19.148 7f47bbfff700 Error: X509 PEM cert len 16694 is larger than buffer len 16383
     As a bonus it should slightly reduce the memory usage as most
     certificates are smaller than the previously used fixed buffer.
     Reported by: Wen Yue

-  - Don't log the applied actions in process_encrypted_request()
-    Log them in continue_https_chat() instead to mirror chat().
-    Prevents the applied actions from getting logged twice
-    for the first request on an https-inspected connection.
-  - OpenSSL generate_host_certificate(): Use config.privoxy.org as Common Name
-    Org and Org Unit if the real host name is too long to get accepted by OpenSSL.
-    Clients should only care about the Subject Alternative Name
-    anyway and we can continue to use the real host name for it.
-    Reported by Miles Wen on privoxy-users@.

   - OpenSSL generate_host_certificate(): Fix two error messsages.
   - Improve description of handle_established_connection()
   - OpenSSL ssl_store_cert(): Translate EVP_PKEY_EC to a string.
   - OpenSSL generate_host_certificate(): Fix two error messsages.
   - Improve description of handle_established_connection()
   - OpenSSL ssl_store_cert(): Translate EVP_PKEY_EC to a string.


@@ -100,33 +252,32 @@ ChangeLog for Privoxy
 - Action file improvements:
   - Disable fast-redirects for .microsoftonline.com/.
   - Disable fast-redirects for idp.springer.com/.
 - Action file improvements:
   - Disable fast-redirects for .microsoftonline.com/.
   - Disable fast-redirects for idp.springer.com/.

-  - Disable fast-redirects for .zeit.de/zustimmung
-  - Unblock adv-archiv.dfn-cert.de/
-  - Block requests to eu-tlp01.kameleoon.eu/
-  - Block requests to fpa-events.arstechnica.com/
+  - Disable fast-redirects for .zeit.de/zustimmung.
+  - Unblock adv-archiv.dfn-cert.de/.
+  - Block requests to eu-tlp01.kameleoon.eu/.
+  - Block requests to fpa-events.arstechnica.com/.

   - Unblock nlnet.nl/.
   - Unblock adguard.com/.
 
 - Privoxy-Log-Parser:
   - Unblock nlnet.nl/.
   - Unblock adguard.com/.
 
 - Privoxy-Log-Parser:

-  - Highlight 'Socket timeout 3 reached: http://127.0.0.1:20000/no-filter/chunked-content/36'
-  - Improve documentation for inactivity-detection mode
-  - Detect date changes when looking for inactivity
+  - Highlight 'Socket timeout 3 reached: http://127.0.0.1:20000/no-filter/chunked-content/36'.
+  - Improve documentation for inactivity-detection mode.
+  - Detect date changes when looking for inactivity.

   - Add a --passed-request-statistics-threshold option
   - Add a --passed-request-statistics-threshold option

-    That can be set to get statistics for requests that
+    that can be set to get statistics for requests that

     were passed.
     were passed.

-  - Add a "inactivity detection" mode
-    Which can be useful for debugging purposes.
-  - Bump version to 0.9.4
-  - Only run print_intro() and print_outro() when syntax highlighting
-  - Rephrase a sentence in the documentation
-  - Highlight 'Client socket 7 is no longer usable. The server socket has been closed.'
-  - Clarify --statistics output
-    by explicitly mentioning that the status codes
-    sent by the server may differ from the ones in
-    "debug 512" messages.
-  - Fix typo in the --statistics output
-  - Remove an unused variable
-  - Highlight 'The peer notified us that the connection on socket 11 is going to be closed'
+  - Add a "inactivity detection" mode which can be useful
+    for debugging purposes.
+  - Bump version to 0.9.4.
+  - Only run print_intro() and print_outro() when syntax highlighting.
+  - Rephrase a sentence in the documentation.
+  - Highlight 'Client socket 7 is no longer usable. The server socket has been closed.'.
+  - Clarify --statistics output by explicitly mentioning that
+    the status codes sent by the server may differ from the ones
+    in "debug 512" messages.
+  - Fix typo in the --statistics output.
+  - Remove an unused variable.
+  - Highlight 'The peer notified us that the connection on socket 11 is going to be closed'.

 
 - Privoxy-Regression-Test:
   - Remove duplicated word in a comment.
 
 - Privoxy-Regression-Test:
   - Remove duplicated word in a comment.


@@ -139,33 +290,37 @@ ChangeLog for Privoxy
   - Add a test for CVE-2021-20217.
 
 - uagen:
   - Add a test for CVE-2021-20217.
 
 - uagen:

-  - Bump generated Firefox version to 91 (ESR)
-  - Bump version to 1.2.3
-  - Bump copyright
+  - Bump generated Firefox version to 91 (ESR).
+  - Bump version to 1.2.3.
+  - Bump copyright.

 
 - Build system:
   - configure: Bump SOURCE_DATE_EPOCH.
   - GNUmakefile.in: Fix typo.
 
 - Build system:
   - configure: Bump SOURCE_DATE_EPOCH.
   - GNUmakefile.in: Fix typo.

-  - configure: Add another warning in case --disable-pthread is used
-    while POSIX threads are available.
+  - configure: Add another warning in case --disable-pthread
+    is used while POSIX threads are available.

     Various features don't even compile when not using threads.
   - Add configure option to enable MemorySanitizer.
   - Add configure option to enable UndefinedBehaviorSanitizer.
   - Add configure option to enable AddressSanitizer.
     Various features don't even compile when not using threads.
   - Add configure option to enable MemorySanitizer.
   - Add configure option to enable UndefinedBehaviorSanitizer.
   - Add configure option to enable AddressSanitizer.

-  - Bump copyright
+  - Bump copyright.

   - Add a configure option to disable pcre JIT compilation.
     While JIT compilation makes filtering faster it can
     cause false-positive valgrind complaints.
     As reported by Gwyn Ciesla in SF bug 924 it also can
     cause problems when the SELinux policy does not grant
     Privoxy "execmem" privileges.
   - Add a configure option to disable pcre JIT compilation.
     While JIT compilation makes filtering faster it can
     cause false-positive valgrind complaints.
     As reported by Gwyn Ciesla in SF bug 924 it also can
     cause problems when the SELinux policy does not grant
     Privoxy "execmem" privileges.

-  - configure: Remove obsolete RPM_BASE check
+  - configure: Remove obsolete RPM_BASE check.

 
 - Windows build system:
   - Update the build script to use mbed tls version 2.6.11.
   - Update build script to use the final 8.45 pcre library.
   - Put all the '--enable-xxx' options in the configure call together.
 
 
 - Windows build system:
   - Update the build script to use mbed tls version 2.6.11.
   - Update build script to use the final 8.45 pcre library.
   - Put all the '--enable-xxx' options in the configure call together.
 

+- macOS build system:
+  - The OSXPackageBuilder repository has been updated and
+    can be used to create macOS packages again.
+

 - Documentation:
   - contacting: Remove obsolete reference to announce.sgml.
   - contacting: Request that the browser cache is cleared before
 - Documentation:
   - contacting: Remove obsolete reference to announce.sgml.
   - contacting: Request that the browser cache is cleared before


@@ -185,13 +340,13 @@ ChangeLog for Privoxy
     Make it easier for users to do the right thing by having all those
     options present in the config.
   - Update TODO list item #184 to note that WolfSSL support will
     Make it easier for users to do the right thing by having all those
     options present in the config.
   - Update TODO list item #184 to note that WolfSSL support will

-    (hopefully) appear after the 3.0.34 release
+    (hopefully) appear after the 3.0.34 release.

   - Update max-client-connections's description.
     On modern systems other than Windows Privoxy should
     use poll() in which case the FD_SETSIZE value isn't
     releveant.
   - Add a warning that the socket-timeout does not apply
   - Update max-client-connections's description.
     On modern systems other than Windows Privoxy should
     use poll() in which case the FD_SETSIZE value isn't
     releveant.
   - Add a warning that the socket-timeout does not apply

-    to operations done by TLS libraries
+    to operations done by TLS libraries.

   - Make documentation slightly less "offensive" for some people
     by avoiding the word "hell".
 
   - Make documentation slightly less "offensive" for some people
     by avoiding the word "hell".
 


@@ -3374,7 +3529,7 @@ being a mix of "U.S. English", "U.K. English" and "Irish English".
 
 
 ----------------------------------------------------------------------
 
 
 ----------------------------------------------------------------------

-Copyright   :  Written by and Copyright (C) 2001-2021 the
+Copyright   :  Written by and Copyright (C) 2001-2023 the

                Privoxy team. https://www.privoxy.org/
 
                Based on the Internet Junkbuster originally written
                Privoxy team. https://www.privoxy.org/
 
                Based on the Internet Junkbuster originally written