From: Fabian Keil Date: Mon, 1 Apr 2024 15:19:06 +0000 (+0200) Subject: wolfSSL: Use LIBWOLFSSL_VERSION_HEX to decide whether or not to use WOLFSSL_X509_V_OK X-Git-Url: http://www.privoxy.org/gitweb/?p=privoxy.git;a=commitdiff_plain;h=HEAD;hp=0958f024d9e4fca6ff8ca2356f10a6e226389365 wolfSSL: Use LIBWOLFSSL_VERSION_HEX to decide whether or not to use WOLFSSL_X509_V_OK As Roland pointed out, in older wolfSSL releases X509_V_OK is an enum so the previous commit 838bc3c0e did not work as expected there. --- diff --git a/AUTHORS b/AUTHORS index 16ad8b1c..e61a0b22 100644 --- a/AUTHORS +++ b/AUTHORS @@ -50,6 +50,7 @@ include (in alphabetical order): Anatoly Arzhnikov Ken Arromdee Natxo Asenjo + avoidr Devin Bayer Havard Berland David Binderman @@ -62,6 +63,8 @@ include (in alphabetical order): Andrew J. Caines Clifford Caoile Edward Carrel + Celejar + Chakib Benziane Pak Chan Wan-Teh Chang Sam Chen @@ -166,6 +169,7 @@ include (in alphabetical order): Andreas Rutkauskas Sam Saperski + Andrew Savchenko Bart Schelstraete Richard Schneidt Gregory Seidman diff --git a/ChangeLog b/ChangeLog index f3991203..a1964beb 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,7 +1,157 @@ -------------------------------------------------------------------- ChangeLog for Privoxy -------------------------------------------------------------------- -*** Version 3.0.33 UNRELEASED *** +*** Version 3.0.34 stable *** + +- Bug fixes: + - Improve the handling of chunk-encoded responses by buffering the data + even if filters are disabled and properly keeping track of where the + various chunks are supposed to start and end. Previously Privoxy would + merely check the last bytes received to see if they looked like the + last-chunk. This failed to work if the last-chunk wasn't received in one + read and could also result in actual data being misdetected + as last-chunk. + Should fix: SF support request #1739. + Reported by: withoutname. + - remove_chunked_transfer_coding(): Refuse to de-chunk invalid data + Previously the data could get corrupted even further. + Now we simply pass the unmodified data to the client. + - gif_deanimate(): Tolerate multiple image extensions in a row. + This allows to deanimate all the gifs on: + https://commons.wikimedia.org/wiki/Category:Animated_smilies + Fixes SF bug #795 reported by Celejar. + - OpenSSL generate_host_certificate(): Use X509_get_subject_name() + instead of X509_get_issuer_name() to get the issuer for generated + website certificates so there are no warnings in the browser when using + an intermediate CA certificate instead of a self-signed root certificate. + Problem reported and patch submitted by Chakib Benziane. + - can_filter_request_body(): Fix a log message that contained a spurious u. + - handle_established_connection(): Check for pending TLS data from the client + before checking if data is available on the connection. + The TLS library may have already consumed all the data from the client + response in which case poll() and select() will not detect that data is + available to be read. + Sponsored by: Robert Klemme. + - ssl_send_certificate_error(): Don't crash if there's no certificate + information available. This is only relevant when Privoxy is built with + wolfSSL 5.0.0 or later (code not yet published). Earlier wolfSSL versions + or the other TLS backends don't seem to trigger the crash. + - socks5_connect(): Add support for target hosts specified as IPv4 address + Previously the IP address was sent as domain. + +- General improvements: + - Add a client-body-tagger action which creates tags based on + the content of the request body. + Sponsored by: Robert Klemme. + - When client-body filters are enabled, buffer the whole request + before opening a connection to the server. + Makes it less likely that the server connection times out + and we don't open a connection if the buffering fails anyway. + Sponsored by: Robert Klemme. + - Add periods to a couple of log messages. + - accept_connection(): Add missing space to a log message. + - Initialize ca-related defaults with strdup_or_die() so errors + aren't silently ignored. + - make_path: Use malloc_or_die() in cases where allocation errors + were already fatal anyway. + - handle_established_connection(): Improve an error message slightly. + - receive_client_request(): Reject https URLs without CONNECT request. + - Include all requests in the statistics if mutexes are available. + Previously in case of reused connections only the last request got + counted. The statistics still aren't perfect but it's an improvement. + - Add read_socks_reply() and start using it in socks5_connect() + to apply the socket timeout more consistently. + - socks5_connect(): Deal with domain names in the socks reply + - Add a filter for bundeswehr.de that hides the cookie and + privacy info banner. + +- Action file improvements: + - Disable filter{banners-by-size} for .freiheitsfoo.de/. + - Disable filter{banners-by-size} for freebsdfoundation.org/. + - Disable fast-redirects for consent.youtube.com/. + - Block requests to ups.xplosion.de/. + - Block requests for elsa.memoinsights.com/t. + - Fix a typo in a test. + - Disable fast-redirects for launchpad.net/. + - Unblock .eff.org/. + - Stop unblocking .org/.*(image|banner) which appears to be too generous + It let requests like: + https://stats.noblogs.org/piwik.php?action_name=anti%20gentrifizierungs%20fest&idsite=10175&rec=1&r=220192&h=17&m=7&s=44&url=https%3A%2F%2Fmuellemcalling.noblogs.org%2F&urlref=https%3A%2F%2Fmuellemcalling.noblogs.org%2Finfostande%2F&_id=&_idn=1&_refts=0&send_image=0&cookie=1&res=1366x768&pv_id=eqr7jX&pf_net=7&pf_srv=3&pf_tfr=2281&pf_dm1=156 + pass. + The example URL http://www.gnu.org/graphics/gnu-head-banner.png is + already unblocked due to .gnu.org being unblocked. + - Unblock adfd.org/. + - Disable filter{banners-by-link} for .eff.org/. + - Block requests to odb.outbrain.com/. + - Disable fast-redirects for .gandi.net/. + - Disable fast-redirects{} for .onion/.*/status/. + - Disable fast-redirects{} for twitter.com/.*/status/. + - Unblock pinkstinks.de/. + - Disable fast-redirects for .hagalil.com/. + +- Privoxy-Log-Parser: + - Bump version to 0.9.5. + - Highlight more log messages. + - Highlight the Crunch reason only once. Previously the "crunch reason" + could also be highlighted when the URL contained a matching string. + The real crunch reason only occurs once per line, so there's no need + to continue looking for it after it has been found once. + While at it, add a comment with an example log line. + +- uagen: + - Bump version to 1.2.4. + - Update BROWSER_VERSION and BROWSER_REVISION to 102.0 + to match the User-Agent of the current Firefox ESR. + - Explicitly document that changing the 'Gecko token' is suspicious. + - Consistently use a lower-case 'c' as copyright symbol. + - Bump copyright. + - Add 'aarch64' as Linux architecture. + - Add OpenBSD architecture 'arm64'. + - Stop using sparc64 as FreeBSD architecture. + It hasn't been supported for a while now. + +- Build system: + - Makefile: Add a 'dok' target that depends on the 'error' target + to show the "You are not using GNU make or did nor run configure" + message. + - configure: Fix --with-msan option. + Also (probably) reported by Andrew Savchenko. + +- macOS build system: + - Enable HTTPS inspection when building the macOS binary + (using OpenSSL as TLS library). + +- Documentation: + - Add OpenSSL to the list of libraries that may be licensed under the + Apache 2.0 license in which case the linked Privoxy binary has to be + distributed under the GPLv3 or later. + - config: Fix the documented ca-directory default value. + Reported by avoidr. + - Rebuild developer-manual and tidy with 'HTML Tidy for FreeBSD version 5.8.0'. + - Update developer manual with new macOS packaging instructions. + - Note that the FreeBSD installation instructions work for + ElectroBSD as well. + - Note that FreeBSD/ElectroBSD users can try to install Privoxy + as binary package using 'pkg'. + +*** Version 3.0.33 stable *** +- Security/Reliability: + - cgi_error_no_template(): Encode the template name to prevent + XSS (cross-site scripting) when Privoxy is configured to servce + the user-manual itself. + Commit 0e668e9409c. OVE-20211102-0001. CVE-2021-44543. + Reported by: Artem Ivanov + - get_url_spec_param(): Free memory of compiled pattern spec + before bailing. + Reported by Joshua Rogers (Opera) who also provided the fix. + Commit 652b4b7cb0. OVE-20211201-0003. CVE-2021-44540. + - process_encrypted_request_headers(): Free header memory when + failing to get the request destination. + Reported by Joshua Rogers (Opera) who also provided the fix. + Commit 0509c58045. OVE-20211201-0002. CVE-2021-44541. + - send_http_request(): Prevent memory leaks when handling errors + Reported by Joshua Rogers (Opera) who also provided the fix. + Commit c48d1d6d08. OVE-20211201-0001. CVE-2021-44542. - Bug fixes: - handle_established_connection(): Skip the poll()/select() calls @@ -18,39 +168,159 @@ ChangeLog for Privoxy when unloading an action file with a TAG pattern while Privoxy has been compiled without FEATURE_PCRE_HOST_PATTERNS. Closes: SF patch request #147. Patch by Maxim Antonov. - - Establish the TLS connection with the client earlier and decide - how to route the request afterwards. This allows to change the - forwarding settings based on information from the https-inspected - request, for example the path. - Adjust build_request_line() to create a CONNECT request line when + - Adjust build_request_line() to create a CONNECT request line when https-inspecting and forwarding to a HTTP proxy. Fixes SF bug #925 reported by Wen Yue. - load_config(): Add a space that was missing in a log message. + - read_http_request_body(): Fix two error messages that used an + incorrect variable. + - If the the response is chunk-encoded, ignore the Content-Length + header sent by the server. + Allows to load https://redmine.lighttpd.net/ with filtering enabled. - General improvements: + - Allow to edit the add-header action through the CGI editor by + generalizing the code that got added with the suppress-tag action. + Closes SF patch request #146. Patch by Maxim Antonov. + - Add a CGI handler for /wpad.dat that returns a + Proxy Auto-Configuration (PAC) file. + Among other things, it can be used to instruct clients + through DHCP to use Privoxy as proxy. + For example with the dnsmasq option: + dhcp-option=252,http://config.privoxy.org/wpad.dat + Initial patch by Richard Schneidt. + - Don't log the applied actions in process_encrypted_request() + Log them in continue_https_chat() instead to mirror chat(). + Prevents the applied actions from getting logged twice + for the first request on an https-inspected connection. + - OpenSSL generate_host_certificate(): Use config.privoxy.org as Common Name + Org and Org Unit if the real host name is too long to get accepted by OpenSSL. + Clients should only care about the Subject Alternative Name + anyway and we can continue to use the real host name for it. + Reported by Miles Wen on privoxy-users@. + - Establish the TLS connection with the client earlier and decide + how to route the request afterwards. This allows to change the + forwarding settings based on information from the https-inspected + request, for example the path. + - listen_loop(): When shutting down gracefully, close listening ports + before waiting for the threads to exit. Allows to start a second + Privoxy with the same config file while the first Privoxy is still + running. - serve(): Close the client socket as well if the server socket for an inspected connection has been closed. Privoxy currently can't establish a new server connection when the client socket is reused and would drop the connection in continue_https_chat() anyway. - - Don't disable redirect checkers in redirect_url() + - Don't disable redirect checkers in redirect_url(). Disable them in handle_established_connection() instead. Doing it in redirect_url() prevented the +redirect{} and +fast-redirects{} actions from being logged with LOG_LEVEL_ACTIONS. - - handle_established_connection(): Slightly improve a comment - - handle_established_connection(): Fix a comment + - handle_established_connection(): Slightly improve a comment. + - handle_established_connection(): Fix a comment. - socks5_connect(): Fix indentation. - - handle_established_connection(): Improve an error message - - create_pattern_spec(): Fix ifdef indentation - - Fix comment typos + - handle_established_connection(): Improve an error message. + - create_pattern_spec(): Fix ifdef indentation. + - Fix comment typos. + - process_encrypted_request(): Improve a log message. + The function only processes request headers and there + may still be unread request body data left to process. + - chat(): Log the applied actions before deciding how to forward the request. + - parse_time_header(): Silence a coverity complaint when building without assertions. + - receive_encrypted_request_headers(): Improve a log message. + - mbedTLS get_ciphersuites_from_string(): Use strlcpy() instead of strncpy(). + Previously the terminating NUL wasn't copied which resulted + in a compiler warning. This didn't cause actual problems as + the target buffer was initialized by zalloc_or_die() so the + last byte of the target buffer was NUL already. + Actually copying the terminating NUL seems clearer, though. + - Remove compiler warnings. "log_error(LOG_LEVEL_FATAL, ..." + doesn't return but apparently the compiler doesn't know that. + Get rid of several "this statement may fall through + [-Wimplicit-fallthrough=]" warnings. + - Store the PEM certificate in a dynamically allocated buffer + when https-inspecting. Should prevent errors like: + 2021-03-16 22:36:19.148 7f47bbfff700 Error: X509 PEM cert len 16694 is larger than buffer len 16383 + As a bonus it should slightly reduce the memory usage as most + certificates are smaller than the previously used fixed buffer. + Reported by: Wen Yue + - OpenSSL generate_host_certificate(): Fix two error messsages. + - Improve description of handle_established_connection() + - OpenSSL ssl_store_cert(): Translate EVP_PKEY_EC to a string. + - OpenSSL ssl_store_cert(): Remove pointless variable initialization. + - OpenSSL ssl_store_cert(): Initialize pointer with NULL instead of 0. - Action file improvements: - Disable fast-redirects for .microsoftonline.com/. - Disable fast-redirects for idp.springer.com/. + - Disable fast-redirects for .zeit.de/zustimmung. + - Unblock adv-archiv.dfn-cert.de/. + - Block requests to eu-tlp01.kameleoon.eu/. + - Block requests to fpa-events.arstechnica.com/. + - Unblock nlnet.nl/. + - Unblock adguard.com/. + +- Privoxy-Log-Parser: + - Highlight 'Socket timeout 3 reached: http://127.0.0.1:20000/no-filter/chunked-content/36'. + - Improve documentation for inactivity-detection mode. + - Detect date changes when looking for inactivity. + - Add a --passed-request-statistics-threshold option + that can be set to get statistics for requests that + were passed. + - Add a "inactivity detection" mode which can be useful + for debugging purposes. + - Bump version to 0.9.4. + - Only run print_intro() and print_outro() when syntax highlighting. + - Rephrase a sentence in the documentation. + - Highlight 'Client socket 7 is no longer usable. The server socket has been closed.'. + - Clarify --statistics output by explicitly mentioning that + the status codes sent by the server may differ from the ones + in "debug 512" messages. + - Fix typo in the --statistics output. + - Remove an unused variable. + - Highlight 'The peer notified us that the connection on socket 11 is going to be closed'. - Privoxy-Regression-Test: - Remove duplicated word in a comment. +- regression-tests.action: + - Add fetch test for http://p.p/wpad.dat. + - Bump for-privoxy-version to 3.0.33 which introduced the wpad.dat support. + - Add more tests for the '/send-banner' code. + - Add test for OVE-20210203-0001. + - Add a test for CVE-2021-20217. + +- uagen: + - Bump generated Firefox version to 91 (ESR). + - Bump version to 1.2.3. + - Bump copyright. + +- Build system: + - configure: Bump SOURCE_DATE_EPOCH. + - GNUmakefile.in: Fix typo. + - configure: Add another warning in case --disable-pthread + is used while POSIX threads are available. + Various features don't even compile when not using threads. + - Add configure option to enable MemorySanitizer. + - Add configure option to enable UndefinedBehaviorSanitizer. + - Add configure option to enable AddressSanitizer. + - Bump copyright. + - Add a configure option to disable pcre JIT compilation. + While JIT compilation makes filtering faster it can + cause false-positive valgrind complaints. + As reported by Gwyn Ciesla in SF bug 924 it also can + cause problems when the SELinux policy does not grant + Privoxy "execmem" privileges. + - configure: Remove obsolete RPM_BASE check. + +- Windows build system: + - Update the build script to use mbed tls version 2.6.11. + - Update build script to use the final 8.45 pcre library. + - Put all the '--enable-xxx' options in the configure call together. + +- macOS build system: + - The OSXPackageBuilder repository has been updated and + can be used to create macOS packages again. + - Documentation: - contacting: Remove obsolete reference to announce.sgml. - contacting: Request that the browser cache is cleared before @@ -61,6 +331,24 @@ ChangeLog for Privoxy filter https responses. - developer-manual: Mention that announce.txt should be updated when doing a release. + - config: Explicitly mention that the CGI pages disclosing the + ca-password can be blocked and upgrade the disclosure paragraphs + to a warning. + - Put all the requested debug options in the config file. + Section 11.1 of the Privoxy user manual lists all the debug + options that should be enabled when reporting problems or requesting support. + Make it easier for users to do the right thing by having all those + options present in the config. + - Update TODO list item #184 to note that WolfSSL support will + (hopefully) appear after the 3.0.34 release. + - Update max-client-connections's description. + On modern systems other than Windows Privoxy should + use poll() in which case the FD_SETSIZE value isn't + releveant. + - Add a warning that the socket-timeout does not apply + to operations done by TLS libraries. + - Make documentation slightly less "offensive" for some people + by avoiding the word "hell". *** Version 3.0.32 stable *** @@ -3241,7 +3529,7 @@ being a mix of "U.S. English", "U.K. English" and "Irish English". ---------------------------------------------------------------------- -Copyright : Written by and Copyright (C) 2001-2021 the +Copyright : Written by and Copyright (C) 2001-2023 the Privoxy team. https://www.privoxy.org/ Based on the Internet Junkbuster originally written diff --git a/GNUmakefile.in b/GNUmakefile.in index 9476426c..cf79aae1 100644 --- a/GNUmakefile.in +++ b/GNUmakefile.in @@ -243,6 +243,10 @@ OPENSSL_SRC = @FEATURE_HTTPS_INSPECTION_ONLY_OPENSSL@openssl.c OPENSSL_OBJS = @FEATURE_HTTPS_INSPECTION_ONLY_OPENSSL@$(OPENSSL_SRC:.c=.o) OPENSSL_HDRS = @FEATURE_HTTPS_INSPECTION_ONLY_OPENSSL@$(OPENSSL_SRC:.c=.h) +WOLFSSL_SRC = @FEATURE_HTTPS_INSPECTION_ONLY_WOLFSSL@wolfssl.c +WOLFSSL_OBJS = @FEATURE_HTTPS_INSPECTION_ONLY_WOLFSSL@$(WOLFSSL_SRC:.c=.o) +WOLFSSL_HDRS = @FEATURE_HTTPS_INSPECTION_ONLY_WOLFSSL@$(WOLFSSL_SRC:.c=.h) + PCRS_SRC = @STATIC_PCRS_ONLY@pcrs.c PCRS_OBJS = @STATIC_PCRS_ONLY@$(PCRS_SRC:.c=.@OBJEXT@) PCRS_HDRS = @STATIC_PCRS_ONLY@$(PCRS_SRC:.c=.h) @@ -262,9 +266,9 @@ SOCKET_LIB = @SOCKET_LIB@ # PThreads library, if needed. PTHREAD_LIB = @PTHREAD_ONLY@@PTHREAD_LIB@ -SRCS = $(C_SRC) $(CLIENT_TAG_SRC) $(FUZZ_SRC) $(W32_SRC) $(PCRS_SRC) $(REGEX_SRC) $(SSL_SRC) $(MBEDTLS_SRC) $(OPENSSL_SRC) -OBJS = $(C_OBJS) $(CLIENT_TAG_OBJS) $(FUZZ_OBJS) $(W32_OBJS) $(PCRS_OBJS) $(REGEX_OBJS) $(SSL_OBJS) $(MBEDTLS_OBJS) $(OPENSSL_OBJS) -HDRS = $(C_HDRS) $(W32_HDRS) $(PCRS_HDRS) $(REGEX_HDRS) $(SSL_HDRS) $(MBEDTLS_HDRS) $(OPENSSL_HDRS) +SRCS = $(C_SRC) $(CLIENT_TAG_SRC) $(FUZZ_SRC) $(W32_SRC) $(PCRS_SRC) $(REGEX_SRC) $(SSL_SRC) $(MBEDTLS_SRC) $(OPENSSL_SRC) $(WOLFSSL_SRC) +OBJS = $(C_OBJS) $(CLIENT_TAG_OBJS) $(FUZZ_OBJS) $(W32_OBJS) $(PCRS_OBJS) $(REGEX_OBJS) $(SSL_OBJS) $(MBEDTLS_OBJS) $(OPENSSL_OBJS) $(WOLFSSL_OBJS) +HDRS = $(C_HDRS) $(W32_HDRS) $(PCRS_HDRS) $(REGEX_HDRS) $(SSL_HDRS) $(MBEDTLS_HDRS) $(OPENSSL_HDRS) $(WOLFSSL_HDRS) LIBS = @LIBS@ $(W32_LIB) $(SOCKET_LIB) $(PTHREAD_LIB) @@ -440,7 +444,7 @@ tarball-dist: dist-check clean clobber for i in `find . -type f -a -not \( -path "*/CVS*" -o -name ".*" \ -o -path "*/debian/*" -o -path "*/actions/*" -o -name "*.php" -o \ -name "PACKAGERS" -o -path "*.git/*" \ - -o -path "*/doc/webserver/feeds*" \) | sort`; do \ + -o -path "*/doc/webserver/feeds*" \) | sort | $(SED) 's@^\./@@'`; do \ files="$$files privoxy-$(VERSION)-$(CODE_STATUS)/$$i"; \ done && \ cd .. && $(TAR) -cvhf privoxy-$(VERSION)-$(CODE_STATUS)-src.tar $$files ; \ @@ -923,7 +927,6 @@ install: CONF_DEST LOG_DEST PID_DEST check_doc GROUP_T $(ECHO) " and to install the config files as that user and/or group!" ;\ $(ECHO) " Please read INSTALL, and create a privoxy user and group!" ;\ $(ECHO) "*******************************************************************" ;\ - exit 1 ;\ fi ;\ else \ GROUP_T=$(GROUP) ;\ diff --git a/INSTALL b/INSTALL index c8e1f96f..cdc5cb2c 100644 --- a/INSTALL +++ b/INSTALL @@ -39,8 +39,8 @@ compiler like gcc are required. When building from a source tarball, first unpack the source: - tar xzvf privoxy-3.0.33-stable-src.tar.gz - cd privoxy-3.0.33-stable + tar xzvf privoxy-3.0.35-beta-src.tar.gz + cd privoxy-3.0.35-beta To build the development version, you can get the source code by doing: diff --git a/Makefile b/Makefile index 493643f6..2dc8180b 100644 --- a/Makefile +++ b/Makefile @@ -70,6 +70,8 @@ error: .PHONY: error +dok: error + ############################################################################# ## Local Variables: diff --git a/README b/README index 2b039912..70787489 100644 --- a/README +++ b/README @@ -32,8 +32,9 @@ * *********************************************************************/ -This README is included with Privoxy 3.0.33. See https://www.privoxy.org/ for -more information. The current code maturity level is "stable". +This README is included with the development version of Privoxy 3.0.35. See +https://www.privoxy.org/ for more information. The current code maturity level +is "UNRELEASED", but seems stable to us :). ------------------------------------------------------------------------------- @@ -104,16 +105,22 @@ try it with FTP or other protocols for the simple reason it does not work. The actions list can be configured via the web interface accessed via http:// p.p/, as well other options. +All configuration files are subject to unannounced changes during the +development process. + ------------------------------------------------------------------------------- 5. DOCUMENTATION -There should be documentation in the 'doc' subdirectory. In particular, see the -User Manual there, the FAQ, and those interested in Privoxy development, should -look at developer-manual. +There should be documentation in the 'doc' subdirectory, but it may not be +completed at this point. In particular, see the User Manual there, the FAQ, and +those interested in Privoxy development, should look at developer-manual. -The source and configuration files are all well commented. The main -configuration files are: 'config', 'default.action', and 'default.filter'. +The most up to date source of information on the current development version, +may still be either comments in the source code, or the included configuration +files. The source and configuration files are all well commented. The main +configuration files are: 'config', 'default.action', and 'default.filter' in +the top-level source directory. Included documentation may vary according to platform and packager. All documentation is posted on https://www.privoxy.org, in case you don't have it, diff --git a/TODO b/TODO index 11b4d806..f6c839f3 100644 --- a/TODO +++ b/TODO @@ -377,7 +377,7 @@ https://www.privoxy.org/donate currently can result in client requests to config.privoxy.org on the Internet which may not be desirable. -150) Add blacklistd support. +150) Add blocklistd support (https://man.netbsd.org/blocklistd.8). 151) Let the dok-tidy target work cross-platform without introducing a ton of white-space changes that hide the content changes. @@ -422,10 +422,6 @@ https://www.privoxy.org/donate 163) Use subdirectories in the certificate-directory to lower the number of files per directory. -164) Evaluate switching from pcreposix(3) to pcre's native api - for URL matching which allows to compile the patterns once - at load-time. - 165) Add a max-connections-per-client directive. 167) Set up a public Privoxy-Filter-Test instance. @@ -532,6 +528,22 @@ https://www.privoxy.org/donate 200) Add a config directive that causes Privoxy to remove all host certificates before exiting. +201) Add an action to change the trusted-cas-file for a section. + This should be useful in countries where a person-in-the-middle + attack is known to happen on some domains but should not be tolerated + on others. It would also allow to limit the accepted CA certificates + for given domains instead of accepting all that are specified with + the trusted-cas-file directive. + +202) Allow to use multiple log files with different debug settings. + Suggested by Jonathan Marten in FR#607. + +203) Add HTTP/2 support. + +204) Make the Privoxy website available over IPv6. + +205) Document how commit messages should look like. + ########################################################################## Hosting wish list (relevant for #53) diff --git a/acconfig.h b/acconfig.h index dbf443bb..9fbe5a72 100644 --- a/acconfig.h +++ b/acconfig.h @@ -225,11 +225,17 @@ /* Define if pcre.h must be included as */ #undef PCRE_H_IN_SUBDIR +#undef PCRE2_H_IN_SUBDIR + +#undef HAVE_PCRE2 +#undef HAVE_PCRE2POSIX /* Define if pcreposix.h must be included as */ #undef PCREPOSIX_H_IN_SUBDIR +#undef PCRE2POSIX_H_IN_SUBDIR + @BOTTOM@ /* diff --git a/actionlist.h b/actionlist.h index fc7f5142..d95b8226 100644 --- a/actionlist.h +++ b/actionlist.h @@ -57,6 +57,7 @@ DEFINE_ACTION_STRING ("change-x-forwarded-for", ACTION_CHANGE_X_FORWARDE DEFINE_CGI_PARAM_RADIO ("change-x-forwarded-for", ACTION_CHANGE_X_FORWARDED_FOR, ACTION_STRING_CHANGE_X_FORWARDED_FOR, "block", 0) DEFINE_CGI_PARAM_RADIO ("change-x-forwarded-for", ACTION_CHANGE_X_FORWARDED_FOR, ACTION_STRING_CHANGE_X_FORWARDED_FOR, "add", 1) DEFINE_ACTION_MULTI ("client-body-filter", ACTION_MULTI_CLIENT_BODY_FILTER) +DEFINE_ACTION_MULTI ("client-body-tagger", ACTION_MULTI_CLIENT_BODY_TAGGER) DEFINE_ACTION_MULTI ("client-header-filter", ACTION_MULTI_CLIENT_HEADER_FILTER) DEFINE_ACTION_MULTI ("client-header-tagger", ACTION_MULTI_CLIENT_HEADER_TAGGER) DEFINE_ACTION_STRING ("content-type-overwrite", ACTION_CONTENT_TYPE_OVERWRITE, ACTION_STRING_CONTENT_TYPE) diff --git a/actions.c b/actions.c index a80ee646..9fd3249c 100644 --- a/actions.c +++ b/actions.c @@ -829,7 +829,7 @@ int update_action_bits_for_tag(struct client_state *csp, const char *tag) } /* and check if one of the tag patterns matches the tag, */ - if (0 == regexec(b->url->pattern.tag_regex, tag, 0, NULL, 0)) + if (regex_matches(b->url->pattern.tag_regex, tag)) { /* if it does, update the action bit map, */ if (merge_current_action(csp->action, b->action)) @@ -884,7 +884,7 @@ jb_err check_negative_tag_patterns(struct client_state *csp, unsigned int flag) } for (tag = csp->tags->first; NULL != tag; tag = tag->next) { - if (0 == regexec(b->url->pattern.tag_regex, tag->str, 0, NULL, 0)) + if (regex_matches(b->url->pattern.tag_regex, tag->str)) { /* * The pattern matches at least one tag, thus the action @@ -1115,6 +1115,8 @@ static const char *filter_type_to_string(enum filter_type filter_type) return "suppress tag filter"; case FT_CLIENT_BODY_FILTER: return "client body filter"; + case FT_CLIENT_BODY_TAGGER: + return "client body tagger"; case FT_ADD_HEADER: return "add-header action"; #ifdef FEATURE_EXTERNAL_FILTERS diff --git a/cgi.c b/cgi.c index cb1f0725..d60166f2 100644 --- a/cgi.c +++ b/cgi.c @@ -7,7 +7,7 @@ * This only contains the framework functions, the * actual handler functions are declared elsewhere. * - * Copyright : Written by and Copyright (C) 2001-2020 + * Copyright : Written by and Copyright (C) 2001-2021 * members of the Privoxy team. https://www.privoxy.org/ * * Based on the Internet Junkbuster originally written @@ -1199,7 +1199,8 @@ jb_err cgi_error_no_template(const struct client_state *csp, ").

\n" "\n" "\n"; - const size_t body_size = strlen(body_prefix) + strlen(template_name) + strlen(body_suffix) + 1; + size_t body_size = strlen(body_prefix) + strlen(body_suffix) + 1; + const char *encoded_template_name; assert(csp); assert(rsp); @@ -1213,9 +1214,17 @@ jb_err cgi_error_no_template(const struct client_state *csp, rsp->head_length = 0; rsp->is_static = 0; + encoded_template_name = html_encode(template_name); + if (encoded_template_name == NULL) + { + return JB_ERR_MEMORY; + } + + body_size += strlen(encoded_template_name); rsp->body = malloc_or_die(body_size); strlcpy(rsp->body, body_prefix, body_size); - strlcat(rsp->body, template_name, body_size); + strlcat(rsp->body, encoded_template_name, body_size); + freez(encoded_template_name); strlcat(rsp->body, body_suffix, body_size); rsp->status = strdup(status); @@ -2014,7 +2023,7 @@ jb_err template_fill(char **template_ptr, const struct map *exports) char buf[BUFFER_SIZE]; char *tmp_out_buffer; char *file_buffer; - size_t size; + size_t buffer_size, new_size; int error; const char *flags; @@ -2023,7 +2032,7 @@ jb_err template_fill(char **template_ptr, const struct map *exports) assert(exports); file_buffer = *template_ptr; - size = strlen(file_buffer) + 1; + buffer_size = strlen(file_buffer) + 1; /* * Assemble pcrs joblist from exports map @@ -2073,7 +2082,10 @@ jb_err template_fill(char **template_ptr, const struct map *exports) } else { - error = pcrs_execute(job, file_buffer, size, &tmp_out_buffer, &size); + error = pcrs_execute(job, file_buffer, buffer_size, &tmp_out_buffer, + &new_size); + + buffer_size = new_size; pcrs_free_job(job); if (NULL == tmp_out_buffer) diff --git a/cgiedit.c b/cgiedit.c index 3b8b12c9..80f20d87 100644 --- a/cgiedit.c +++ b/cgiedit.c @@ -12,7 +12,7 @@ * * Stick to the short names in this file for consistency. * - * Copyright : Written by and Copyright (C) 2001-2014 the + * Copyright : Written by and Copyright (C) 2001-2023 the * Privoxy team. https://www.privoxy.org/ * * Based on the Internet Junkbuster originally written @@ -252,6 +252,12 @@ static const struct action_type_info action_type_info[] = "client-body-filter-all", "client_body_filter_all", "P", "CLIENT-BODY-FILTER" }, + { + ACTION_MULTI_CLIENT_BODY_TAGGER, + "client-body-tagger-params", "client-body-tagger", + "client-body-tagger-all", "client_body_tagger_all", + "Q", "CLIENT-BODY-TAGGER" + }, { ACTION_MULTI_ADD_HEADER, "add-header-params", "add-header", @@ -363,38 +369,6 @@ static jb_err get_file_name_param(struct client_state *csp, const char *param_name, const char **pfilename); -/* Internal convenience functions */ -static char *section_target(const unsigned sectionid); - -/********************************************************************* - * - * Function : section_target - * - * Description : Given an unsigned (section id) n, produce a dynamically - * allocated string of the form #l, for use in link - * targets. - * - * XXX: The hash should be moved into the templates - * to make this function more generic and render - * stringify() obsolete. - * - * Parameters : - * 1 : sectionid = start line number of section - * - * Returns : String with link target, or NULL if out of - * memory - * - *********************************************************************/ -static char *section_target(const unsigned sectionid) -{ - char buf[30]; - - snprintf(buf, sizeof(buf), "#l%u", sectionid); - return(strdup(buf)); - -} - - /********************************************************************* * * Function : stringify @@ -552,7 +526,7 @@ jb_err cgi_edit_actions_url_form(struct client_state *csp, if (!err) err = map(exports, "v", 1, file->version_str, 1); if (!err) err = map(exports, "p", 1, url_encode(lookup(parameters, "p")), 0); if (!err) err = map(exports, "u", 1, html_encode(cur_line->unprocessed), 0); - if (!err) err = map(exports, "jumptarget", 1, section_target(section_start_line_number), 0); + if (!err) err = map(exports, "jumptarget", 1, stringify(section_start_line_number), 0); edit_free_file(file); @@ -712,7 +686,7 @@ jb_err cgi_edit_actions_remove_url_form(struct client_state *csp, if (!err) err = map(exports, "v", 1, file->version_str, 1); if (!err) err = map(exports, "p", 1, url_encode(lookup(parameters, "p")), 0); if (!err) err = map(exports, "u", 1, html_encode(cur_line->unprocessed), 0); - if (!err) err = map(exports, "jumptarget", 1, section_target(section_start_line_number), 0); + if (!err) err = map(exports, "jumptarget", 1, stringify(section_start_line_number), 0); if (!err) err = map(exports, "actions-file", 1, html_encode(file->filename), 0); edit_free_file(file); @@ -1899,12 +1873,12 @@ static jb_err get_url_spec_param(struct client_state *csp, } err = create_pattern_spec(compiled, s); free(s); + free_pattern_spec(compiled); if (err) { free(param); return (err == JB_ERR_MEMORY) ? JB_ERR_MEMORY : JB_ERR_CGI_PARAMS; } - free_pattern_spec(compiled); if (param[strlen(param) - 1] == '\\') { @@ -1935,12 +1909,12 @@ static jb_err get_url_spec_param(struct client_state *csp, } err = create_pattern_spec(compiled, s); free(s); + free_pattern_spec(compiled); if (err) { free(param); return (err == JB_ERR_MEMORY) ? JB_ERR_MEMORY : JB_ERR_CGI_PARAMS; } - free_pattern_spec(compiled); } *pvalue = param; @@ -2797,8 +2771,8 @@ jb_err cgi_edit_actions_for_url(struct client_state *csp, for (i = 0; !err && i < SZ(string_action_type_info); i++) { - err = action_render_string_actions_template(exports, cur_line->data.action, filter_template, - &string_action_type_info[i]); + err = action_render_string_actions_template(exports, + cur_line->data.action, filter_template, &string_action_type_info[i]); } freez(filter_template); @@ -2891,7 +2865,7 @@ jb_err cgi_edit_actions_for_url(struct client_state *csp, if ((csp->rlist[i] != NULL) && (csp->rlist[i]->f != NULL)) { filter_group = csp->rlist[i]->f; - for (;(!err) && (filter_group != NULL); filter_group = filter_group->next) + for (; (!err) && (filter_group != NULL); filter_group = filter_group->next) { char current_mode = 'x'; char number[20]; @@ -2901,6 +2875,7 @@ jb_err cgi_edit_actions_for_url(struct client_state *csp, const int multi_action_index = action_type_info[type].multi_action_index; assert(type < MAX_FILTER_TYPES); + assert(multi_action_index < ACTION_MULTI_COUNT); filter_name = cur_line->data.action->multi_add[multi_action_index]->first; while ((filter_name != NULL) @@ -4487,8 +4462,9 @@ static jb_err actions_to_radio(struct map * exports, * * Function : action_render_string_actions_template * - * Description : Converts a actionsfile entry into HTML template for actions with string - * filters (currently SUPPRESS-TAG actions only) + * Description : Converts an actionsfile entry into HTML template for + * actions with string filters (currently SUPPRESS-TAG + * actions only) * * Parameters : * 1 : exports = List of substitutions to add to. @@ -4500,9 +4476,9 @@ static jb_err actions_to_radio(struct map * exports, * JB_ERR_MEMORY on out-of-memory * *********************************************************************/ -static jb_err action_render_string_actions_template(struct map * exports, +static jb_err action_render_string_actions_template(struct map *exports, const struct action_spec *action, - const char* action_template, + const char *action_template, const struct string_action_type_info *string_action_type) { jb_err err = JB_ERR_OK; @@ -4516,6 +4492,8 @@ static jb_err action_render_string_actions_template(struct map * exports, struct list_entry *list; }; + assert(type->multi_action_index < ACTION_MULTI_COUNT); + struct action_multi desc[] = { { 'y', action->multi_add[type->multi_action_index][0].first }, { 'n', action->multi_remove[type->multi_action_index][0].first } diff --git a/cgisimple.c b/cgisimple.c index 96506273..961510f2 100644 --- a/cgisimple.c +++ b/cgisimple.c @@ -5,7 +5,7 @@ * Purpose : Simple CGIs to get information about Privoxy's * status. * - * Copyright : Written by and Copyright (C) 2001-2021 the + * Copyright : Written by and Copyright (C) 2001-2022 the * Privoxy team. https://www.privoxy.org/ * * Based on the Internet Junkbuster originally written @@ -1262,9 +1262,15 @@ jb_err cgi_show_status(struct client_state *csp, char buf[BUFFER_SIZE]; #ifdef FEATURE_STATISTICS +#ifdef MUTEX_LOCKS_AVAILABLE + float percentage_blocked; + unsigned long long local_number_of_requests_received; + unsigned long long local_number_of_requests_blocked; +#else float perc_rej; /* Percentage of http requests rejected */ int local_urls_read; int local_urls_rejected; +#endif #endif /* ndef FEATURE_STATISTICS */ jb_err err = JB_ERR_OK; @@ -1302,9 +1308,15 @@ jb_err cgi_show_status(struct client_state *csp, } #ifdef FEATURE_STATISTICS +#ifdef MUTEX_LOCKS_AVAILABLE + privoxy_mutex_lock(&block_statistics_mutex); + local_number_of_requests_received = number_of_requests_received; + local_number_of_requests_blocked = number_of_requests_blocked; + privoxy_mutex_unlock(&block_statistics_mutex); +#else local_urls_read = urls_read; local_urls_rejected = urls_rejected; - +#endif /* * Need to alter the stats not to include the fetch of this * page. @@ -1315,7 +1327,11 @@ jb_err cgi_show_status(struct client_state *csp, * urls_rejected--; * This will be incremented subsequently * */ +#ifdef MUTEX_LOCKS_AVAILABLE + if (local_number_of_requests_received == 0) +#else if (local_urls_read == 0) +#endif { if (!err) err = map_block_killer(exports, "have-stats"); } @@ -1323,6 +1339,19 @@ jb_err cgi_show_status(struct client_state *csp, { if (!err) err = map_block_killer(exports, "have-no-stats"); +#ifdef MUTEX_LOCKS_AVAILABLE + percentage_blocked = (float)local_number_of_requests_blocked * 100.0F / + (float)local_number_of_requests_received; + + snprintf(buf, sizeof(buf), "%llu", local_number_of_requests_received); + if (!err) err = map(exports, "requests-received", 1, buf, 1); + + snprintf(buf, sizeof(buf), "%llu", local_number_of_requests_blocked); + if (!err) err = map(exports, "requests-blocked", 1, buf, 1); + + snprintf(buf, sizeof(buf), "%6.2f", percentage_blocked); + if (!err) err = map(exports, "percent-blocked", 1, buf, 1); +#else perc_rej = (float)local_urls_rejected * 100.0F / (float)local_urls_read; @@ -1334,6 +1363,7 @@ jb_err cgi_show_status(struct client_state *csp, snprintf(buf, sizeof(buf), "%6.2f", perc_rej); if (!err) err = map(exports, "percent-blocked", 1, buf, 1); +#endif } #else /* ndef FEATURE_STATISTICS */ diff --git a/client-tags.c b/client-tags.c index 51e8a9c5..d8cbb350 100644 --- a/client-tags.c +++ b/client-tags.c @@ -43,6 +43,7 @@ #include "miscutil.h" #include "errlog.h" #include "parsers.h" +#include "urlmatch.h" struct client_specific_tag { @@ -658,7 +659,7 @@ int client_tag_match(const struct pattern_spec *pattern, for (tag = tags->first; tag != NULL; tag = tag->next) { - if (0 == regexec(pattern->pattern.tag_regex, tag->str, 0, NULL, 0)) + if (regex_matches(pattern->pattern.tag_regex, tag->str)) { log_error(LOG_LEVEL_TAGGING, "Client tag '%s' matches.", tag->str); return 1; diff --git a/config b/config index c38b7f56..779a2160 100644 --- a/config +++ b/config @@ -1,6 +1,6 @@ -# Sample Configuration File for Privoxy 3.0.33 +# Sample Configuration File for Privoxy 3.0.34 # -# Copyright (C) 2001-2021 Privoxy Developers https://www.privoxy.org/ +# Copyright (C) 2001-2023 Privoxy Developers https://www.privoxy.org/ # ##################################################################### # # @@ -15,7 +15,7 @@ # 4. ACCESS CONTROL AND SECURITY # # 5. FORWARDING # # 6. MISCELLANEOUS # -# 7. HTTPS INSPECTION (EXPERIMENTAL) # +# 7. HTTPS INSPECTION # # 8. WINDOWS GUI OPTIONS # # # ##################################################################### @@ -151,8 +151,8 @@ # # Notes: # -# The value of this option only matters if the experimental -# trust mechanism has been activated. (See trustfile below.) +# The value of this option only matters if the trust mechanism +# has been activated. (See trustfile below.) # # If you use the trust mechanism, it is a good idea to write up # some on-line documentation about your trust policy and to @@ -2442,8 +2442,8 @@ socket-timeout 300 # receive-buffer-size 32768 # # -# 7. HTTPS INSPECTION (EXPERIMENTAL) -# =================================== +# 7. HTTPS INSPECTION +# ==================== # # HTTPS inspection allows to filter encrypted requests and # responses. This is only supported when Privoxy has been built with @@ -2465,7 +2465,7 @@ socket-timeout 300 # # Default value: # -# Empty string +# ./CA # # Effect if unset: # diff --git a/configure.in b/configure.in index 10cf7048..1f7256dd 100644 --- a/configure.in +++ b/configure.in @@ -1,6 +1,6 @@ dnl Process this file with autoconf to produce a configure script. dnl -dnl Written by and Copyright (C) 2001-2021 the +dnl Written by and Copyright (C) 2001-2023 the dnl Privoxy team. https://www.privoxy.org/ dnl dnl Based on the Internet Junkbuster originally written @@ -80,12 +80,12 @@ dnl release and no new release is near. VERSION_MAJOR=3 VERSION_MINOR=0 -VERSION_POINT=33 -CODE_STATUS="stable" +VERSION_POINT=35 +CODE_STATUS="UNRELEASED" dnl Timestamp (date +%s) used by the mtree-spec target. dnl Should be updated before releases but forgetting it isn't critical. -SOURCE_DATE_EPOCH=1636218132 +SOURCE_DATE_EPOCH=1672586827 dnl ================================================================= dnl Substitute the version numbers @@ -168,7 +168,7 @@ if test "x$with_asan" = "xyes"; then LDFLAGS="$LDFLAGS -fsanitize=address" fi -AC_ARG_WITH(asan, [ --with-msan Enable MemorySanitizer. Requires compiler support.]) +AC_ARG_WITH(msan, [ --with-msan Enable MemorySanitizer. Requires compiler support.]) if test "x$with_msan" = "xyes"; then CFLAGS="$CFLAGS -fsanitize=memory" LDFLAGS="$LDFLAGS -fsanitize=memory" @@ -216,7 +216,7 @@ else ], [ if test $ID = no ; then - AC_MSG_ERROR(There is no 'id' programm on this system) + AC_MSG_ERROR(There is no 'id' program on this system) else AC_MSG_RESULT(none specified) USER=$with_user @@ -255,7 +255,7 @@ else ], [ if test $BGROUPS = no ; then - AC_MSG_ERROR(There is no 'groups' programm on this system) + AC_MSG_ERROR(There is no 'groups' program on this system) else AC_MSG_RESULT(none specified) GROUP=$with_group; @@ -615,9 +615,9 @@ AC_CHECK_FUNC(gmtime_r, [ AC_TRY_COMPILE([ # include ], [ - struct time *t; - struct tm *tm; - (void) gmtime_r(t, tm) + time_t t; + struct tm tm; + (void) gmtime_r(&t, &tm) ], [ AC_MSG_RESULT(ok) AC_DEFINE(HAVE_GMTIME_R) @@ -633,9 +633,9 @@ AC_CHECK_FUNC(localtime_r, [ AC_TRY_COMPILE([ # include ], [ - struct time *t; - struct tm *tm; - (void) localtime_r(t, tm) + time_t t; + struct tm tm; + (void) localtime_r(&t, &tm) ], [ AC_MSG_RESULT(ok) AC_DEFINE(HAVE_LOCALTIME_R) @@ -863,12 +863,48 @@ else ]) fi +AC_ARG_ENABLE(pcre2, +[ --disable-pcre2 Don't try to use pcre2 even if it's available], +[enableval2=$enableval], +[enableval2=yes]) +if test $enableval2 = yes; then + try_pcre2=yes +else + AC_MSG_WARN([Ignoring pcre2 even if it's available]) + try_pcre2=no + have_pcre2=no +fi + +if test $try_pcre2 != no; then dnl ================================================================= dnl Checks for libraries. dnl ================================================================= dnl Note: Some systems may have the library but not the system header dnl file, so we must check for both. dnl Also check for correct version +AC_CHECK_LIB(pcre2-8, pcre2_compile_8, [ + AC_CHECK_HEADER(pcre2.h, [ + AC_EGREP_HEADER(pcre2_pattern_info, pcre2.h,[have_pcre2=yes; AC_DEFINE(HAVE_PCRE2)], [AC_MSG_WARN([[pcre2 old version installed]]); have_pcre2=no]) + ], [ + AC_CHECK_HEADER(pcre2/pcre2.h, [ + AC_EGREP_HEADER(pcre2_pattern_info, pcre2/pcre2.h, [have_pcre2=yes; AC_DEFINE(PCRE2_H_IN_SUBDIR)], [AC_MSG_WARN([[pcre2 old version installed]]); have_pcre2=no]) + ], [have_pcre2=no]) + ], [#define PCRE2_CODE_UNIT_WIDTH 8]) +], [have_pcre2=no]) + +AC_CHECK_LIB(pcre2-posix, regcomp, [ + AC_CHECK_HEADER(pcre2posix.h, [ + AC_EGREP_HEADER(pcre2_regerror, pcre2posix.h, [have_pcre2posix=yes],[AC_MSG_WARN([[pcre2posix old version installed]]); have_pcre2posix=no]) + ], [ + AC_CHECK_HEADER(pcre/pcre2posix.h, [ + AC_EGREP_HEADER(pcre2_regerror, pcre2/pcre2posix.h, [have_pcre2posix=yes; AC_DEFINE(PCRE2POSIX_H_IN_SUBDIR)],[AC_MSG_WARN([[pcre2posix old version installed]]); have_pcre2posix=no]) + ], [have_pcre2posix=no]) + ]) +], [have_pcre2posix=no], -lpcre2-8) +fi + +if test $have_pcre2 = "no"; then + AC_CHECK_LIB(pcre, pcre_compile, [ AC_CHECK_HEADER(pcre.h, [ AC_EGREP_HEADER(pcre_fullinfo, pcre.h, [have_pcre=yes], [AC_MSG_WARN([[pcre old version installed]]); have_pcre=no]) @@ -889,6 +925,7 @@ AC_CHECK_LIB(pcreposix, regcomp, [ ]) ], [have_pcreposix=no], -lpcre) +fi dnl ================================================================ dnl libpcrs is temporarily disabled. dnl @@ -1095,6 +1132,33 @@ fi # we don't need pcreposix, then link pcre dynamically; else # build it and link statically # + +#check for libpcre2 first. then regular pcre + +if test $have_pcre2 = "yes"; then + echo "using libpcre2" + STATIC_PCRE_ONLY=# + LIBS="$LIBS -lpcre2-8 -lpcre2-posix" + if test "$use_static_pcre" = "yes"; then + pcre_dyn=no + AC_DEFINE(PCRE2_STATIC,1,[Define to statically link to the pcre2 library on Windows.]) +# see pcre2-10.42/src/pcre2.h line 54 +# #if defined(_WIN32) && !defined(PCRE2_STATIC) +# # ifndef PCRE2_EXP_DECL +# # define PCRE2_EXP_DECL extern __declspec(dllimport) +# # endif +# #endif +# If you want to statically link a program against a PCRE2 library in the form of +# a non-dll .a file, you must define PCRE2_STATIC before including src/pcre2.h. +# Otherwise the exported functions will be declared __declspec(dllimport), +# with unwanted results. eg: build failures with error messages like +# undefined reference to `_imp__pcre2_compile_8' + else + pcre_dyn=yes + AC_DEFINE(FEATURE_DYNAMIC_PCRE,1,[Define to dynamically link to pcre.]) + fi +else + if test $have_pcre = "yes"; then echo "using libpcre" STATIC_PCRE_ONLY=# @@ -1116,7 +1180,8 @@ if test $have_pcre = "yes"; then AC_DEFINE(FEATURE_DYNAMIC_PCRE,1,[Define to dynamically link to pcre.]) fi else - AC_MSG_ERROR(pcre library not detected.) + AC_MSG_ERROR(Detected neither pcre2 nor pcre library.) +fi fi AC_DEFINE(FEATURE_CONNECTION_KEEP_ALIVE) @@ -1230,6 +1295,45 @@ if test X"$OPT_OPENSSL" != Xno; then fi AC_SUBST(FEATURE_HTTPS_INSPECTION_ONLY_OPENSSL) +dnl ======================================================== +dnl check for wolfSSL which can be used for https inspection +dnl ======================================================== +FEATURE_HTTPS_INSPECTION_ONLY_WOLFSSL=# +OPT_WOLFSSL=no +AC_ARG_WITH(wolfssl,dnl +AC_HELP_STRING([--with-wolfssl], [Enable Wolfssl/LibreSSL detection for https inspection.]) +AC_HELP_STRING([--without-wolfssl], [Disable Wolfssl/LibreSSL detection]), + OPT_WOLFSSL=$withval) + +if test X"$OPT_WOLFSSL" != Xno; then + if test X"$OPT_MBEDTLS" != Xno; then + AC_MSG_ERROR([wolfSSL and mbedTLS support can't be enabled at the same time]) + fi + if test X"$OPT_OPENSSL" != Xno; then + AC_MSG_ERROR([wolfSSL and OpenSSL support can't be enabled at the same time]) + fi + + AC_CHECK_LIB(wolfssl, wolfSSL_Init, [HAVE_WOLFSSL_INIT="yes"]) + + if test "x$HAVE_WOLFSSL_INIT" = "xyes"; then + AC_DEFINE(FEATURE_HTTPS_INSPECTION, 1, [if SSL/TLS is enabled]) + AC_DEFINE(FEATURE_HTTPS_INSPECTION_WOLFSSL, 1, [if Wolfssl is enabled]) + AC_SUBST(FEATURE_HTTPS_INSPECTION_WOLFSSL, [1]) + FEATURE_HTTPS_INSPECTION="yes" + FEATURE_HTTPS_INSPECTION_WOLFSSL="yes" + fi + + if test "x$FEATURE_HTTPS_INSPECTION_WOLFSSL" = "xyes"; then + AC_MSG_NOTICE([Detected wolfSSL. Enabling https inspection.]) + + LIBS="$LIBS -lwolfssl" + + FEATURE_HTTPS_INSPECTION_ONLY= + FEATURE_HTTPS_INSPECTION_ONLY_WOLFSSL= + fi +fi +AC_SUBST(FEATURE_HTTPS_INSPECTION_ONLY_WOLFSSL) + AC_SUBST(FEATURE_HTTPS_INSPECTION_ONLY) dnl ======================================================== diff --git a/deanimate.c b/deanimate.c index a6759da4..ae13d89c 100644 --- a/deanimate.c +++ b/deanimate.c @@ -6,7 +6,7 @@ * fly. High-level functions include: * - Deanimation of GIF images * - * Copyright : Written by and Copyright (C) 2001 - 2004, 2006 by the + * Copyright : Written by and Copyright (C) 2001-2021 by the * Privoxy team. https://www.privoxy.org/ * * Based on the GIF file format specification (see @@ -328,6 +328,7 @@ int gif_deanimate(struct binbuffer *src, struct binbuffer *dst, int get_first_im { unsigned char c; struct binbuffer *image; + int image_buffered = 0; if (NULL == src || NULL == dst) { @@ -382,9 +383,9 @@ int gif_deanimate(struct binbuffer *src, struct binbuffer *dst, int get_first_im * Parse the GIF block by block and copy the relevant * parts to dst */ - while(src->offset < src->size) + while (src->offset < src->size) { - switch(buf_getbyte(src, 0)) + switch (buf_getbyte(src, 0)) { /* * End-of-GIF Marker: Append current image if we got @@ -398,8 +399,14 @@ int gif_deanimate(struct binbuffer *src, struct binbuffer *dst, int get_first_im * Image block: Extract to current image buffer. */ case 0x2c: - image->offset = 0; + if (image_buffered == 1) + { + /* Discard previous image. */ + image->offset = 0; + image_buffered = 0; + } if (gif_extract_image(src, image)) goto failed; + image_buffered = 1; if (get_first_image) goto write; continue; @@ -410,14 +417,16 @@ int gif_deanimate(struct binbuffer *src, struct binbuffer *dst, int get_first_im switch (buf_getbyte(src, 1)) { /* - * Image extension: Copy extension header and image - * to the current image buffer + * Image extension: Copy extension header + * and continue looking for new blocks. */ case 0xf9: - image->offset = 0; - if (buf_copy(src, image, 8) || buf_getbyte(src, 0) != 0x2c) goto failed; - if (gif_extract_image(src, image)) goto failed; - if (get_first_image) goto write; + if (image_buffered == 1) + { + image->offset = 0; + image_buffered = 0; + } + if (buf_copy(src, image, 8)) goto failed; continue; /* diff --git a/debian/changelog b/debian/changelog index 97814c53..3d95a933 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,10 +1,75 @@ -privoxy (3.0.33~gitsnapshot-1) UNRELEASED; urgency=medium +privoxy (3.0.35~gitsnapshot-1) UNRELEASED; urgency=medium * Unreleased GIT snapshot. - * Adapt Debian patches to GIT changes. + * The following patches are now incorporated upstream: 35_pcre2-support, + 36_pcre2-patternfix. + + -- Roland Rosenfeld Sun, 05 Feb 2023 10:58:57 +0100 + +privoxy (3.0.34-3) unstable; urgency=medium + + * 35_pcre2-support: Update patch based on upstream GIT. + * 36_pcre2-patternfix: Fix using anchored pattern in pcre2. + + -- Roland Rosenfeld Sun, 27 Aug 2023 16:20:57 +0200 + +privoxy (3.0.34-2) unstable; urgency=medium + + * Add ro debconf translation. Thanks to Remus-Gabriel Chelu + (Closes: #1033443). + * 35_pcre2-support: Build with libpcre2-dev instead of obsolete pcre3 + (Closes: #999981). + + -- Roland Rosenfeld Fri, 30 Jun 2023 09:28:22 +0200 + +privoxy (3.0.34-1) unstable; urgency=medium + + * New upstream version 3.0.34. + * The following patches are now incorporated upstream: 35_gif_deanimate, + 36_pending_tls, 37_https_connect. + + -- Roland Rosenfeld Sun, 05 Feb 2023 10:41:26 +0100 + +privoxy (3.0.33-4) unstable; urgency=medium + + * Add it debconf translation. Thanks to Ceppo + (Closes: #1024071). + * Update to Standards-Version 4.6.2 (no changes). + * Remove lsb-base dependency, since it is essential. + * 35_gif_deanimate: Tolerate multiple image extensions in a row. + * 36_pending_tls: Check for pending TLS data from the client before + checking if data is available on the connection. + * 37_https_connect: Reject https URLs without CONNECT request. + * d/maintscript: fix version numbers. + + -- Roland Rosenfeld Sat, 28 Jan 2023 15:46:54 +0100 + +privoxy (3.0.33-3) unstable; urgency=medium + + * d/tests/conditional-defines: ignore 32-bit time_t on 32bit systems. + + -- Roland Rosenfeld Sun, 25 Sep 2022 19:49:17 +0200 + +privoxy (3.0.33-2) unstable; urgency=medium + + * d/maintscript: Remove orphaned + templates/edit-actions-for-url-string-filter (Closes: #1001501). + * d/tests/conditional-defines: Check that expected #defines are enabled. + * d/rules: --enable-compression at build time (still disabled in config). + * d/tests/privoxy-regression-tests: 2 pass check with different options. + * Update to Standards-Version 4.6.1 (no changes). + * Add systemd timer support for cert cleanup. + + -- Roland Rosenfeld Sun, 25 Sep 2022 11:52:25 +0200 + +privoxy (3.0.33-1) unstable; urgency=medium + + * New upstream version 3.0.33. + * Adapt all patches to new version. + * Remove pcre dir from copyright and debian/source/lintian-overrides. * Undo lintian override for bug 992465. - -- Roland Rosenfeld Thu, 02 Dec 2021 09:04:45 +0100 + -- Roland Rosenfeld Wed, 08 Dec 2021 16:29:31 +0100 privoxy (3.0.32-3) unstable; urgency=medium diff --git a/debian/control b/debian/control index 9e7bde5e..0396f097 100644 --- a/debian/control +++ b/debian/control @@ -11,14 +11,14 @@ Build-Depends: autoconf, groff , libbrotli-dev, libmbedtls-dev, - libpcre3-dev, + libpcre2-dev, man2html , opensp:native , po-debconf, sgmlspl , w3m , zlib1g-dev -Standards-Version: 4.6.0 +Standards-Version: 4.6.2 Homepage: https://www.privoxy.org/ Vcs-Git: https://salsa.debian.org/debian/privoxy.git Vcs-Browser: https://salsa.debian.org/debian/privoxy @@ -29,7 +29,6 @@ Architecture: any Depends: adduser, debconf, logrotate, - lsb-base (>= 3.2-13), ucf, ${misc:Depends}, ${perl:Depends}, diff --git a/debian/copyright b/debian/copyright index 00a73b08..917e794b 100644 --- a/debian/copyright +++ b/debian/copyright @@ -6,7 +6,7 @@ Source: https://sourceforge.net/projects/ijbswa/files/Sources/ Files: * Copyright: Copyright (C) 2001-2021 by the Privoxy team. https://www.privoxy.org/ - Copyright (C) 2001-2021 by Privoxy Developers + Copyright (C) 2001-2023 by Privoxy Developers Parts Copyright (c) 1997 by Anonymous Coders and Junkbusters, Inc. Parts Copyright (C) 1999 Adam Lock Parts Copyright (C) 1998-1999 Steve Cheng @@ -58,35 +58,9 @@ License: GPL-2+ Files: debian/* Copyright: - 2002-2021 Roland Rosenfeld + 2002-2023 Roland Rosenfeld License: GPL-2+ -Files: ./pcre/* -Copyright: Copyright (c) 1997-2000 University of Cambridge -Comment: - Written by: Philip Hazel - . - University of Cambridge Computing Service, - Cambridge, England. Phone: +44 1223 334714. -License: BSD-like - Permission is granted to anyone to use this software for any purpose on any - computer system, and to redistribute it freely, subject to the following - restrictions: - . - 1. This software is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. - . - 2. The origin of this software must not be misrepresented, either by - explicit claim or by omission. - . - 3. Altered versions must be plainly marked as such, and must not be - misrepresented as being the original software. - . - 4. If PCRE is embedded in any software that is released under the GNU - General Purpose Licence (GPL), then the terms of that licence shall - supersede any condition above with which it is incompatible. - License: GPL-2+ This program is free software; you can redistribute it and/or modify it under the terms of the GNU General diff --git a/debian/maintscript b/debian/maintscript index 77e4ed98..051d5185 100644 --- a/debian/maintscript +++ b/debian/maintscript @@ -1,3 +1,4 @@ -rm_conffile /etc/privoxy/global.action 3.0.11-1~ privoxy -rm_conffile /etc/privoxy/standard.action 3.0.11-1~ privoxy -rm_conffile /etc/privoxy/templates/show-version 3.0.28-2~ privoxy +rm_conffile /etc/privoxy/global.action 3.0.33-4~ privoxy +rm_conffile /etc/privoxy/standard.action 3.0.33-4~ privoxy +rm_conffile /etc/privoxy/templates/show-version 3.0.33-4~ privoxy +rm_conffile /etc/privoxy/templates/edit-actions-for-url-string-filter 3.0.33-3~ privoxy diff --git a/debian/po/it.po b/debian/po/it.po new file mode 100644 index 00000000..32fca702 --- /dev/null +++ b/debian/po/it.po @@ -0,0 +1,33 @@ +# privoxy po-debconf italian translation +# Copyright (C) 2022 privoxy's copyright holder +# This file is distributed under the same license as the privoxy package. +# Ceppo \n" +"Language-Team: Italian \n" +"Language: it\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" + +#. Type: string +#. Description +#: ../templates:1001 +msgid "Adresses on with Privoxy listens:" +msgstr "Indirizzi su cui Privoxy deve ascoltare:" + +#. Type: string +#. Description +#: ../templates:1001 +msgid "" +"Please enter a space separated list of address:port combinations on which " +"Privoxy will listen for client requests." +msgstr "" +"Inserire una lista separata da spazi di combinazioni indirizzo:porta su cui " +"Privoxy resterà in ascolto per le richieste dei client." diff --git a/debian/po/ro.po b/debian/po/ro.po new file mode 100644 index 00000000..6549093b --- /dev/null +++ b/debian/po/ro.po @@ -0,0 +1,43 @@ +# Mesajele în limba română pentru pachetul privoxy. +# Romanian translation of privoxy. +# Copyright © 2023 THE PACKAGE'S COPYRIGHT HOLDER +# This file is distributed under the same license as the privoxy package. +# +# Remus-Gabriel Chelu , 2023. +# +# Cronologia traducerii fișierului „privoxy”: +# Traducerea inițială, făcută de R-GC, pentru versiunea privoxy 3.0.34-1(2006-11-01). +# Actualizare a traducerii pentru versiunea Y, făcută de X, Y(anul). +# +msgid "" +msgstr "" +"Project-Id-Version: privoxy 3.0.34-1\n" +"Report-Msgid-Bugs-To: privoxy@packages.debian.org\n" +"POT-Creation-Date: 2016-04-08 17:17+0200\n" +"PO-Revision-Date: 2023-03-19 08:06+0100\n" +"Last-Translator: Remus-Gabriel Chelu \n" +"Language-Team: Romanian \n" +"Language: ro\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" +"Plural-Forms: nplurals=3; plural=(n==1 ? 0 : n==0 || (n!=1 && n%100>=1 && " +"n%100<=19) ? 1 : 2);\n" +"X-Bugs: Report translation errors to the Language-Team address.\n" +"X-Generator: Poedit 3.2.2\n" + +#. Type: string +#. Description +#: ../templates:1001 +msgid "Adresses on with Privoxy listens:" +msgstr "Adresele pe care Privoxy ascultă:" + +#. Type: string +#. Description +#: ../templates:1001 +msgid "" +"Please enter a space separated list of address:port combinations on which " +"Privoxy will listen for client requests." +msgstr "" +"Introduceți o listă de combinații de adresă:port, separată de spații, pe care " +"Privoxy va asculta cererile clienților." diff --git a/debian/privoxy-cleanup.service b/debian/privoxy-cleanup.service new file mode 100644 index 00000000..88871817 --- /dev/null +++ b/debian/privoxy-cleanup.service @@ -0,0 +1,14 @@ +[Unit] +Description=privoxy certificate cleanup +Documentation=man:privoxy(8) https://www.privoxy.org/user-manual/ +ConditionACPower=true + +[Service] +Type=oneshot +ExecStart=/etc/cron.daily/privoxy systemd-timer + +# performance options +Nice=19 +IOSchedulingClass=best-effort +IOSchedulingPriority=7 + diff --git a/debian/privoxy-cleanup.timer b/debian/privoxy-cleanup.timer new file mode 100644 index 00000000..28b254fd --- /dev/null +++ b/debian/privoxy-cleanup.timer @@ -0,0 +1,12 @@ +[Unit] +Description=privoxy certificate cleanup +Documentation=man:privoxy(8) https://www.privoxy.org/user-manual/ +Before=logrotate.timer + +[Timer] +OnCalendar=daily +AccuracySec=12h +Persistent=true + +[Install] +WantedBy=timers.target diff --git a/debian/privoxy.cron.daily b/debian/privoxy.cron.daily index 114fd412..97eff8a7 100644 --- a/debian/privoxy.cron.daily +++ b/debian/privoxy.cron.daily @@ -4,9 +4,15 @@ set -e +# skip in favour of systemd timer if called from cron.daily +if [ -d /run/systemd/system ] && [ "$1" != "systemd-timer" ]; then + exit 0 +fi + + CERTDIR=/var/lib/privoxy/certs EXPIREDAYS=90 if [ -d $CERTDIR ]; then - find $CERTDIR -type f -mtime +$EXPIREDAYS | xargs -r rm -f + find $CERTDIR -type f -mtime +$EXPIREDAYS -print0 | xargs -0 -r rm -f fi diff --git a/debian/rules b/debian/rules index d6c23b39..5b49300f 100755 --- a/debian/rules +++ b/debian/rules @@ -1,6 +1,6 @@ #!/usr/bin/make -f # -# (c) 2002-2021 Roland Rosenfeld +# (c) 2002-2022 Roland Rosenfeld # # Uncomment this to turn on verbose mode. #export DH_VERBOSE=1 @@ -25,6 +25,7 @@ override_dh_auto_configure: --enable-external-filters \ --enable-extended-statistics \ --enable-pcre-host-patterns \ + --enable-compression \ --with-mbedtls \ --with-brotli \ --with-docbook=/usr/share/sgml/docbook/stylesheet/dsssl/modular @@ -116,3 +117,7 @@ endif override_dh_perl: # We only need perl-base: dh_perl -d + +override_dh_installsystemd: + dh_installsystemd + dh_installsystemd --name privoxy-cleanup diff --git a/debian/source/lintian-overrides b/debian/source/lintian-overrides deleted file mode 100644 index 3ddafbb2..00000000 --- a/debian/source/lintian-overrides +++ /dev/null @@ -1,4 +0,0 @@ -# we don't use the local pcre source but use the ones from libpcre3-dev, -# so we can ignore the warnings about old libtool versions: -privoxy source: ancient-libtool pcre/ltconfig -privoxy source: ancient-libtool pcre/ltmain.sh 1.3.4 diff --git a/debian/tests/conditional-defines b/debian/tests/conditional-defines new file mode 100755 index 00000000..0f00d33f --- /dev/null +++ b/debian/tests/conditional-defines @@ -0,0 +1,41 @@ +#!/bin/sh +# +# Check http://config.privoxy.org/show-status for Conditional #defines enabled +# This wrapper starts privoxy on port 8119 and runs conditional-defines.pl +# +# (c) 2022 Roland Rosenfeld + +PORT=8119 + +TESTSDIR=$(dirname "$0") + +if [ -z "$AUTOPKGTEST_TMP" ]; then + AUTOPKGTEST_TMP=$(mktemp -d) +fi + +trap 'rm -rf "$AUTOPKGTEST_TMP"' EXIT + +CONFIG=$AUTOPKGTEST_TMP/config +PIDFILE=$AUTOPKGTEST_TMP/privoxy.pid +PRIVOXY=$AUTOPKGTEST_TMP/privoxy + +sed -e "s/^listen-address.*/listen-address 127.0.0.1:$PORT/" \ + -e "s%^logdir.*%logdir $AUTOPKGTEST_TMP%" \ + < /usr/share/privoxy/config > "$CONFIG" + +cp /usr/sbin/privoxy "$PRIVOXY" + +echo "Starting privoxy on port $PORT" +$PRIVOXY --pidfile "$PIDFILE" "$CONFIG" + +http_proxy=http://127.0.0.1:$PORT/ +export http_proxy + +"$TESTSDIR"/conditional-defines.pl +EXITVAL=$? + +echo "Stopping privoxy on port $PORT" +# shellcheck disable=SC2046 +kill $(cat "$PIDFILE") + +exit $EXITVAL diff --git a/debian/tests/conditional-defines.pl b/debian/tests/conditional-defines.pl new file mode 100755 index 00000000..693f4fbd --- /dev/null +++ b/debian/tests/conditional-defines.pl @@ -0,0 +1,75 @@ +#!/usr/bin/perl +# +# Check http://config.privoxy.org/show-status for Conditional #defines enabled +# +# (c) 2022 Roland Rosenfeld + +use strict; +use warnings; +use LWP::UserAgent (); +use HTML::TreeBuilder 5 -weak; + +my $exitcode = 0; + +my $ua = LWP::UserAgent->new(timeout => 10); +$ua->env_proxy; +my $response = $ua->get('http://config.privoxy.org/show-status'); +if (!$response->is_success) { + die $response->status_line; +} +my $tree = HTML::TreeBuilder->new; +$tree->parse($response->decoded_content); + +# Search for "Conditional #defines:" table: +my $summary = 'The state of some ./configure options and what they do.'; +my $table = $tree->look_down('_tag' => 'table', + 'summary' => $summary); +unless (defined $table) { + die "summary '$summary' not found in tables"; +} + +# These features are intentionaly disabled, all others should be enabled: +my %disabled_features = ('FEATURE_ACCEPT_FILTER' => 1, # BSD only + 'FEATURE_STRPTIME_SANITY_CHECKS' =>1, # BSD libc only + 'FEATURE_GRACEFUL_TERMINATION' =>1, # devel only + ); + +my $enabled = 0; +my $disabled_ok = 0; +my $disabled_nok = 0; +foreach my $tr ($table->look_down('_tag' => 'tr')) { + my $td2 = ($tr->look_down('_tag' => 'td')) [1]; + next unless defined $td2; + my $code = $tr->look_down('_tag' => 'code'); + my $feature = $code->detach_content; + my $value = $td2->detach_content; + if ($value !~ /Yes/) { + # feature disabled, check whitelist + if ($feature eq 'FEATURE_64_BIT_TIME_T') { + # See https://en.wikipedia.org/wiki/Year_2038_problem + # On Linux >= 5.6 time_t should be 64bit, too. + printf "%s is disabled, which is ok on 32bit systems", $feature; + $disabled_ok++; + } elsif (! defined $disabled_features{$feature}) { + printf "%s is disabled, but should be enabled\n", $feature; + $exitcode = 1; + $disabled_nok++; + } else { + $disabled_ok++; + } + } else { + $enabled++; + } +} + +printf "%d features enabled\n", $enabled; +printf "%d features intentionally disabled\n", $disabled_ok; +printf "%d features unintentionally disabled\n", $disabled_nok; + +if ($enabled < 10) { + printf STDERR "Found only %d Conditional #defines, seems test ist broken\n", + $enabled; + $exitcode = 1; +} + +exit $exitcode; diff --git a/debian/tests/control b/debian/tests/control index 1566c8fe..6ff73cfe 100644 --- a/debian/tests/control +++ b/debian/tests/control @@ -1,2 +1,5 @@ Tests: privoxy-regression-test Depends: curl, @ + +Tests: conditional-defines +Depends: privoxy, libwww-perl, libhtml-tree-perl diff --git a/debian/tests/privoxy-regression-test b/debian/tests/privoxy-regression-test index 1c29fa06..e8fdd9d9 100755 --- a/debian/tests/privoxy-regression-test +++ b/debian/tests/privoxy-regression-test @@ -3,7 +3,7 @@ # Run privoxy-regression-test.pl on a configuration extended by # regression-tests.action. # -# (c) 2018-2021 Roland Rosenfeld +# (c) 2018-2022 Roland Rosenfeld PORT=8119 @@ -11,37 +11,57 @@ if [ -z "$AUTOPKGTEST_TMP" ]; then AUTOPKGTEST_TMP=$(mktemp -d) fi -trap "rm -rf $AUTOPKGTEST_TMP" EXIT +trap 'rm -rf "$AUTOPKGTEST_TMP"' EXIT CONFIG=$AUTOPKGTEST_TMP/config PIDFILE=$AUTOPKGTEST_TMP/privoxy.pid PRIVOXY=$AUTOPKGTEST_TMP/privoxy +cp /usr/sbin/privoxy "$PRIVOXY" + +http_proxy=http://127.0.0.1:$PORT/ +export http_proxy + +OUTFILE=$AUTOPKGTEST_TMP/output + + +echo "#### pass 1: some optiones disabled" + sed -e "s/^listen-address.*/listen-address 127.0.0.1:$PORT/" \ -e "s%^logdir.*%logdir $AUTOPKGTEST_TMP%" \ - -e "s/^enable-edit-actions.*/enable-edit-actions 1/" \ -e "s/^keep-alive-timeout.*/keep-alive-timeout 21/" \ - -e "s/^#connection-sharing.*/connection-sharing 1/" \ - < /usr/share/privoxy/config > $CONFIG -echo "actionsfile regression-tests.action" >> $CONFIG - -cp /usr/sbin/privoxy $PRIVOXY + -e "s/^#connection-sharing.*/connection-sharing 0/" \ + < /usr/share/privoxy/config > "$CONFIG" +echo "actionsfile regression-tests.action" >> "$CONFIG" echo "Starting privoxy on port $PORT" -$PRIVOXY --pidfile $PIDFILE $CONFIG +$PRIVOXY --pidfile "$PIDFILE" "$CONFIG" -http_proxy=http://127.0.0.1:$PORT/ -export http_proxy +/usr/bin/privoxy-regression-test --max-level 200 --show-skipped-tests \ + | tee "$OUTFILE" 2>&1 -OUTFILE=$AUTOPKGTEST_TMP/output + +echo "#### pass 2 with some more options enabled" + +sed -e "s/^listen-address.*/listen-address 127.0.0.1:$PORT/" \ + -e "s%^logdir.*%logdir $AUTOPKGTEST_TMP%" \ + -e "s/^enable-remote-toggle.*/enable-remote-toggle 1/" \ + -e "s/^enable-edit-actions.*/enable-edit-actions 1/" \ + -e "s/^enable-proxy-authentication-forwarding.*/enable-proxy-authentication-forwarding 1/" \ + -e "s/^keep-alive-timeout.*/keep-alive-timeout 21/" \ + -e "s/^#connection-sharing.*/connection-sharing 1/" \ + < /usr/share/privoxy/config > "$CONFIG" +echo "actionsfile regression-tests.action" >> "$CONFIG" /usr/bin/privoxy-regression-test --max-level 200 --show-skipped-tests \ - | tee $OUTFILE 2>&1 + | tee -a "$OUTFILE" 2>&1 + echo "Stopping privoxy on port $PORT" -kill $(cat $PIDFILE) +kill "$(cat "$PIDFILE")" -if grep -q 'Executed.*, 0 failures.' $OUTFILE +# Check that 2 passes have both 0 failures: +if [ "$(grep -c 'Executed.*, 0 failures' "$OUTFILE")" = 2 ] then exit 0 else diff --git a/default.action.master b/default.action.master index f0b76227..98d95508 100644 --- a/default.action.master +++ b/default.action.master @@ -21,7 +21,7 @@ # and enhancements are better placed in user.action, # the match-all section has been moved to match-all.action. # -# Copyright : Written by and Copyright (C) 2001-2021 the +# Copyright : Written by and Copyright (C) 2001-2023 the # Privoxy team. https://www.privoxy.org/ # # Feedback welcome, for details please have a look at: @@ -119,7 +119,7 @@ # the domain syntax above. A comprehensive discussion of regular expressions # wouldn't fit here. # -# Perl compatible regular expressions are used. See the pcre/docs/ direcory or +# Perl compatible regular expressions are used. See the pcre/docs/ directory or # man perlre (also available at http://perldoc.perl.org/perlre.html) for # details. The appendix to our User Manual also has some detail. # @@ -232,7 +232,7 @@ # not pixels!) # If the option "first" is given, the first frame of the animation # is used as the replacement. If "last" is given, the last frame of -# the animation is used instead, which propably makes more sense for +# the animation is used instead, which probably makes more sense for # most banner animations, but also has the risk of not showing the # entire last frame (if it is only a delta to an earlier frame). # @@ -712,8 +712,8 @@ ada*. adri*. # URL = https://adguard.com/ adguard.com/ -# URL = https://adv-archiv.dfn-cert.de/adv/2021-2222/ -adv-archiv.dfn-cert.de/ +# URL = https://adfd.org/austausch/ +adfd.org/ ############################################################################# # Generic block patterns by path: @@ -773,9 +773,6 @@ support./(.*/)?track .gov .hs-*.de .fh-*.de -#MASTER# REMARKS: Try to avoid harmless names in non-commercial organizations. Added 10/24/06 -# URL = http://www.gnu.org/graphics/gnu-head-banner.png -.org/.*(image|banner) ############################################################################# # Catch-all for false-positives that are just TOO obvious to let go @@ -956,6 +953,20 @@ ih.adscale.de/ #MASTER# BLOCK-REFERRER: https://www.ksta.de/ratgeber/digital/plattform-moodle-streikt-viele-schueler-koennen-nicht-arbeiten---gebauer-weiss-von-nichts-37913640 # Blocked URL = https://api.theadex.com/collector/v1/d/285/5184/cmframe/Lw0EHAiwAxA2GAIUhAK0AXaMBA?c=4248866896491805402 api.theadex.com/ +# Blocked URL = https://odb.outbrain.com/utils/get?url=https%3A%2F%2Fwww.ksta.de%2Fratgeber%2Fgesundheit%2Fhausaerztin-erklaert-milder-verlauf-bei-einer-omikron-infektion---was-heisst-das--39393164&idx=0&rand=18368&key=NANOWDGT01&widgetJSId=AR_2&va=true&et=true&format=html&pdobuid=-1&adblck=false&abwl=false&px=205&py=1389&vpd=0&cw=940&activeTab=true&darkMode=false&settings=true&recs=true&version=2000570&sig=bvSz55IV&apv=false&osLang=en-CA&winW=1350&winH=621&scrW=1366&scrH=768&dpr=1&secured=true&cnsntv2=CPSvfxSPSvfxSAGABCENB9CgAP_AAEAAAAYgIXBVJD7dTWlAMHZ5GNpkCYAU1sAUIOQCCBCAAwAFAEGA8IAC0SACEEQABAACAQAAgRABAAAEEABUAECgQAAEAQEkAAQAhAAIAAJEAAEQAgAQAAoIAAAAAAAIAAABCgSAkBiQQ8LGRGAghIAwQgAQwAABgIACAAMASAAYABAAAAIAAABAAgIEEELoAIELgEQkC0ABAAFQAMgAcgA8AEAAMgAaQBEAEUAJgATwArABvADmAH4AQgAhoBEAESAJYAUoAtwBhwD7AP0AgYBFACNAEpALmAYoA2gBuADiAHoAPkAhsBIgCdgFDgLzAYMAyQBpwDWQHBAPHAhCEACAAkAXQAyECBgaAOAFYALgAhgB-AHyASIAnYMABAOoIgDABWAEMAPwA-QCRAE7CAAIAJBUAUAJgAXAB-AJBAXmMgBgBMgH2AfgC8xgAEAsQ6BsABUADIAHIAPgBAADIAGgAPoAiACKAEwAJ4AVgAuABfADeAHMAPwAhoBEAESAJYATAAowBSgCxAFuAMMAaMA-wD9AIGARQAiwBKQCxAFzAMUAbQA3ABxADqAHoAQ2Ai8BIICRAE7AKHAXmAwYBiQDJAGWANOAcWA8cB-I4AeAAgAC4AJAAyABoAIiAXoAwAB5AD5AIQAXQAyEBpoDbCEBQADIATAAuABfADeALGAfYB-AEUAJSAXMAxQBtADqAHoASCAkQBbQDEgHjgQoIABgAEABoALEAYABdADbCUBYABAAGQAOAAfACIAEwALgAXwBDQCIAIkAUYApQBbgD8AMUAbgA6gB8gEXgJEAXmAywkADAAuAGQBdBSBOABUADIAHIAPgBAADIAGkARABFACYAE8AKQAXwA5gB-AENAIgAiQBRgClAFiALcAaMA-wD9AIsASkAuYBigDaAG4APQAi8BIgCdgFDgLzAZIAywBrIDggHjgQhKACgALgAkACsAGQAsQBgADyALoAaaBAwAAA.YAAAAAAAAAAA&cmpStat=1&ccpaStat=0&ref=https%3A%2F%2Fwww.ksta.de%2Fhtml%2Fdumont-consent%2Findex.html%3Fparam%3DeyJyZWRpcmVjdFVybCI6Ii9yYXRnZWJlci9nZXN1bmRoZWl0L2hhdXNhZXJ6dGluLWVya2xhZXJ0LW1pbGRlci12ZXJsYXVmLWJlaS1laW5lci1vbWlrcm9uLWluZmVrdGlvbi0tLXdhcy1oZWlzc3QtZGFzLS0zOTM5MzE2ND9jYj0xNjQzMDgyNTMxNDU5JmRtY2lkPXNtX3R3X3B1IiwicmVmZXJyZXIiOiIiLCJzdWJkb21haW4iOiJ3d3cifQ%3D%3D +odb.outbrain.com/ +# Blocked URL = https://elsa.memoinsights.com/t?pid=62012a7a19351c07620394e0&url=https%3A%2F%2Farstechnica.com%2Ftech-policy%2F2022%2F08%2Fthe-women-calling-out-apples-handling-of-misconduct-claims%2F&author%5B%5D=Financial%20Times&title=The%20women%20calling%20out%20Apple%E2%80%99s%20handling%20of%20misconduct%20claims&date=2022-08-04T13%3A39%3A42Z&referrer=&ref_url=&page_url=https%3A%2F%2Farstechnica.com%2Ftech-policy%2F2022%2F08%2Fthe-women-calling-out-apples-handling-of-misconduct-claims%2F%3Fcomments%3D1&cb=MEMO.API.callbacks.cbakynzcplf&v=v3.0.6&t=5000&e=5000&s=7362 +elsa.memoinsights.com/t +# Blocked URL = https://ups.xplosion.de/ctx?event_id=ctx_json&_sid=24141&hostSiteUrl=https://www.presseportal.de/blaulicht/pm/12415/5302821&referrer=&userAgent=Mozilla/5.0%20(X11;%20FreeBSD%20amd64;%20rv:91.0)%20Gecko/20100101%20Firefox/91.0&userLang=en-CA&gdpr=1&gdpr_consent=[...] +ups.xplosion.de/ +# Blocked URL = https://s.cpx.to/fire.js?pid=13058&ref=https%3A%2F%2Fexplosm.net%2Fcomics%2Fnewyears-2008&url=https%3A%2F%2Fexplosm.net%2Fcomics%2Fdave-shortnew-8%23comic&hn_ver=57&fid=715e92a0-477c-4ef4-b655-6ed9edc0e94b&gcv=CPuOj0APuOj0AAKAvAENDFCsAP_AAH_AAAwIJbtX_H__bW9r8f5_aft0eY1P9_j77uQzDhfNk-4F3L_W_JwX52E7NF36tq4KmR4Eu3LBIUNlHNHUTVmwaokVryHsak2cpTNKJ6BEkHMRO2dYGF5umxtjeQKY5_p_d3fx2D-t_dv-39z3z81Xn3dZf-_0-PCdU5_9Dfn9fRfb-9IL9_78v8v8_9_rk2_eX_3_79_7_H9-f_84JcAEmGrcQBdmUODNoGEUCIEYVhARQKACCgGFogIAHBwU7IwCfWESAFAKAIwIgQ4AoyIBAAAJAEhEAEgRYIAAABAIAAQAIBEIAGBgEFABYCAQAAgOgYohQACBIQJEREQpgQFQJBAS2VCCUF0hphAFWWAFAIjYKABEEgIrAAEBYOAYIkBKxYIEmINogAGAFAKJUK1FJ6aAhYzMAAAA.YAAAAAAAAAAA +s.cpx.to/ +# Blocked URL = https://track.venatusmedia.com/dual/track_enc +track.venatusmedia.com/ +# Blocked URL = https://o2.mouseflow.com/init?v=17.96&p=bd33e8bf-4f45-437f-9bda...&s=a6ac920f1ba63b3...&page=090...2f0d2&ret=0&u=62...fa1&href=https%3A%2F%2Fauth.leaseweb.com%2FloginCustomer&url=auth.leaseweb.com%2Flogincustomer&ref=https%3A%2F%2Fsecure.leaseweb.com%2F&title=Leaseweb%20Authorization%20Server&res=1366x768&tz=-60&to=0&dnt=1&ori=&dw=1362&dh=621&time=727&pxr=1&gdpr=1 +o2.mouseflow.com/ +# Blocked URL = https://secure-eu.nmrodam.com/cgi-bin/gn?prd=dcr&ci=de-910483&ch=de-910483_b01__vermischtes__P&asn=_vermischtes_&fp_id=&fp_cr_tm=&fp_acc_tm=&fp_emm_tm=&ve_id=&sessionId=vu0qoho115ipzkihhkxsztnvalijv1695918791&prv=1&c6=vc,b01&ca=NA&c13=asid,PE3FB63C1-D0BE-4D42-95B5-7B383C43C5F8&c32=segA,a_vermischtes_article&c33=segB,CPI&c34=segC,Firefox%20102&c15=apn,&sup=1&segment2=&segment1=&forward=0&plugv=&playerv=&ad=0&cr=D&c9=devid,&enc=true&c1=nuid,999&at=timer&rt=text&c16=sdkv,bj.6.0.0&c27=cln,30&crs=&lat=&lon=&c29=plid,16959187913574064&c30=bldv,6.0.0.673&st=dcragf&c7=osgrp,&c8=devgrp,&c10=plt,&c40=adbid,&c14=osver,NA&c26=dmap,1&dd=&hrd=&wkd=&c35=adrsid,&c36=cref1,&c37=cref2,&c11=agg,1&c12=apv,&c51=adl,0&c52=noad,0&pc=NA&c53=fef,n&c54=oad,&c55=cref3,&c57=adldf,2&ai=247660590&c3=st,c&c64=starttm,1695918795&adid=247660590&c58=isLive,false&c59=sesid,&c61=createtm,1695918850&c63=pipMode,&uoo=&c68=bndlid,&nodeTM=&logTM=&c73=phtype,&c74=dvcnm,&c76=adbsnid,&c44=progen,&davty=2&si=https%3A%2F%2Fwww.welt.de%2Fvermischtes%2Farticle247660590%2FBerlin-Mann-traegt-islamische-Muetze-als-Mode-Accessoire-und-wird-attackiert.html%3Fsource%3Dpuerto-reco-2_ABC-V32.7.C_already_read&c66=mediaurl,&sdd=&c62=sendTime,1695918850&rnd=889711 +secure-eu.nmrodam.com/ {+block{Might be a web-bug that is an image.} -handle-as-empty-document +handle-as-image} #MASTER# BLOCK-REFERRER: http://versiontracker.com and many others. 10/20/06 @@ -1022,6 +1033,8 @@ eu-tlp01.kameleoon.eu/ #MASTER# BLOCK-REFERRER: https://arstechnica.com/gadgets/2021/04/xinuos-finishes-picking-up-scos-mantle-by-suing-red-hat-and-ibm/ # Blocked URL = https://fpa-events.arstechnica.com/plogger/?rand=1617351238373&plid=36453403&idsite=arstechnica.com&url=https%3A%2F%2Farstechnica.com%2Fgadgets%2F2021%2F04%2Fxinuos-finishes-picking-up-scos-mantle-by-suing-red-hat-and-ibm%2F&urlref=&screen=1366x768%7C1366x768%7C24&data=%7B%7D&sid=1&surl=https%3A%2F%2Farstechnica.com%2Fgadgets%2F2021%2F04%2Fxinuos-finishes-picking-up-scos-mantle-by-suing-red-hat-and-ibm%2F&sref=&sts=1617351085989&slts=0&date=Fri+Apr+02+2021+10%3A13%3A58+GMT%2B0200+(CEST)&action=heartbeat&inc=4&tt=4371&pvid=50890683&u=pid%3Df5e180d3fca67aa29521f1510e59850b fpa-events.arstechnica.com/ +# Blocked URL = https://i.clean.gg/1a +i.clean.gg/ ############################################################################# @@ -1953,7 +1966,7 @@ www.peereboom.us/adsuck/ #MASTER# REMARKS: Ebay enlarge picture function doesn't work. # URL = http://include.ebaystatic.com/v4js/en_GB/e637i/SYS-LIGER_Omniture_e637i10177164_5_en_GB.js include.ebaystatic.com/.*omniture.*\.js -#MASTER# REMARKS: Allow Yahoo news and mail javascipt pages +#MASTER# REMARKS: Allow Yahoo news and mail javascript pages # URL = http://l.yimg.com/d/combo?news/p/common/generic/news/p/common/generic/popular-searches-min-12622.js&news/p/common/generic/ads-min-11050.js&news/p/common/generic/foundation/popup-min-12622.js .yimg.com/d/combo\? #MASTER# REMARKS: Page formatting problems when .css files are blocked @@ -2042,6 +2055,14 @@ belco24.de/ metrics.sr.ht/ # URL = https://nlnet.nl/logo/banner.png nlnet.nl/ +# URL = https://adv-archiv.dfn-cert.de/adv/2021-2222/ +adv-archiv.dfn-cert.de/ +# URL = https://pinkstinks.de/werbung-ohne-diskriminierung/ +pinkstinks.de/ +# URL = https://www.eff.org/files/styles/teaser/public/banner_library/repro-rights-hd-3b.jpg?itok=uCnCHOj5 +.eff.org/ +# URL = https://www.datenschmutz.de/banner-square.png +.datenschmutz.de/ ############################################################################# @@ -2265,6 +2286,25 @@ idp.springer.com/ .microsoftonline.com/ # URL = https://www.zeit.de/zustimmung?url=https%3A%2F%2Fwww.zeit.de%2F2021%2F46%2Fchristian-drosten-coronavirus-virologie-pandemie-wissenschaft-impfung%2Fseite-4 .zeit.de/zustimmung +# URL = https://07.hagalil.com/www/delivery/ajs.php?zoneid=14&cb=35782002576&charset=UTF-8&loc=https%3A//www.hagalil.com/2021/12/rechtes-von-a-z-5/ +.hagalil.com/ +# URL = https://twitter.com/mariowitte/status/1479481418887663624?ref_src=twsrc%5Etfw%7Ctwcamp%5Etweetembed%7Ctwterm%5E1479481418887663624%7Ctwgr%5E%7Ctwcon%5Es1_&ref_url=https%3A%2F%2Ffuturezone.at%2Fprodukte%2Fcanon-toner-kopierschutz-gefaelscht-chips-chipmangel%2F401865218 +twitter.com/.*/status/ +#MASTER# REMARKS: There are lots of Nitter instances with onion addresses ... +# URL = http://vfaomgh4jxphpbdfizkm5gbtjahmei234giqj4facbwhrfjtcldauqad.onion/mariowitte/status/1479481418887663624?ref_src=twsrc%5Etfw%7Ctwcamp%5Etweetembed%7Ctwterm%5E1479481418887663624%7Ctwgr%5E%7Ctwcon%5Es1_&ref_url=https%3A%2F%2Ffuturezone.at%2Fprodukte%2Fcanon-toner-kopierschutz-gefaelscht-chips-chipmangel%2F401865218 +.onion/.*/status/ +# URL = https://admin.gandi.net/dashboard/api/v5/login?redirect=https%3A%2F%2Fadmin.gandi.net%2Fdashboard%3Flocale%3Dde +.gandi.net/ +# URL = https://launchpad.net/+openid-callback?starting_url=https%3A%2F%2Fanswers.launchpad.net%2Fdvdbackup%2F%2Bquestion%2F702512&janrain_nonce=2022-08-03T13%3A11%3A13ZnhmdLK... +launchpad.net/ +# URL = https://consent.youtube.com/m?continue=https%3A%2F%2Fwww.youtube.com%2Fplaylist%3Flist%3DPLeF8ZihVdpFfttOtsot131byFrDVztt8T%26cbrd%3D1%26cbrd%3D1&gl=AT&m=0&pc=yt&hl=en&src=1 +consent.youtube.com/ +# URL = https://services.akteneinsichtsportal.de/oauth/authorize?client_id=ae-portal&redirect_uri=https%3A%2F%2Fwww.akteneinsichtsportal.de%2Fweb%2Fguest%2Faktenuebersicht&response_type=code&scope=read&state=0yT8EN +services.akteneinsichtsportal.de/ +# URL = https://bpe-online.de/wp-content/plugins/pdf-viewer-for-elementor/assets/pdfjs/web/viewer.html?file=https://bpe-online.de/wp-content/uploads/2020/11/HarmReductionLeitfadenzumrisikoarmenAbsetzenvonPsychopharmaka2EdOnline.pdf&embedded=true +/wp-content/plugins/pdf-viewer-for-elementor +# URL = https://syndication.twitter.com/i/jot/embeds?l=%7B%22_category_%22%3A%22tfw_client_event%22%2C%22triggered_on%22%3A1709138733184%2C%22event_namespace%22%3A%7B%22client%22%3A%22tfw%22%2C%22page%22%3A%22tweet%22%2C%22action%22%3A%22seen%22%2C%22section%22%3A%22main%22%2C%22component%22%3A%22privacy-notice%22%7D%2C%22context%22%3A%22horizon%22%2C%22client_version%22%3A%222615f7e52b7e0%3A1702314776716%22%2C%22dnt%22%3Afalse%2C%22widget_id%22%3A%22twitter-widget-6%22%2C%22widget_origin%22%3A%22https%3A%2F%2Fwww.danisch.de%2Fblog%2F2024%2F01%2F31%2Fwas-ist-faschismus%2F%22%2C%22widget_frame%22%3A%22false%22%2C%22widget_partner%22%3A%22%22%2C%22widget_site_screen_name%22%3A%22%22%2C%22widget_site_user_id%22%3A%22%22%2C%22widget_creator_screen_name%22%3A%22%22%2C%22widget_creator_user_id%22%3A%22%22%2C%22widget_iframe_version%22%3A%22da31e07063cce%3A1708130301554%22%2C%22item_ids%22%3A%5B%221752320689309143351%22%5D%2C%22item_details%22%3A%7B%221752320689309143351%22%3A%7B%22item_type%22%3A0%7D%7D%7D +syndication.twitter.com/ {+redirect{s@.*url=@http://@} -block} # Sticky Actions = +redirect -block @@ -2436,11 +2476,17 @@ kggp.de/ .plasmaservice.de/ # URL = http://www.black-mosquito.org/index.php/kein-mensch-ist-illegal-fight-racism-now-40-aufkleber.html .black-mosquito.org/ +# URL = https://freebsdfoundation.org/about-us/board-of-directors/ +freebsdfoundation.org/ +# URL = https://freiheitsfoo.de/2022/10/03/wahlplakat-show-nds-ltw-2022/ +.freiheitsfoo.de/ {-filter{banners-by-link}} # Sticky Actions = -filter{banners-by-link} # URL = http://www.encyclopediadramatica.com/Advertisement .encyclopediadramatica.com/ +# URL = https://www.eff.org/deeplinks/2022/03/eff-investigation-mystery-gps-tracker-supporters-car +.eff.org/ #---------------------------------------------------------------------------- # These don't work without the referrer information: diff --git a/default.filter b/default.filter index e19f6cf6..aaf70a3e 100644 --- a/default.filter +++ b/default.filter @@ -4,7 +4,7 @@ # # Purpose : Rules to process the content of web pages # -# Copyright : Written by and Copyright (C) 2001-2021 the +# Copyright : Written by and Copyright (C) 2001-2022 the # Privoxy team. https://www.privoxy.org/ # # This program is free software; you can redistribute it @@ -718,6 +718,17 @@ s@(background:\#[a-f\d]{3})( url\(\"http://www.blogblog.com/rounders[^\"]*\"\).* s@([^<]*)(?:\.\.\.)?\s*\s*\ (\s*\1)@$2@ig + +################################################################################# +# +# bundeswehr.de: Hide the cookie and privacy info banner on bundeswehr.de. +# +# The relevant parts of the page work without accepting cookies. +# +################################################################################# +FILTER: bundeswehr.de Hide the cookie and privacy info banner on bundeswehr.de +s@