-------------------------------------------------------------------- ChangeLog for Privoxy -------------------------------------------------------------------- *** Version 3.0.26 stable *** - Bug fixes: - accept_connection(): Fix crashes with "listen-addr :8118" After jbsockets.c v1.136 a valid text representation of the host address is required for the $listen-address variable. If no host address has been specified, use an empty string to prevent NULL pointer dereferences. The problem was reported by Marvin Renich in Debian bug #834941, the offending commit was tracked down by Roland in SF Bug #902. - General improvements: - Log when privoxy is toggled on or off via cgi interface. - Highlight the "Info: Now toggled " on/off log message in the windows log viewer. - Highlight the loading actions/filter file log message in the windows log viewer - Mention client-specific tags on the toggle page as a potentionally more appropriate alternative - Documentation improvements: - Update download section on the homepage The downloads are available from the website now. - Add sponsor FAQ - Remove obsolete reference to mailing lists hosted at SourceForge - Update the "Before the Release" section of the developer manual - Infrastructure improvements: - Add perl script to generate an RSS feed for the packages Submitted by "Unknown": https://lists.privoxy.org/pipermail/privoxy-devel/2016-July/000068.html - Build system improvements: - strptime.h: fix a compiler warning about ambiguous else - configure.in: Check for Docbook goo on the BSDs as well - GNUMakefile.in: Let the dok-user target remove temporary files *** Version 3.0.25 beta *** - Bug fixes: - Always use the current toggle state for new requests. Previously new requests on reused connections inherited the toggle state from the previous request even though the toggle state could have changed. Reported by Robert Klemme. - Fixed two buffer-overflows in the (deprecated) static pcre code. These bugs are not considered security issues as the input is trusted. Found with afl-fuzz and ASAN. - General improvements: - Added support for client-specific tags which allow Privoxy admins to pre-define tags that are set for all requests from clients that previously opted in through the CGI interface. They are useful in multi-user setups where admins may want to allow users to disable certain actions and filters for themselves without affecting others. In single-user setups they are useful to allow more fine-grained toggling. For example to disable request blocking while still crunching cookies, or to disable experimental filters only. This is an experimental feature, the syntax and behaviour may change in future versions. Sponsored by Robert Klemme. - Dynamic filters and taggers now support a $listen-address variable which contains the address the request came in on. For external filters the variable is called $PRIVOXY_LISTEN_ADDRESS. Original patch contributed by pursievro. - Add client-header-tagger 'listen-address'. - Include the listen-address in the log message when logging new requests. Patch contributed by pursievro. - Turn invalid max-client-connections values into fatal errors. - The show-status page now shows whether or not dates before 1970 and after 2038 are expected to be handled properly. This is mainly useful for Privoxy-Regression-Test but could also come handy when dealing with time-related support requests. - On Mac OS X the thread id in log messages are more likely to be unique now. - When complaining about missing filters, the filter type is logged as well. - A couple of harmless coverity warnings were silenced (CID #161202, CID #161203, CID #161211). - Action file improvements: - Filtering is disabled for Range requests to let download resumption and Windows updates work with the default configuration. - Unblock ".ardmediathek.de/". Reported by ThTomate in #932. - Documentation improvements: - Add FAQ entry for crashes caused by memory limits. - Remove obsolete FAQ entry about a bug in PHP 4.2.3. - Mention the new mailing lists were appropriate. As the archives have not been migrated, continue to mention the archives at SF in the contacting section for now. - Note that the templates should be adjusted if Privoxy is running as intercepting proxy without getting all requests. - A bunch of links were converted to https://. - Rephrase onion service paragraph to make it more obvious that Tor is involved and that the whole website (and not just the homepage) is available as onion service. - Streamline the "More information" section on the homepage further by additionally ditching the link to the 'See also' section of the user manual. The section contains mostly links that are directly reachable from the homepage already and the rest is not significant enough to get a link from the homepage. - Change the add-header{} example to set the DNT header and use a complete section to make copy and pasting more convenient. Add a comment to make it obvious that adding the header is not recommended for obvious reasons. Using the DNT header as example was suggested by Leo Wzukw. - Streamline the support-and-service template Instead of linking to the various support trackers (whose URLs hopefully change soon), link to the contact section of the user manual to increase the chances that users actually read it. - Add a FAQ entry for tainted sockets. - More sections in the documentation have stable URLs now. - FAQ: Explain why 'ping config.privoxy.org' is not expected to reach a local Privoxy installation. - Note that donations done through Zwiebelfreunde e.V. currently can't be checked automatically. - Updated section regarding starting Privoxy under OS X. - Use dedicated start instructions for FreeBSD and ElectroBSD. - Removed release instructions for AIX. They haven't been working for years and unsurprisingly nobody seems to care. - Removed obsolete reference to the solaris-dist target. - Updated the release instructions for FreeBSD. - Removed unfinished release instructions for Amiga OS and HP-UX 11. - Added a pointer to the Cygwin Time Machine for getting the last release of Cygwin version 1.5 to use for building Privoxy on Windows. - Various typos have been fixed. - Infrastructure improvements: - The website is no longer hosted at SourceForge and can be reached through https now. - The mailing lists at SourceForge have been deprecated, you can subscribe to the new ones at: https://lists.privoxy.org/ - Migrating the remaining services from SourceForge is work in progress (TODO list item #53). - Build system improvements: - Add configure argument to optimistically redefine FD_SETSIZE with the intent to change the maximum number of client connections Privoxy can handle. Only works with some libcs. Sponsored by Robert Klemme. - Let the tarball-dist target skip files in ".git". - Let the tarball-dist target work in cwds other than current. - Make the 'clean' target faster when run from a git repository. - Include tools in the generic distribution. - Let the gen-dist target work in cwds other than current. - Sort find output that is used for distribution tarballs to get reproducible results. - Don't add '-src' to the name of the tar ball generated by the gen-dist target. The package isn't a source distribution but a binary package. While at it, use a variable for the name to reduce the chances that the various references get out of sync and fix the gen-upload target which was looking in the wrong directory. - Add regression-tests.action to the files that are distributed. - The gen-dist target which was broken since 2002 (r1.92) has been fixed. - Remove genclspec.sh which has been obsolete since 2009. - Remove obsolete reference to Redhat spec file. - Remove the obsolete announce target which has been commented out years ago. - Let rsync skip files if the checksums match. - Privoxy-Regression-Test: - Add a "Default level offset" directive which can be used to change the default level by a given value. This directive affects all tests located after it until the end of the file or a another "Default level offset" directive is reached. The purpose of this directive is to make it more convenient to skip similar tests in a given file without having to remove or disable the tests completely. - Let test level 17 depend on FEATURE_64_BIT_TIME_T instead of FEATURE_PTHREAD which has no direct connection to the time_t size. - Fix indentation in perldoc examples. - Don't overlook directives in the first line of the action file. - Bump version to 0.7. - Fix detection of the Privoxy version now that https:// is used for the website. *** Version 3.0.24 stable *** - Security fixes (denial of service): - Prevent invalid reads in case of corrupt chunk-encoded content. CVE-2016-1982. Bug discovered with afl-fuzz and AddressSanitizer. - Remove empty Host headers in client requests. Previously they would result in invalid reads. CVE-2016-1983. Bug discovered with afl-fuzz and AddressSanitizer. - Bug fixes: - When using socks5t, send the request body optimistically as well. Previously the request body wasn't guaranteed to be sent at all and the error message incorrectly blamed the server. Fixes #1686 reported by Peter Müller and G4JC. - Fixed buffer scaling in execute_external_filter() that could lead to crashes. Submitted by Yang Xia in #892. - Fixed crashes when executing external filters on platforms like Mac OS X. Reported by Jonathan McKenzie on ijbswa-users@. - Properly parse ACL directives with ports when compiled with HAVE_RFC2553. Previously the port wasn't removed from the host and in case of 'permit-access 127.0.0.1 example.org:80' Privoxy would try (and fail) to resolve "example.org:80" instead of example.org. Reported by Pak Chan on ijbswa-users@. - Check requests more carefully before serving them forcefully when blocks aren't enforced. Privoxy always adds the force token at the beginning of the path, but would previously accept it anywhere in the request line. This could result in requests being served that should be blocked. For example in case of pages that were loaded with force and contained JavaScript to create additionally requests that embed the origin URL (thus inheriting the force prefix). The bug is not considered a security issue and the fix does not make it harder for remote sites to intentionally circumvent blocks if Privoxy isn't configured to enforce them. Fixes #1695 reported by Korda. - Normalize the request line in intercepted requests to make rewriting the destination more convenient. Previously rewrites for intercepted requests were expected to fail unless $hostport was being used, but they failed "the wrong way" and would result in an out-of-memory message (vanilla host patterns) or a crash (extended host patterns). Reported by "Guybrush Threepwood" in #1694. - Enable socket lingering for the correct socket. Previously it was repeatedly enabled for the listen socket instead of for the accepted socket. The bug was found by code inspection and did not cause any (reported) issues. - Detect and reject parameters for parameter-less actions. Previously they were silently ignored. - Fixed invalid reads in internal and outdated pcre code. Found with afl-fuzz and AddressSanitizer. - Prevent invalid read when loading invalid action files. Found with afl-fuzz and AddressSanitizer. - Windows build: Use the correct function to close the event handle. It's unclear if this bug had a negative impact on Privoxy's behaviour. Reported by Jarry Xu in #891. - In case of invalid forward-socks5(t) directives, use the correct directive name in the error messages. Previously they referred to forward-socks4t failures. Reported by Joel Verhagen in #889. - General improvements: - Set NO_DELAY flag for the accepting socket. This significantly reduces the latency if the operating system is not configured to set the flag by default. Reported by Johan Sintorn in #894. - Allow to build with mingw x86_64. Submitted by Rustam Abdullaev in #135. - Introduce the new forwarding type 'forward-webserver'. Currently it is only supported by the forward-override{} action and there's no config directive with the same name. The forwarding type is similar to 'forward', but the request line only contains the path instead of the complete URL. - The CGI editor no longer treats 'standard.action' special. Nowadays the official "standards" are part of default.action and there's no obvious reason to disallow editing them through the cgi editor anyway (if the user decided that the lack of authentication isn't an issue in her environment). - Improved error messages when rejecting intercepted requests with unknown destination. - A couple of log messages now include the number of active threads. - Removed non-standard Proxy-Agent headers in HTTP snipplets to make testing more convenient. - Include the error code for pcre errors Privoxy does not recognize. - Config directives with numerical arguments are checked more carefully. - Privoxy's malloc() wrapper has been changed to prevent zero-size allocations which should only occur as the result of bugs. - Various cosmetic changes. - Action file improvements: - Unblock ".deutschlandradiokultur.de/". Reported by u302320 in #924. - Add two fast-redirect exceptions for "yandex.ru". - Disable filter{banners-by-size} for ".plasmaservice.de/". - Unblock "klikki.fi/adv/". - Block requests for "resources.infolinks.com/". Reported by "Black Rider" on ijbswa-users@. - Block a bunch of criteo domains. Reported by Black Rider. - Block "abs.proxistore.com/abe/". Reported by Black Rider. - Disable filter{banners-by-size} for ".black-mosquito.org/". - Disable fast-redirects for "disqus.com/". - Documentation improvements: - FAQ: Explicitly point fingers at ASUS as an example of a company that has been reported to force malware based on Privoxy upon its customers. - Correctly document the action type for a bunch of "multi-value" actions that were incorrectly documented to be "parameterized". Reported by Gregory Seidman on ijbswa-users@. - Fixed the documented type of the forward-override{} action which is obviously 'parameterized'. - Website improvements: - Users who don't trust binaries served by SourceForge can get them from a mirror. Migrating away from SourceForge is planned for 2016 (TODO list item #53). - The website is now available as onion service (http://jvauzb4sb3bwlsnc.onion/). *** Version 3.0.23 stable *** - Bug fixes: - Fixed a DoS issue in case of client requests with incorrect chunk-encoded body. When compiled with assertions enabled (the default) they could previously cause Privoxy to abort(). Reported by Matthew Daley. CVE-2015-1380. - Fixed multiple segmentation faults and memory leaks in the pcrs code. This fix also increases the chances that an invalid pcrs command is rejected as such. Previously some invalid commands would be loaded without error. Note that Privoxy's pcrs sources (action and filter files) are considered trustworthy input and should not be writable by untrusted third-parties. CVE-2015-1381. - Fixed an 'invalid read' bug which could at least theoretically cause Privoxy to crash. So far, no crashes have been observed. CVE-2015-1382. - Compiles with --disable-force again. Reported by Kai Raven. - Client requests with body that can't be delivered no longer cause pipelined requests behind them to be rejected as invalid. Reported by Basil Hussain. - General improvements: - If a pcrs command is rejected as invalid, Privoxy now logs the cause of the problem as text. Previously the pcrs error code was logged. - The tests are less likely to cause false positives. - Action file improvements: - '.sify.com/' is no longer blocked. Apparently it is not actually a pure tracking site (anymore?). Reported by Andrew on ijbswa-users@. - Unblock banners on .amnesty.de/ which aren't ads. - Documentation improvements: - The 'Would you like to donate?' section now also contains a "Paypal" address. - The list of supported operating systems has been updated. - The existence of the SF support and feature trackers has been deemphasized because they have been broken for months. Most of the time the mailing lists still work. - The claim that default.action updates are sometimes released on their own has been removed. It hasn't happened in years. - Explicitly mention that Tor's port may deviate from the default when using a bundle. Requested by Andrew on ijbswa-users@. *** Version 3.0.22 stable *** - Bug fixes: - Fixed a memory leak when rejecting client connections due to the socket limit being reached (CID 66382). This affected Privoxy 3.0.21 when compiled with IPv6 support (on most platforms this is the default). CVE-2015-1030. - Fixed an immediate-use-after-free bug (CID 66394) and two additional unconfirmed use-after-free complaints made by Coverity scan (CID 66391, CID 66376). CVE-2015-1031. - Actually show the FORCE_PREFIX value on the show-status page. - Properly deal with Keep-Alive headers with timeout= parameters If the timeout still can't be parsed, use the configured timeout instead of preventing the client from keeping the connection alive. Fixes #3615312/#870 reported by Bernard Guillot. - Not using any filter files no longer results in warning messages unless an action file is referencing header taggers or filters. Reported by Stefan Kurtz in #3614835. - Fixed a bug that prevented Privoxy from reusing some reusable connections. Two bit masks with different purpose unintentionally shared the same bit. - A couple of additional bugs were discovered by Coverity Scan. The fixes that are not expected to affect users are not explicitly mentioned here, for details please have a look at the CVS logs. - General improvements: - Introduced negative tag patterns NO-REQUEST-TAG and NO-RESPONSE-TAG. They apply if no matching tag is found after parsing client or server headers. - Add support for external filters which allow to process the response body with a script or program written in any language the platform supports. External filters are enabled with +external-filter{} after they have been defined in one of the filter files with a header line starting with "EXTERNAL-FILTER:". External filter support is experimental, not compiled by default and known not to work on all platforms. - Add support for the 'PATCH' method as defined in RFC5789. - Reject requests with unsupported Expect header values. Fixes a couple of Co-Advisor tests. - Normalize the HTTP-version in forwarded requests and responses. This is an explicit RFC 2616 MUST and RFC 7230 mandates that intermediaries send their own HTTP-version in forwarded messages. - Server 'Keep-Alive' headers are no longer forwarded. From a user's point of view it doesn't really matter, but RFC 2616 (obsolete) mandates that the header is removed and this fixes a Co-Advisor complaint. - Change declared template file encoding to UTF-8. The templates already used a subset of UTF-8 anyway and changing the declaration allows to properly display UTF-8 characters used in the action files. This change may require existing action files with ISO-8859-1 characters that aren't valid UTF-8 to be converted to UTF-8. Requested by Sam Chen in #582. - Do not pass rejected keep-alive timeouts to the server. It might not have caused any problems (we know of), but doing the right thing shouldn't hurt either. - Let log_error() use its own buffer size #define to make changing the log buffer size slightly less inconvenient. - Turned single-threaded into a "proper" toggle directive with arguments. - CGI templates no longer enforce new windows for some links. - Remove an undocumented workaround ('HOST' header removal) for an Apple iTunes bug that according to #729900 got fixed in 2003. - Action file improvements: - The pattern 'promotions.' is no longer being blocked. Reported by rakista in #3608540. - Disable fast-redirects for .microsofttranslator.com/. - Disable filter{banners-by-size} for .dgb-tagungszentren.de/. - Add adn.speedtest.net as a site-specific unblocker. Support request #3612908. - Disable filter{banners-by-size} for creativecommons.org/. - Block requests to data.gosquared.com/. Reported by cbug in #3613653. - Unblock .conrad./newsletter/. Reported by David Bo in #3614238. - Unblock .bundestag.de/. - Unblock .rote-hilfe.de/. - Disable fast-redirects for .facebook.com/plugins/like.php. - Unblock Stackexchange popup URLs that aren't used to serve ads. Reported by David Wagner in #3615179. - Disable fast-redirects for creativecommons.org/. - Unblock .stopwatchingus.info/. - Block requests for .adcash.com/script/. Reported by Tyrexionibus in #3615289. - Disable HTML filters if the response was tagged as JavaScript. Filtering JavaScript code with filters intended to deal with HTML is usually a waste of time and, more importantly, may break stuff. - Use a custom redirect{} for .washingtonpost.com/wp-apps/imrs\.php\?src= Previously enabling the 'Advanced' settings (or manually enabling +fast-redirects{}) prevented some images from being loaded properly. - Unblock "adina*." Fixes #919 reported by Morton A. Goldberg. - Block '/.*DigiAd'. - Unblock 'adele*.'. Reported by Adele Lime in #1663. - Disable banners-by-size for kggp.de/. - Filter file improvements & bug fixes: - Decrease the chances that js-annoyances creates invalid JavaScript. Submitted by John McGowan on ijbswa-users@. - Let the msn filter hide 'related' ads again. - Remove a stray '1' in the 'html-annoyances' filter. - Prevent img-reorder from messing up img tags with empty src attributes. Fixes #880 reported by Duncan. - Documentation improvements: - Updated the 'Would you like to donate?' section. - Note that invalid forward-override{} parameter syntax isn't detected until the parameter is used. - Add another +redirect{} example: a shortcut for illumos bugs. - Make it more obvious that many operating systems support log rotation out of the box. - Fixed dead links. Reported by Mark Nelson in #3614557. - Rephrased the 'Why is the configuration so complicated?' answer to be slightly less condescending. Anonymously suggested in #3615122. - Be more explicit about accept-intercepted-requests's lack of MITM support. - Make 'demoronizer' FAQ entries more generic. - Add an example hostname to the --pre-chroot-nslookup description. - Add an example for a host pattern that matches an IP address. - Rename the 'domain pattern' to 'host pattern' as it may contain IP addresses as well. - Recommend forward-socks5t when using Tor. It seems to work fine and modifying the Tor configuration to profit from it hasn't been necessary for a while now. - Add another redirect{} example to stress that redirect loops can and should be avoided. - The usual spelling and grammar fixes. Parts of them were reported by Reuben Thomas in #3615276. - Mention the PCRS option letters T and D in the filter section. - Clarify that handle-as-empty-doc-returns-ok is still useful and will not be removed without replacement. - Note that security issues shouldn't be reported using the bug tracker. - Clarify what Privoxy does if both +block{} and +redirect{} apply. - Removed the obsolete bookmarklets section. - Build system improvements: - Let --with-group properly deal with secondary groups. Patch submitted by Anatoly Arzhnikov in #3615187. - Fix web-actions target. - Add a web-faq target that only updates the FAQ on the webserver. - Remove already-commented-out non-portable DOSFILTER alternatives. - Remove the obsolete targets dok-put and dok-get. - Add a sf-shell target. *** Version 3.0.21 stable *** - Bug fixes: - On POSIX-like platforms, network sockets with file descriptor values above FD_SETSIZE are properly rejected. Previously they could cause memory corruption in configurations that allowed the limit to be reached. - Proxy authentication headers are removed unless the new directive enable-proxy-authentication-forwarding is used. Forwarding the headers potentially allows malicious sites to trick the user into providing them with login information. Reported by Chris John Riley. - Compiles on OS/2 again now that unistd.h is only included on platforms that have it. - General improvements: - The show-status page shows the FEATURE_STRPTIME_SANITY_CHECKS status. - A couple of assert()s that could theoretically dereference NULL pointers in debug builds have been relocated. - Added an LSB info block to the generic start script. Based on a patch from Natxo Asenjo. - The max-client-connections default has been changed to 128 which should be more than enough for most setups. - Action file improvements: - Block rover.ebay./ar.*\&adtype= instead of "/.*\&adtype=" which caused too man false positives. Reported by u302320 in #360284, additional feedback from Adam Piggott. - Unblock '.advrider.com/' and '/.*ADVrider'. Anonymously reported in #3603636. - Stop blocking '/js/slider\.js'. Reported by Adam Piggott in #3606635 and _lvm in #2791160. - Filter file improvements: - Added an iframes filter. - Documentation improvements: - The whole GPLv2 text is included in the user manual now, so Privoxy can serve it itself and the user can read it without having to wade through GPLv3 ads first. - Properly numbered and underlined a couple of section titles in the config that where previously overlooked due to a flaw in the conversion script. Reported by Ralf Jungblut. - Improved the support instruction to hopefully make it harder to unintentionally provide insufficient information when requesting support. Previously it wasn't obvious that the information we need in bug reports is usually also required in support requests. - Removed documentation about packages that haven't been provided in years. - Privoxy-Regression-Test: - Only log the test number when not running in verbose mode The position of the test is rarely relevant and it previously wasn't exactly obvious which one of the numbers was useful to repeat the test with --test-number. - GNUmakefile improvements: - Factor generate-config-file out of config-file to make testing more convenient. - The clean target now also takes care of patch leftovers. *** Version 3.0.20 beta *** - Bug fixes: - Client sockets are now properly shutdown and drained before being closed. This fixes page truncation issues with clients that aggressively pipeline data on platforms that otherwise discard already written data. The issue mainly affected Opera users and was initially reported by Kevin in #3464439, szotsaki provided additional information to track down the cause. - Fix latency calculation for shared connections (disabled by default). It was broken since their introduction in 2009. The calculated latency for most connections would be 0 in which case the timeout detection failed to account for the real latency. - Reject URLs with invalid port. Previously they were parsed incorrectly and characters between the port number and the first slash were silently dropped as shown by curl test 187. - The default-server-timeout and socket-timeout directives accept 0 as valid value. - Fix a race condition on Windows that could cause Privoxy to become unresponsive after toggling it on or off through the taskbar icon. Reported by Tim H. in #3525694. - Fix the compilation on Windows when configured without IPv6 support. - Fix an assertion that could cause debug builds to abort() in case of socks5 connection failures with "debug 2" enabled. - Fix an assertion that could cause debug builds to abort() if a filter contained nul bytes in the replacement text. - General improvements: - Significantly improved keep-alive support for both client and server connections. - New debug log level 65536 which logs all actions that were applied to the request. - New directive client-header-order to forward client headers in a different order than the one in which they arrived. - New directive tolerate-pipelining to allow client-side pipelining. If enabled (3.0.20 beta enables it by default), Privoxy will keep pipelined client requests around to deal with them once the current request has been served. - New --config-test option to let Privoxy exit after checking whether or not the configuration seems valid. The limitations noted in TODO #22 and #23 still apply. Based on a patch by Ramkumar Chinchani. - New limit-cookie-lifetime{} action to let cookies expire before the end of the session. Suggested by Rick Sykes in #1049575. - Increase the hard-coded maximum number of actions and filter files from 10 to 30 (each). It doesn't significantly affect Privoxy's memory usage and recompiling wasn't an option for all Privoxy users that reached the limit. - Add support for chunk-encoded client request bodies. Previously chunk-encoded request bodies weren't guaranteed to be forwarded correctly, so this can also be considered a bug fix although chunk-encoded request bodies aren't commonly used in the real world. - Add support for Tor's optimistic-data SOCKS extension, which can reduce the latency for requests on newly created connections. Currently only the headers are sent optimistically and only if the client request has already been read completely which rules out requests with large bodies. - After preventing the client from pipelining, don't signal keep-alive intentions. When looking at the response headers alone, it previously wasn't obvious from the client's perspective that no additional responses should be expected. - Stop considering client sockets tainted after receiving a request with body. It hasn't been necessary for a while now and unnecessarily causes test failures when using curl's test suite. - Allow HTTP/1.0 clients to signal interest in keep-alive through the Proxy-Connection header. While such client are rare in the real world, it doesn't hurt and couple of curl tests rely on it. - Only remove duplicated Content-Type headers when filters are enabled. If they are not it doesn't cause ill effects and the user might not want it. Downgrade the removal message to LOG_LEVEL_HEADER to clarify that it's not an error in Privoxy and is unlikely to cause any problems in general. Anonymously reported in #3599335. - Set the socket option SO_LINGER for the client socket. - Move several variable declarations to the beginning of their code block. It's required when compiling with gcc 2.95 which is still used on some platforms. Initial patch submitted by Simon South in #3564815. - Optionally try to sanity-check strptime() results before trusting them. Broken strptime() implementations have caused problems in the past and the most recent offender seems to be FreeBSD's libc (standards/173421). - When filtering is enabled, let Range headers pass if the range starts at the beginning. This should work around (or at least reduce) the video playback issues with various Apple clients as reported by Duc in #3426305. - Do not confuse a client hanging up with a connection time out. If a client closes its side of the connection without sending a request line, do not send the CLIENT_CONNECTION_TIMEOUT_RESPONSE, but report the condition properly. - Allow closing curly braces as part of action values as long as they are escaped. - On Windows, the logfile is now written before showing the GUI error message which blocks until the user acknowledges it. Reported by Adriaan in #3593603. - Remove an unreasonable parameter limit in the CGI interface. The new parameter limit depends on the memory available and is currently unlikely to be reachable, due to other limits in both Privoxy and common clients. Reported by Andrew on ijbswa-users@. - Decrease the chances of parse failures after requests with unsupported methods were sent to the CGI interface. - Action file improvements: - Remove the comment that indicated that updated default.action versions are released on their own. - Block 'optimize.indieclick.com/' and 'optimized-by.rubiconproject.com/' - Unblock 'adjamblog.wordpress.com/' and 'adjamblog.files.wordpress.com/'. Reported by Ryan Farmer in #3496116. - Unblock '/.*Bugtracker'. Reported by pwhk in #3522341. - Add test URLs for '.freebsd.org' and '.watson.org'. - Unblock '.urbandictionary.com/popular'. - Block '.adnxs.com/'. - Block 'farm.plista.com/widgetdata.php'. - Block 'rotation.linuxnewmedia.com/'. - Block 'reklamy.sfd.pl/'. Reported by kacperdominik in #3399948. - Block 'g.adspeed.net/'. - Unblock 'websupport.wdc.com/'. Reported by Adam Piggot in #3577851. - Block '/openx/www/delivery/'. - Disable fast-redirects for '.googleapis.com/'. - Block 'imp.double.net/'. Reported by David Bo in #3070411. - Block 'gm-link.com/' which is used for email tracking. Reported by David Bo in #1812733. - Verify that requests to "bwp." are blocked. URL taken from #1736879 submitted by Francois Marier. - Block '/.*bannerid='. Reported by Adam Piggott in #2975779. - Block 'cltomedia.info/delivery/' and '.adexprt.com/'. Anonymously reported in #2965254. - Block 'de17a.com/'. Reported by David Bo in #3061472. - Block 'oskar.tradera.com/'. Reported by David Bo in #3060596. - Block '/scripts/webtrends\.js'. Reported by johnd16 in #3002729. - Block requests for 'pool.*.adhese.com/'. Reported by johnd16 in #3002716. - Update path pattern for Coremetrics and add tests. Pattern and URLs submitted by Adam Piggott #3168443. - Enable +fast-redirects{check-decoded-url} for 'tr.anp.se/'. Reported by David Bo in #3268832. - Unblock '.conrad.se/newsletter/banners/'. Reported by David Bo in #3413824. - Block '.tynt.com/'. Reported by Dan Stahlke in #3421767. - Unblock '.bbci.co.uk/radio/'. Reported by Adam Piggott in #3569603. - Block requests to 'service.maxymiser.net/'. Reported by johnd16 in #3118401 (with a previous URL). - Disable fast-redirects for Google's "let's pretend your computer is infected" page. - Unblock '/.*download' to resolve actionsfile feedback #3498129. Submitted by Steven Kolins (soundcloud.com not working). - Unblock '.wlxrs.com/' which is required by hotmail.com. Fixes #3413827 submitted by David Bo. - Add two unblock patterns for popup radio and TV players. Submitted by Adam Piggott in #3596089. - Filter file improvements & bug fixes: - Add a referer tagger. - Reduce the likelihood that the google filter messes up HTML-generating JavaScript. Reported by Zeno Kugy in #3520260. - Documentation improvements: - Revised all OS X sections due to new packaging module (OSXPackageBuilder). - Update the list of supported operating systems to clarify that all Windows versions after 95 are expected to work and note that the platform-specific code for AmigaOS and QNX currently isn't maintained. - Update 'Signals' section, the only explicitly handled signals are SIGINT, SIGTERM and SIGHUP. - Add Haiku to the list of operating systems on which Privoxy is known to run. - Add DragonFly to the list of BSDs on which Privoxy is known to run. - Removed references to redhat-specific documentation set since it no longer exists. - Removed references to building PDFs since we no longer do so. - Multiple listen-address directives are supported since 3.0.18, correct the documentation to say so. - Remove bogus section about long and short being preferable to int. - Corrected some Internet JunkBuster references to Privoxy. - Removed references to www.junkbusters.com since it is no longer maintained. Reported by Angelina Matson. - Various grammar and spelling corrections - Add a client-header-tagger{} example for disabling filtering for range requests. - Correct a URL in the "Privoxy with Tor" FAQ. - Spell 'refresh-tags' correctly. Reported by Don in #3571927. - Sort manpage options alphabetically. - Remove an incorrect sentence in the toggle section. The toggle state doesn't affect whether or not the Windows version uses the tray icon. Reported by Zeno Kugy in #3596395. - Add new contributors since 3.0.19. - Log message improvements: - When stopping to watch a client socket due to pipelining, additionally log the socket number. - Log the client socket and its condition before closing it. This makes it more obvious that the socket actually gets closed and should help when diagnosing problems like #3464439. - In case of SOCKS5 failures, do not explicitly log the server's response. It hasn't helped so far and the response can already be logged by enabling "debug 32768" anyway. This reverts v1.81 and the follow-up bug fix v1.84. - Relocate the connection-accepted message from listen_loop() to serve(). This way it's printed by the thread that is actually serving the connection which is nice when grepping for thread ids in log files. - Code cleanups: - Remove compatibility layer for versions prior to 3.0 since it has been obsolete for more than 10 years now. - Remove the ijb_isupper() and ijb_tolower() macros from parsers.c since they aren't used in this file. - Removed the 'Functions declared include:' comment sections since they tend to be incomplete, incorrect and out of date and the benefit seems questionable. - Various comment grammar and comprehensibility improvements. - Remove a pointless fflush() call in chat(). Flushing all streams pretty much all the time for no obvious reason is ridiculous. - Relocate ijb_isupper()'s definition to project.h and get the ijb_tolower() definition from there, too. - Relocate ijb_isdigit()'s definition to project.h. - Rename ijb_foo macros to privoxy_foo. - Add malloc_or_die() which will allow to simplify code paths where malloc() failures don't need to be handled gracefully. - Add strdup_or_die() which will allow to simplify code paths where strdup() failures don't need to be handled gracefully. - Replace strdup() calls with strdup_or_die() calls where it's safe and simplifies the code. - Fix white-space around parentheses. - Add missing white-space behind if's and the following parentheses. - Unwrap a memcpy() call in resolve_hostname_to_ip(). - Declare pcrs_get_delimiter()'s delimiters[] static const. - Various optimisations to remove dead code and merge inefficient code structures for improved clarity, performance or code compactness. - Various data type corrections. - Change visibility of several code segments when compiling without FEATURE_CONNECTION_KEEP_ALIVE enabled for clarity. - In pcrs_get_delimiter(), do not use delimiters outside the ASCII range. Fixes a clang complaint. - Fix an error message in get_last_url() nobody is supposed to see. Reported by Matthew Fischer in #3507301. - Fix a typo in the no-zlib-support complaint. Patch submitted by Matthew Fischer in #3507304. - Shorten ssplit()'s prototype by removing the last two arguments. We always want to skip empty fields and ignore leading delimiters, so having parameters for this only complicates the API. - Use an enum for the type of the action value. - Rename action_name's member takes_value to value_type as it isn't used as boolean. - Turn family mismatches in match_sockaddr() into fatal errors. - Let enlist_unique_header() verify that the caller didn't pass a header containing either \r or \n. - Change the hashes used in load_config() to unsigned int. That's what hash_string() actually returns and using a potentially larger type is at best useless. - Use privoxy_tolower() instead of vanilla tolower() with manual casting of the argument. - Catch ssplit() failures in parse_cgi_parameters(). - Privoxy-Regression-Test: - Add an 'Overwrite condition' directive to skip any matching tests before it. As it has a global scope, using it is more convenient than clowning around with the Ignore directive. - Log to STDOUT instead of STDERR. - Include the Privoxy version in the output. - Various grammar and spelling corrections in documentation and code. - Additional tests for range requests with filtering enabled. - Tests with mostly invalid range request. - Add a couple of hide-if-modified-since{} tests with different date formats. - Cleaned up the format of the regression-tests.action file to match the format of default.action. - Remove the "Copyright" line from print_version(). When using --help, every line of screen space matters and thus shouldn't be wasted on things the user doesn't care about. - Privoxy-Log-Parser: - Improve the --statistics performance by skipping sanity checks for input that shouldn't affect the results anyway. Add a --strict-checks option that enables some of the checks again, just in case anybody cares. - The distribution of client requests per connection is included in the --statistic output. - The --accept-unknown-messages option has been removed and the behavior is now the default. - Accept and (mostly) highlight new log messages introduced with Privoxy 3.0.20. - uagen: - Bump generated Firefox version to 17. - GNUmakefile improvements: - The dok-tidy target no longer taints documents with a tidy-mark - Change RA_MODE from 0664 to 0644. Suggested by Markus Dittrich in #3505445. - Remove tidy's clean flag as it changes the scope of attributes. Link-specific colors end up being applied to all text. Reported by Adam Piggott in #3569551. - Leave it up to the user whether or not smart tags are inserted. - Let w3m itself do the line wrapping for the config file. It works better than fmt as it can honour pre tags causing less unintentional line breaks. - Ditch a pointless '-r' passed to rm to delete files. - The config-file target now requires less manual intervention and updates the original config. - Change WDUMP to generate ASCII. Add WDUMP_UTF8 to allow UTF-8 in the AUTHORS file so the names are right. - Stop pretending that lynx and links are supported for the documentation. - configure improvements: - On Haiku, do not pass -lpthread to the compiler. Haiku's pthreads implementation is contained in its system library, libroot, so no additional library needs to be searched. Patch submitted by Simon South in #3564815. - Additional Haiku-specific improvements. Disable checks intended for multi-user systems as Haiku is presently single-user. Group Haiku-specific settings in their own section, following the pattern for Solaris, OS/2 and AmigaOS. Add additional library-related settings to remove the need for providing configure with custom LDFLAGS. Submitted by Simon South in #3574538. *** Version 3.0.19 Stable *** - Bug fixes: - Prevent a segmentation fault when de-chunking buffered content. It could be triggered by malicious web servers if Privoxy was configured to filter the content and running on a platform where SIZE_T_MAX isn't larger than UINT_MAX, which probably includes most 32-bit systems. On those platforms, all Privoxy versions before 3.0.19 appear to be affected. To be on the safe side, this bug should be presumed to allow code execution as proving that it doesn't seems unrealistic. - Do not expect a response from the SOCKS4/4A server until it got something to respond to. This regression was introduced in 3.0.18 and prevented the SOCKS4/4A negotiation from working. Reported by qqqqqw in #3459781. - General improvements: - Fix an off-by-one in an error message about connect failures. - Use a GNUMakefile variable for the webserver root directory and update the path. Sourceforge changed it which broke various web-related targets. - Update the CODE_STATUS description. *** Version 3.0.18 Stable *** - Bug fixes: - If a generated redirect URL contains characters RFC 3986 doesn't permit, they are (re)encoded. Not doing this makes Privoxy versions from 3.0.5 to 3.0.17 susceptible to HTTP response splitting (CWE-113) attacks if the +fast-redirects{check-decoded-url} action is used. - Fix a logic bug that could cause Privoxy to reuse a server socket after it got tainted by a server-header-tagger-induced block that was triggered before the whole server response had been read. If keep-alive was enabled and the request following the blocked one was to the same host and using the same forwarding settings, Privoxy would send it on the tainted server socket. While the server would simply treat it as a pipelined request, Privoxy would later on fail to properly parse the server's response as it would try to parse the unread data from the first response as server headers for the second one. Regression introduced in 3.0.17. - When implying keep-alive in client_connection(), remember that the client didn't. Fixes a regression introduced in 3.0.13 that would cause Privoxy to wait for additional client requests after receiving a HTTP/1.1 request with "Connection: close" set and connection sharing enabled. With clients which terminates the client connection after detecting that the whole body has been received it doesn't really matter, but with clients that don't the connection would be kept open until it timed out. - Fix a subtle race condition between prepare_csp_for_next_request() and sweep(). A thread preparing itself for the next client request could briefly appear to be inactive. If all other threads were already using more recent files, the thread could get its files swept away under its feet. So far this has only been reproduced while stress testing in valgrind while touching action files in a loop. It's unlikely to have caused any actual problems in the real world. - Disable filters if SDCH compression is used unless filtering is forced. If SDCH was combined with a supported compression algorithm, Privoxy previously could try to decompress it and ditch the Content-Encoding header even though the SDCH compression wasn't dealt with. Reported by zebul666 in #3225863. - Make a copy of the --user value and only mess with that when splitting user and group. On some operating systems modifying the value directly is reflected in the output of ps and friends and can be misleading. Reported by zepard in #3292710. - If forwarded-connect-retries is set, only retry if Privoxy is actually forwarding the request. Previously direct connections would be retried as well. - Fixed a small memory leak when retrying connections with IPv6 support enabled. - Remove an incorrect assertion in compile_dynamic_pcrs_job_list() It could be triggered by a pcrs job with an invalid pcre pattern (for example one that contains a lone quantifier). - If the --user argument user[.group] contains a dot, always bail out if no group has been specified. Previously the intended, but undocumented (and apparently untested), behaviour was to try interpreting the whole argument as user name, but the detection was flawed and checked for '0' instead of '\0', thus merely preventing group names beginning with a zero. - In html_code_map[], use a numeric character reference instead of ' which wasn't standardized before XHTML 1.0. - Fix an invalid free when compiled with FEATURE_GRACEFUL_TERMINATION and shut down through http://config.privoxy.org/die - In get_actions(), fix the "temporary" backwards compatibility hack to accept block actions without reason. It also covered other actions that should be rejected as invalid. Reported by Billy Crook. - General improvements: - Privoxy can (re)compress buffered content before delivering it to the client. Disabled by default as most users wouldn't benefit from it. - The +fast-redirects{check-decoded-url} action checks URL segments separately. If there are other parameters behind the redirect URL, this makes it unnecessary to cut them off by additionally using a +redirect{} pcrs command. Initial patch submitted by Jamie Zawinski in #3429848. - When loading action sections, verify that the referenced filters exist. Currently missing filters only result in an error message, but eventually the severity will be upgraded to fatal. - Allow to bind to multiple separate addresses. Patch set submitted by Petr Pisar in #3354485. - Set socket_error to errno if connecting fails in rfc2553_connect_to(). Previously rejected direct connections could be incorrectly reported as DNS issues if Privoxy was compiled with IPv6 support. - Adjust url_code_map[] so spaces are replaced with %20 instead of '+' While '+' can be used by client's submitting form data, this is not actually what Privoxy is using the lookups for. This is more of a cosmetic issue and doesn't fix any known problems. - When compiled without FEATURE_FAST_REDIRECTS, do not silently ignore +fast-redirect{} directives - Added a workaround for GNU libc's strptime() reporting negative year values when the parsed year is only specified with two digits. On affected systems cookies with such a date would not be turned into session cookies by the +session-cookies-only action. Reported by Vaeinoe in #3403560 - Fixed bind failures with certain GNU libc versions if no non-loopback IP address has been configured on the system. This is mainly an issue if the system is using DHCP and Privoxy is started before the network is completely configured. Reported by Raphael Marichez in #3349356. Additional insight from Petr Pisar. - Privoxy log messages now use the ISO 8601 date format %Y-%m-%d. It's only slightly longer than the old format, but contains the full date including the year and allows sorting by date (when grepping in multiple log files) without hassle. - In get_last_url(), do not bother trying to decode URLs that do not contain at least one '%' sign. It reduces the log noise and a number of unnecessary memory allocations. - In case of SOCKS5 failures, dump the socks response in the log message. - Simplify the signal setup in main(). - Streamline socks5_connect() slightly. - In socks5_connect(), require a complete socks response from the server. Previously Privoxy didn't care how much data the server response contained as long as the first two bytes contained the expected values. While at it, shrink the buffer size so Privoxy can't read more than a whole socks response. - In chat(), do not bother to generate a client request in case of direct CONNECT requests. It will not be used anyway. - Reduce server_last_modified()'s stack size. - Shorten get_http_time() by using strftime(). - Constify the known_http_methods pointers in unknown_method(). - Constify the time_formats pointers in parse_header_time(). - Constify the formerly_valid_actions pointers in action_used_to_be_valid(). - Introduce a GNUMakefile MAN_PAGE variable that defaults to privoxy.1. The Debian package uses section 8 for the man page and this should simplify the patch. - Deduplicate the INADDR_NONE definition for Solaris by moving it to jbsockets.h - In block_url(), ditch the obsolete workaround for ancient Netscape versions that supposedly couldn't properly deal with status code 403. - Remove a useless NULL pointer check in load_trustfile(). - Remove two useless NULL pointer checks in load_one_re_filterfile(). - Change url_code_map[] from an array of pointers to an array of arrays It removes an unnecessary layer of indirection and on 64bit system reduces the size of the binary a bit. - Fix various typos. Fixes taken from Debian's 29_typos.dpatch by Roland Rosenfeld. - Add a dok-tidy GNUMakefile target to clean up the messy HTML generated by the other dok targets. - GNUisms in the GNUMakefile have been removed. - Change the HTTP version in static responses to 1.1 - Synced config.sub and config.guess with upstream 2011-11-11/386c7218162c145f5f9e1ff7f558a3fbb66c37c5. - Add a dedicated function to parse the values of toggles. Reduces duplicated code in load_config() and provides better error handling. Invalid or missing toggle values are now a fatal error instead of being silently ignored. - Terminate HTML lines in static error messages with \n instead of \r\n. - Simplify cgi_error_unknown() a bit. - In LogPutString(), don't bother looking at pszText when not actually logging anything. - Change ssplit()'s fourth parameter from int to size_t. Fixes a clang complaint. - Add a warning that the statistics currently can't be trusted. Mention Privoxy-Log-Parser's --statistics option as an alternative for the time being. - In rfc2553_connect_to(), start setting cgi->error_message on error. - Change the expected status code returned for http://p.p/die depending on whether or not FEATURE_GRACEFUL_TERMINATION is available. - In cgi_die(), mark the client connection for closing. If the client will fetch the style sheet through another connection it gets the main thread out of the accept() state and should thus trigger the actual shutdown. - Add a proper CGI message for cgi_die(). - Don't enforce a logical line length limit in read_config_line(). - Slightly refactor server_last_modified() to remove useless gmtime*() calls. - In get_content_type(), also recognize '.jpeg' as JPEG extension. - Add '.png' to the list of recognized file extensions in get_content_type(). - In block_url(), consistently use the block reason "Request blocked by Privoxy" In two places the reason was "Request for blocked URL" which hides the fact that the request got blocked by Privoxy and isn't necessarily correct as the block may be due to tags. - In listen_loop(), reload the configuration files after accepting a new connection instead of before. Previously the first connection that arrived after a configuration change would still be handled with the old configuration. - In chat()'s receive-data loop, skip a client socket check if the socket will be written to right away anyway. This can increase the transfer speed for unfiltered content on fast network connections. - The socket timeout is used for SOCKS negotiations as well which previously couldn't timeout. - Don't keep the client connection alive if any configuration file changed since the time the connection came in. This is closer to Privoxy's behaviour before keep-alive support for client connection has been added and also less confusing in general. - Treat all Content-Type header values containing the pattern 'script' as a sign of text. Reported by pribog in #3134970. - Action file improvements: - Moved the site-specific block pattern section below the one for the generic patterns so for requests that are matched in both, the block reason for the domain is shown which is usually more useful than showing the one for the generic pattern. - Remove -prevent-compression from the fragile alias. It's no longer used anywhere by default and isn't known to break stuff anyway. - Add a (disabled) section to block various Facebook tracking URLs. Reported by Dan Stahlke in #3421764. - Add a (disabled) section to rewrite and redirect click-tracking URLs used on news.google.com. Reported by Dan Stahlke in #3421755. - Unblock linuxcounter.net/. Reported by Dan Stahlke in #3422612. - Block 'www91.intel.com/' which is used by Omniture. Reported by Adam Piggott in #3167370. - Disable the handle-as-empty-doc-returns-ok option and mark it as deprecated. Reminded by tceverling in #2790091. - Add ".ivwbox.de/" to the "Cross-site user tracking" section. Reported by Nettozahler in #3172525. - Unblock and fast-redirect ".awin1.com/.*=http://". Reported by Adam Piggott in #3170921. - Block "b.collective-media.net/". - Widen the Debian popcon exception to "qa.debian.org/popcon". Seen in Debian's 05_default_action.dpatch by Roland Rosenfeld. - Block ".gemius.pl/" which only seems to be used for user tracking. Reported by johnd16 in #3002731. Additional input from Lee and movax. - Disable banners-by-size filters for '.thinkgeek.com/'. The filter only seems to catch pictures of the inventory. - Block requests for 'go.idmnet.bbelements.com/please/showit/'. Reported by kacperdominik in #3372959. - Unblock adainitiative.org/. - Add a fast-redirects exception for '.googleusercontent.com/.*=cache'. - Add a fast-redirects exception for webcache.googleusercontent.com/. - Unblock http://adassier.wordpress.com/ and http://adassier.files.wordpress.com/. - Filter file improvements: - Let the yahoo filter hide '.ads'. - Let the msn filter hide overlay ads for Facebook 'likes' in search results and elements with the id 's_notf_div'. They only seem to be used to advertise site 'enhancements'. - Let the js-events filter additionally disarm setInterval(). Suggested by dg1727 in #3423775. - Documentation improvements: - Clarify the effect of compiling Privoxy with zlib support. Suggested by dg1727 in #3423782. - Point out that the SourceForge messaging system works like a black hole and should thus not be used to contact individual developers. - Mention some of the problems one can experience when not explicitly configuring an IP addresses as listen address. - Explicitly mention that hostnames can be used instead of IP addresses for the listen-address, that only the first address returned will be used and what happens if the address is invalid. Requested by Calestyo in #3302213. - Log message improvements: - If only the server connection is kept alive, do not pretend to wait for a new client request. - Remove a superfluous log message in forget_connection(). - In chat(), properly report missing server responses as such instead of calling them empty. - In forwarded_connect(), fix a log message nobody should ever see. - Fix a log message in socks5_connect(), a failed write operation was logged as failed read operation. - Let load_one_actions_file() properly complain about a missing '{' at the beginning of the file. Simply stating that a line is invalid isn't particularly helpful. - Do not claim to listen on a socket until Privoxy actually does. Patch submitted by Petr Pisar #3354485 - Prevent a duplicated LOG_LEVEL_CLF message when sending out the "no-server-data" response. - Also log the client socket when dropping a connection. - Include the destination host in the 'Request ... marked for blocking. limit-connect{...} doesn't allow CONNECT ...' message Patch submitted by Saperski in #3296250. - Prevent a duplicated log message if none of the resolved IP addresses were reachable. - In connect_to(), do not pretend to retry if forwarded-connect-retries is zero or unset. - When a specified user or group can't be found, put the name in single-quotes when logging it. - In rfc2553_connect_to(), explain getnameinfo() errors better. - Remove a useless log message in chat(). - When retrying to connect, also log the maximum number of connection attempts. - Rephrase a log message in compile_dynamic_pcrs_job_list(). Divide the error code and its meaning with a colon. Call the pcrs job dynamic and not the filter. Filters may contain dynamic and non-dynamic pcrs jobs at the same time. Only mention the name of the filter or tagger, but don't claim it's a filter when it could be a tagger. - In a fatal error message in load_one_actions_file(), cover both URL and TAG patterns. - In pcrs_strerror(), properly report unknown positive error code values as such. Previously they were handled like 0 (no error). - In compile_dynamic_pcrs_job_list(), also log the actual error code as pcrs_strerror() doesn't handle all errors reported by pcre. - Don't bother trying to continue chatting if the client didn't ask for it. Reduces log noise a bit. - Make two fatal error message in load_one_actions_file() more descriptive. - In cgi_send_user_manual(), log when rejecting a file name due to '/' or '..'. - In load_file(), log a message if opening a file failed. The CGI error message alone isn't too helpful. - In connection_destination_matches(), improve two log messages to help understand why the destinations don't match. - Rephrase a log message in serve(). Client request arrival should be differentiated from closed client connections now. - In serve(), log if a client connection isn't reused due to a configuration file change. - Let mark_server_socket_tainted() always mark the server socket tainted, just don't talk about it in cases where it has no effect. It doesn't change Privoxy's behaviour, but makes understanding the log file easier. - configure: - Added a --disable-ipv6-support switch for platforms where support is detected but doesn't actually work. - Do not check for the existence of strerror() and memmove() twice - Remove a useless test for setpgrp(2). Privoxy doesn't need it and it can cause problems when cross-compiling. - Rename the --disable-acl-files switch to --disable-acl-support. Since about 2001, ACL directives are specified in the standard config file. - Update the URL of the 'Removing outdated PCRE version after the next stable release' posting. The old URL stopped working after one of SF's recent site "optimizations". Reported by Han Liu. - Privoxy-Regression-Test: - Added --shuffle-tests option to increase the chances of detection race conditions. - Added a --local-test-file option that allows to use Privoxy-Regression-Test without Privoxy. - Added tests for missing socks4 and socks4a forwarders. - The --privoxy-address option now works with IPv6 addresses containing brackets, too. - Perform limited sanity checks for parameters that are supposed to have numerical values. - Added a --sleep-time option to specify a number of seconds to sleep between tests, defaults to 0. - Disable the range-requests tagger for tests that break if it's enabled. - Log messages use the ISO 8601 date format %Y-%m-%d. - Fix spelling in two error messages. - In the --help output, include a list of supported tests and their default levels. - Adjust the tests to properly deal with FEATURE_TOGGLE being disabled. - Privoxy-Log-Parser: - Perform limited sanity checks for command line parameters that are supposed to have numerical values. - Implement a --unbreak-lines-only option to try to revert MUA breakage. - Accept and highlight: Added header: Content-Encoding: deflate - Accept and highlight: Compressed content from 29258 to 8630 bytes. - Accept and highlight: Client request arrived in time on socket 21. - Highlight: Didn't receive data in time: a.fsdn.com:443 - Accept log messages with ISO 8601 time stamps, too. - uagen: - Bump generated Firefox version to 8.0. - Only randomize the release date if the new --randomize-release-date option is enabled. Firefox versions after 4 use a fixed date string without meaning. *** Version 3.0.17 Stable *** - Fixed last-chunk-detection for responses where the body was small enough to be read with the headers, causing Privoxy to wait for the end of the content until the server closed the connection or the request timed out. Reported by "Karsten" in #3028326. - Responses with status code 204 weren't properly detected as body-less like RFC2616 mandates. Like the previous bug, this caused Privoxy to wait for the end of the content until the server closed the connection or the request timed out. Fixes #3022042 and #3025553, reported by a user with no visible name. Most likely also fixes a bunch of other AJAX-related problem reports that got closed in the past due to insufficient information and lack of feedback. - Fixed an ACL bug that made it impossible to build a blacklist. Usually the ACL directives are used in a whitelist, which worked as expected, but blacklisting is still useful for public proxies where one only needs to deny known abusers access. - Added LOG_LEVEL_RECEIVED to log the not-yet-parsed data read from the network. This should make debugging various parsing issues a lot easier. - The IPv6 code is enabled by default on Windows versions that support it. Patch submitted by oCameLo in #2942729. - In mingw32 versions, the user.filter file is reachable through the GUI, just like default.filter is. Feature request 3040263. - Added the configure option --enable-large-file-support to set a few defines that are required by platforms like GNU/Linux to support files larger then 2GB. Mainly interesting for users without proper logfile management. - Logging with "debug 16" no longer stops at the first nul byte which is pretty useless. Non-printable characters are replaced with their hex value so the result can't span multiple lines making parsing them harder then necessary. - Privoxy logs when reading an action, filter or trust file. - Fixed incorrect regression test markup which caused a test in 3.0.16 to fail while Privoxy itself was working correctly. While Privoxy accepts hide-referer, too, the action name is actually hide-referrer which is also the name used one the final results page, where the test expected the alias. - CGI interface improvements: - In finish_http_response(), continue to add the 'Connection: close' header if the client connection will not be kept alive. Anonymously pointed out in #2987454. - Apostrophes in block messages no longer cause parse errors when the blocked page is viewed with JavaScript enabled. Reported by dg1727 in #3062296. - Fix a bunch of anchors that used underscores instead of dashes. - Allow to keep the client connection alive after crunching the previous request. Already opened server connections can be kept alive, too. - In cgi_show_url_info(), don't forget to prefix URLs that only contain http:// or https:// in the path. Fixes #2975765 reported by Adam Piggott. - Show the 404 CGI page if cgi_send_user_manual() is called while local user manual delivery is disabled. - Action file improvements: - Enable user.filter by default. Suggested by David White in #3001830. - Block .sitestat.com/. Reported by johnd16 in #3002725. - Block .atemda.com/. Reported by johnd16 in #3002723. - Block js.adlink.net/. Reported by johnd16 in #3002720. - Block .analytics.yahoo.com/. Reported by johnd16 in #3002713. - Block sb.scorecardresearch.com, too. Reported by dg1727 in #2992652. - Fix problems noticed on Yahoo mail and news pages. - Remove the too broad yahoo section, only keeping the fast-redirects exception as discussed on ijbswa-devel@. - Don't block adesklets.sourceforge.net. Reported in #2974204. - Block chartbeat ping tracking. Reported in #2975895. - Tag CSS and image requests with cautious and medium settings, too. - Don't handle view.atdmt.com as image. It's used for click-throughs so users should be able to "go there anyway". Reported by Adam Piggott in #2975927. - Also let the refresh-tags filter remove invalid refresh tags where the 'url=' part is missing. Anonymously reported in #2986382. While at it, update the description to mention the fact that only refresh tags with refresh times above 9 seconds are covered. - javascript needs to be blocked with +handle-as-empty-document to work around Firefox bug 492459. So move .js blockers from +block{Might be a web-bug.} -handle-as-empty-document to +block{Might be a web-bug.} +handle-as-empty-document. - ijbswa-Feature Requests-3006719 - Block 160x578 Banners. - Block another omniture tracking domain. - Added a range-requests tagger. - Added two sections to get Flickr's Ajax interface working with default pre-settings. If you change the configuration to block cookies by default, you'll need additional exceptions. Reported by Mathias Homann in #3101419 and by Patrick on ijbswa-users@. - Documentation improvements: - Explicitly mention how to match all URLs. - Consistently recommend socks5 in the Tor FAQ entry and mention its advantage compared to socks4a. Reported by David in #2960129. - Slightly improve the explanation of why filtering may appear slower than it is. - Grammar fixes for the ACL section. - Fixed a link to the 'intercepting' entry and add another one. - Rename the 'Other' section to 'Mailing Lists' and reword it to make it clear that nobody is forced to use the trackers - Note that 'anonymously' posting on the trackers may not always be possible. - Suggest to enable debug 32768 when suspecting parsing problems. - Privoxy-Log-Parser improvements: - Gather statistics for ressources, methods, and HTTP versions used by the client. - Also gather statistics for blocked and redirected requests. - Provide the percentage of keep-alive offers the client accepted. - Add a --url-statistics-threshold option. - Add a --host-statistics-threshold option to also gather statistics about how many request where made per host. - Fix a bug in handle_loglevel_header() where a 'scan: ' got lost. - Add a --shorten-thread-ids option to replace the thread id with a decimal number. - Accept and ignore: Looks like we got the last chunk together with the server headers. We better stop reading. - Accept and ignore: Continue hack in da house. - Accept and higlight: Rejecting connection from 10.0.0.2. Maximum number of connections reached. - Accept and highlight: Loading actions file: /usr/local/etc/privoxy/default.action - Accept and highlight: Loading filter file: /usr/local/etc/privoxy/default.filter - Accept and highlight: Killed all-caps Host header line: HOST: bestproxydb.com - Accept and highlight: Reducing expected bytes to 0. Marking the server socket tainted after throwing 4 bytes away. - Accept: Merged multiple header lines to: 'X-FORWARDED-PROTO: http X-HOST: 127.0.0.1' - Code cleanups: - Remove the next member from the client_state struct. Only the main thread needs access to all client states so give it its own struct. - Garbage-collect request_contains_null_bytes(). - Ditch redundant code in unload_configfile(). - Ditch LogGetURLUnderCursor() which doesn't seem to be used anywhere. - In write_socket(), remove the write-only variable write_len in an ifdef __OS2__ block. Spotted by cppcheck. - In connect_to(), don't declare the variable 'flags' on OS/2 where it isn't used. Spotted by cppcheck. - Limit the scope of various variables. Spotted by cppcheck. - In add_to_iob(), turn an interestingly looking for loop into a boring while loop. - Code cleanup in preparation for external filters. - In listen_loop(), mention the socket on which we accepted the connection, not just the source IP address. - In write_socket(), also log the socket we're writing to. - In log_error(), assert that escaped characters get logged completely or not at all. - In log_error(), assert that ival and sval have reasonable values. There's no reason not to abort() if they don't. - Remove an incorrect cgi_error_unknown() call in a cannot-happen-situation in send_crunch_response(). - Clean up white-space in http_response definition and move the crunch_reason to the beginning. - Turn http_response.reason into an enum and rename it to http_response.crunch_reason. - Silence a 'gcc (Debian 4.3.2-1.1) 4.3.2' warning on i686 GNU/Linux. - Fix white-space in a log message in remove_chunked_transfer_coding(). While at it, add a note that the message doesn't seem to be entirely correct and should be improved later on. - GNUmakefile improvements: - Use $(SSH) instead of ssh, so one only needs to specify a username once. - Removed references to the action feedback thingy that hasn't been working for years. - Consistently use shell.sourceforge.net instead of shell.sf.net so one doesn't need to check server fingerprints twice. - Removed GNUisms in the webserver and webactions targets so they work with standard tar. *** Version 3.0.16 Stable *** - Added the config file option handle-as-empty-doc-returns-ok to work around Firefox bug #492459, which causes Firefox to hang if JavaScripts are blocked in certain situations. The option is enabled in the default config file. - Added the config file option default-server-timeout to control the assumed default server timeout. Since Privoxy no longer returns an error message for connection resets on reused client connections, assuming larger server timeout values appears to actually work pretty well as long as connections aren't shared. - Added optional support for FreeBSD's accf_http(9). Use the configure option --enable-accept-filter to enable it. - Added fancier Privoxy icons for win32. Contributed by Jeff H. - In daemon mode, fd 0, 1 and 2 are bound to /dev/null. - Resolve localhost using whatever address family the operating system feels like. Previous betas would try to use IPv4 as this is what most users expect, but this didn't work reliably on GNU/Linux systems. - In the action lists on CGI pages, actions and their parameters are no longer separated with a space. The action file parser doesn't actually allow this and will throw an invalid syntax error if actions and parameters in the action files are separated. Not adding the spaces means copy and pasting CGI output into the action files works. - The default keep-alive timeout has been reduced to 5 seconds to work around hangs in clients that treat the proxy like any other host and stop allowing any new connections if the "maximum number of connections per host" is reached. - Several webbug URLs that look like they are leading to images are now blocked as image instead of empty documents. Doing the latter causes WebKit-based clients to show a "missing image" icon which may mess up the layout. - The no-such-domain template is used for DNS resolution problems with FEATURE_IPV6_SUPPORT enabled. Previously the connect-failed template was used. Reported by 'zebul666'. - Accepts quoted expiration dates even though RFC 2109 10.1.2 doesn't seem to allow them. Reported anonymously. - Don't try to forget connections if connection sharing is disabled. This wasn't a real problem but caused an unnecessary log message. - The still undocumented --enable-extended-host-patterns configure option has a better description. - Fixed an error message that would claim a write to the server failed when actually writing to the client failed. - Log the crunch reason before trying to write to the client. The log is easier to read that way. - Several log messages about client connections also mention the socket number. - handle-as-empty-document no longer depends on the image blocking code being enabled. - Privoxy-Log-Parser is roughly 40% faster in highlighting mode. - uagen, a Firefox User-Agent generator for Privoxy and Mozilla browsers has been imported and is available in the tarball's tools directory. - The scripts in the tools directory treat unknown parameters as fatal errors. *** Version 3.0.15 beta *** - In case of missing server data, no error message is send to the client if the request arrived on a reused connection. The client is then supposed to silently retry the request without bothering the user. This should significantly reduce the frequency of the "No server or forwarder data received" error message many users reported. - More reliable detection of prematurely closed client sockets with keep-alive enabled. - FEATURE_CONNECTION_KEEP_ALIVE is decoupled from FEATURE_CONNECTION_SHARING and now available on all platforms. - Improved handling of POST requests on reused connections. Should fix problems with stalled connections after submitting form data with some browser configurations. - Fixed various latency calculation issues. - Allows the client to pass NTLM authentication requests to a forwarding proxy. This was already assumed and hinted to work in 3.0.13 beta but actually didn't. Now it's confirmed to work with IE, Firefox and Chrome. Thanks to Francois Botha and Wan-Teh Chang - Fixed a calculation problem if receiving the server headers takes more than two reads, that could cause Privoxy to terminate the connection prematurely. Reported by Oliver. - Compiles again on platforms such as OpenBSD and systems using earlier glibc version that don't support AI_ADDRCONFIG. Anonymously submitted in #2872591. - A bunch of MS VC project files and Suse and Redhat RPM spec files have been removed as they were no longer maintained for quite some time. - Overly long action lines are properly rejected with a proper error message. Previously they would be either rejected as invalid or cause a core dump through abort(). - Already timed-out connections are no longer temporarily remembered. They weren't reused anyway, but wasted a socket slot. - len refers to the number of bytes actually read which might differ from the ones received. Adjust log messages accordingly. - The optional JavaScript on the CGI page uses encodeURIComponent() instead of escape() which doesn't encode all characters that matter. Anonymously reported in #2832722. - Fix gcc45 warnings in decompress_iob(). - Various log message improvements. - Privoxy-Regression-Test supports redirect tests. - Privoxy-Log-Parser can gather some connection statistics. *** Version 3.0.14 beta *** - The latency is taken into account when evaluating whether or not to reuse a connection. This should significantly reduce the number of connections problems several users reported. - If the server doesn't specify how long the connection stays alive, Privoxy errs on the safe side of caution and assumes it's only a second. - The error pages for connection timeouts or missing server data use a Last-Modified date in the past. Retry attempts are detected and Privoxy removes the If-Modified-Since header to prevent the server from responding with status code 304 in which case the client would reuse the error message. - Setting keep-alive-timeout to 0 disables keep-alive support. Previously Privoxy would claim to allow persistence but not reuse the connection. - Pipelined requests are less likely to be mistaken for the request body of the previous request. Note that Privoxy still has no real pipeline support and will either serialize pipelined requests or drop them in which case the client has to resent them. - Fixed a crash on some Windows versions when header randomization is enabled and the date couldn't be parsed. - Privoxy's keep-alive timeout for the current connection is reduced to the one specified in the client's Keep-Alive header. - For HTTP/1.1 requests, Privoxy implies keep-alive support by not setting any Connection header instead of using 'Connection: keep-alive'. - If the socket isn't reusable, Privoxy doesn't temporarily waste a socket slot to remember the connection. - If keep-alive support is disabled but compiled in, the client's Keep-Alive header is removed. - Fixed a bug on mingw32 where downloading large files failed if keep-alive support was enabled. - Fixed a bug that (at least theoretically) could cause log timestamps to be occasionally off by about a second. - No Proxy-Connection header if added if there already is one. - The configure script respects the $PATH variable when searching for groups and id. *** Version 3.0.13 beta *** - Added IPv6 support. Thanks to Petr Pisar who not only provided the initial patch but also helped a lot with the integration. - Added client-side keep-alive support. - The connection sharing code is only used if the connection-sharing option is enabled. - The max-client-connections option has been added to restrict the number of client connections below a value enforced by the operating system. - Fixed a regression reintroduced in 3.0.12 that could cause crashes on mingw32 if header date randomization was enabled. - Compressed content with extra fields couldn't be decompressed and would get passed to the client unfiltered. This problem has only be detected through statical analysis with clang as nobody seems to be using extra fields anyway. - If the server resets the Connection after sending only the headers Privoxy forwards what it got to the client. Previously Privoxy would deliver an error message instead. - Error messages in case of connection timeouts use the right HTTP status code. - If spawning a child to handle a request fails, the client gets an error message and Privoxy continues to listen for new requests right away. - The error messages in case of server-connection timeouts or prematurely closed server connections are now template-based. - If zlib support isn't compiled in, Privoxy no longer tries to filter compressed content unless explicitly asked to do so. - In case of connections that are denied based on ACL directives, the memory used for the client IP is no longer leaked. - Fixed another small memory leak if the client request times out while waiting for client headers other than the request line. - The client socket is kept open until the server socket has been marked as unused. This should increase the chances that the still-open connection will be reused for the client's next request to the same destination. Note that this only matters if connection-sharing is enabled. - A TODO list has been added to the source tarball to give potential volunteers a better idea of what the current goals are. Donations are still welcome too: http://www.privoxy.org/faq/general.html#DONATE *** Version 3.0.12 *** - The socket-timeout option now also works on platforms whose select() implementation modifies the timeout structure. Previously the timeout was triggered even if the connection didn't stall. Reported by cyberpatrol. - The Connection: keep-alive code properly deals with files larger than 2GB. Previously the connection was closed too early. - The content length for files above 2GB is logged correctly. - The user-manual directive on the show-status page links to the documentation location specified with the directive, not to the Privoxy website. - When running in daemon mode, Privoxy doesn't log anything to the console unless there are errors before the logfile has been opened. - The show-status page prints warnings about invalid directives on the same line as the directives themselves. - Fixed several justified (but harmless) compiler warnings, mostly on 64 bit platforms. - The mingw32 version explicitly requests the default charset to prevent display problems with some fonts available on more recent Windows versions. Patch by Burberry. - The mingw32 version uses the Privoxy icon in the alt-tab windows. Patch by Burberry. - The timestamp and the thread id is omitted in the "Fatal error" message box on mingw32. - Fixed two related mingw32-only buffer overflows. Triggering them required control over the configuration file, therefore this isn't seen as a security issue. - In verbose mode, or if the new option --show-skipped-tests is used, Privoxy-Regression-Test logs skipped tests and the skip reason. *** Version 3.0.11 *** - On most platforms, outgoing connections can be kept alive and reused if the server supports it. Whether or not this improves things depends on the connection. - When dropping privileges, membership in supplementary groups is given up as well. Not doing that can lead to Privoxy running with more rights than necessary and violates the principle of least privilege. Users of the --user option are advised to update. Thanks to Matthias Drochner for reporting the problem, providing the initial patch and testing the final version. - Passing invalid users or groups with the --user option didn't lead to program exit. Regression introduced in 3.0.7. - The match all section has been moved from default.action to a new file called match-all.action. As a result the default.action no longer needs to be touched by the user and can be safely overwritten by updates. - The standard.action file has been removed. Its content is now part of the default.action file. - In some situations the logged content length was slightly too low. - Crunched requests are logged with their own log level. If you used "debug 1" in the past, you'll probably want to additionally enable "debug 1024", otherwise only passed requests will be logged. If you only care about crunched requests, simply replace "debug 1" with "debug 1024". - The crunch reason has been moved to the beginning of the crunch message. For HTTP URLs, the protocol is logged as well. - Log messages are shortened by printing the thread id on its own (as opposed to putting it inside the string "Privoxy()"). - The config option socket-timeout has been added to control the time Privoxy waits for data to arrive on a socket. - Support for remote toggling is controlled by the configure option --disable-toggle only. In previous versions it also depended on the action editor and thus configuring with the --disable-editor option would disable remote toggling support as well. - Requests with invalid HTTP versions are rejected. - The template symbol @date@ can be used to include a date(1)-like time string. Initial patch submitted by Endre Szabo. - Responses from shoutcast servers are accepted again. Problem reported and fix suggested by Stefan. - The hide-forwarded-for-headers action has been replaced with the change-x-forwarded-for{} action which can also be used to add X-Forwarded-For headers. The latter functionality already existed in Privoxy versions prior to 3.0.7 but has been removed as it was often used unintentionally (by not using the hide-forwarded-for-headers action). - A "clear log" view option was added to the mingw32 version to clear out all of the lines in the Privoxy log window. Based on a patch submitted by T Ford. - The mingw32 version uses "critical sections" now, which prevents log message corruption under load. As a side effect, the "no thread-safe PRNG" warning could be removed as well. - The mingw32 version's task bar icon is crossed out and the color changed to gray if Privoxy is toggled off. *** Version 3.0.10 *** - Ordinary configuration file changes no longer cause program termination on OS/2 if the name of the logfile hasn't been changed as well. This regression probably crept in with the logging improvements in 3.0.7. Reported by Maynard. - The img-reorder filter is less likely to mess up JavaScript code in img tags. Problem and solution reported by Glenn Washburn in #2014552. - The source tar ball now includes Privoxy-Log-Parser, a syntax-highlighter for Privoxy logs. For fancy screenshots see: http://www.fabiankeil.de/sourcecode/privoxy-log-parser/ Documentation is available through perldoc(1). *** Version 3.0.9 beta *** - Added SOCKS5 support (with address resolution done by the SOCKS5 server). Patch provided by Eric M. Hopper. - The "blocked" CGI pages include a block reason that was provided as argument to the last-applying block action. - If enable-edit-actions is disabled (the default since 3.0.7 beta) the show-status page hides the edit buttons and explains why. Previously the user would get the "this feature has been disabled" message after using the edit button. - Forbidden CONNECT requests are treated like blocks by default. The now-pointless treat-forbidden-connects-like-blocks action has been removed. - Not enabling limit-connect now allows CONNECT requests to all ports. In previous versions it would only allow CONNECT requests to port 443. Use +limit-connect{443} if you think you need the old default behaviour. - The CGI editor gets turned off after three edit requests with invalid file modification timestamps. This makes life harder for attackers who can leverage browser bugs to send fake Referers and intend to brute-force edit URLs. - Action settings for multiple patterns in the same section are shared in memory. As a result these sections take up less space (and are loaded slightly faster). Problem reported by Franz Schwartau. - Linear white space in HTTP headers will be normalized to single spaces before parsing the header's content, headers split across multiple lines get merged first. This should prevent problems like: * letting the session-cookies-only action slip some Cookies through unmodified, * only suppressing the first line of a header, thus creating an invalid one, and * to incorrectly block headers with valid timestamps that weren't properly recognized. Headers that could trigger these problems are unlikely to appear in "normal" web traffic, but could be intentionally generated to fool some of Privoxy's header parsers. - Host information is gathered outside the main thread so it's less likely to delay other incoming connections if the host is misconfigured. - New config option "hostname" to use a hostname other than the one returned by the operating system. Useful to speed-up responses for CGI requests on misconfigured systems. Requested by Max Khon. - The CGI editor supports the "disable all filters of this type" directives "-client-header-filter", "-server-header-filter", "-client-header-tagger" and "-server-header-tagger". - Fixed false-positives with the link-by-url filter and URLs that contain the pattern "/jump/". - The less-download-windows filter no longer messes "Content-Type: application/x-shockwave-flash" headers up. - In the show-url-info page's "Final results" section active and inactive actions are listed separately. Patch provided by Lee. - The GNUmakefile supports the DESTDIR variable. Patch for the install target submitted by Radoslaw Zielinski. - Embedding the content of configuration files in the show-status page is significantly faster now. For a largish action file (1 MB) a speedup of about 2450 times has been measured. This is mostly interesting if you are using large action files or regularly use Privoxy-Regression-Test while running Privoxy through Valgrind, for stock configuration files it doesn't really matter. - If zlib support is unavailable and there are content filters active but the prevent-compression action is disabled, the show-url-info page includes a warning that compression might prevent filtering. - The show-url-info page provides an OpenSearch Description that allows to access the page through browser search plugins. - Custom client-header filters that rewrite the request line incorrectly no longer cause Privoxy to crash. Reported by din_a4. - The obsolete kill-popups action has been removed as the PCRS-based popup filters can do the same and are slightly less unreliable. - The inspect-jpegs action has been removed. - The send-wafer and send-vanilla-wafer actions have been removed. They weren't particular useful and their behaviour could be emulated with add-header anyway. - Privoxy-Regression-Test has been significantly improved. - Most sections in the default.action file contain tests for Privoxy-Regression-Test to verify that they are working as intended. - Parts of Privoxy have been refactored to increase maintainability. - Building with zlib (if available) is done by default. *** Version 3.0.8 *** - Fixed a small memory leak when listen-address only specifies the port. - The source tar balls now include Privoxy-Regression-Test which (upon other things) can be used to automatically detect some packaging problems. Packagers are welcome to give it a try. - Reverted a change in 3.0.7 that caused path patterns to be checked even if the host pattern match already failed. While this doesn't noticeable affect the performance, it makes it less likely to run out of stack space with overly-complex path patterns the user might have added. - Updated the msn, yahoo and google filters to work as advertised again. - The warning message shown by the show-status CGI page is easier to understand. Previously it wasn't clear that the error message is shown below the invalid directive. (Reported by Lee) - When regenerating Content-Disposition headers the more common spelling is used for the name. Previously it was written without caps. - Less confusing log message if the content type isn't overwritten because force-text-type wasn't used but the old type doesn't look like content that would be filtered normally. - Better log messages if the user tries to execute filters that don't exist. - Treat the non-standard Request-Range headers like standard range headers and suppress them if content filtering is enabled. - Prevent the log messages for CONNECT requests to unacceptable ports from printing the limit-connect argument as [null] if limit-connect hasn't been explicitly enabled. - Don't disable the mingw32 log window if the logfile directive isn't used. While it was an intentional change in 3.0.7 at least one user perceived it as a regression and the same effect can be achieved by disabling all debug directives. - Fixed two minor problems related to the win32 build process: a css file was not being in the installer and the trustfile comment in the config.txt referenced a nonexisting file - Minor documentation fixes. *** Version 3.0.7 beta *** - Added zlib support to filter content with gzip and deflate encoding. (Patch provided by Wil Mahan) - Dedicated filters and actions are used for header filtering. "filter-client-headers" and "filter-client-headers" are no longer supported, use server-header-filter{} and client-header-filter{} instead. - Tags can be used to change actions based on HTTP headers. - New server-header filter: less-download-windows. - New client-header taggers: css-requests, image-requests, client-ip-address, http-method, allow-post, complete-url, user-agent and privoxy-control. - New server-header taggers: content-type and privoxy-control. - The forward-override{} action allows to change the forwarding settings through the action files, for example based on client headers like the User-Agent, or the request origin. - Socks errors are no longer handled by the CGI page for DNS resolution failures. - CGI pages use favicons to signal whether they are error or control pages. This is useful if you rely heavily on browser tabs. - The show-url-info CGI page shows the forwarding settings. - "Crunch!" log messages (used when Privoxy answers requests by itself) now also contain the reason. - Allow to rewrite the request destination behind the client's back. - Fix socks requests on big-endian platforms. Patch provided by Song Weijia. - Fixes possible deadlocks and crashes on OpenBSD. Patch provided by Ralf Horstmann. - The CGI action editor allows to edit actionfiles with previously forbidden characters like dots. - New trust entries are saved with a comment that contains the trusted referring URL (Suggested by Daniel Griscom). - Filter descriptions are HTML encoded automatically. - New config option "split-large-forms" to work around a browser bug that caused IE6 and IE7 to ignore the Submit button on the edit-actions-for-url CGI page. - New config option "allow-cgi-request-crunching" to allow requests for Privoxy's CGI pages to be blocked, redirected or (un)trusted like ordinary requests. - Empty filter files no longer interrupt the filtering process prematurely and are correctly listed on the show-status CGI page. - New config option "accept-intercepted-requests" to combine Privoxy with any packet filter to build an intercepting proxy for HTTP/1.1 requests (and for HTTP/1.0 requests with Host header set). - fast-redirects{} catch redirects to https URLs as well. - redirect{s@foo@bar@} can be used to redirect to a rewritten version of the original URL. - Trap unsupported gopher proxy requests. - Fixed a bug in the User Manual delivery on Windows (mingw32 only). Images now show up correctly and HTML pages are no longer padded with garbage data. - Fixed several minor memory leaks, most of them discovered with Valgrind. - Only unlink the pidfile if it's actually used. - Retries after connection problems with forced requests aren't blocked again. - On Unix SIGABRT causes a core dump as expected and is no longer treated as normal shutdown signal. - The "access denied" CGI page is more descriptive and allows retries to circumvent the referrer check. - Updated PCRS to handle unexpected PCRE errors properly. Fixed crashes that could occur if Privoxy was build with external PCRE versions newer than Privoxy's internal one. (Reported by Chung-chieh Shan) - Fixed crashes with null bytes in PCRS replacement strings (Patch provided by Felix Gröbert). - Fixed crashes with header time randomization on mingw32. - The CGI style sheet is no longer delivered if the referring page isn't a Privoxy CGI page. This prevents a JavaScript-based Privoxy detection "attack". Note that detecting Privoxy is still possible through other ways and Privoxy was never intended to be invisible anyway. - Added support for AmigaOS 4, fixed build for AmigaOS 3.x. - The show-url-info CGI page displays a warning if Privoxy is currently toggled off. - The show-status CGI page suppresses the edit button for action files if Privoxy has no write access. - Most CGI error pages react properly to HEAD requests. - Requests with RFC 3253 HTTP methods (used by Subversion) are accepted. (Patch provided by Petr Kadlec) - New config option "templdir" to change the location of the CGI templates to make sure customized templates aren't "updated". - Better handling of "HTTP/1.1 100 Continue" responses. - The background of the PNG pattern is transparent. - Fixed XML syntax errors caused by banners-by-size and banners-by-url. - Fixed crashes and possible action file corruptions when lines containing hashes are written through the CGI editor. - Supports dynamic filters which can contain variables. - Supports tags to change the actions based on client or server headers. - Incorrect actions are logged before program termination. - The "actionsfile" syntax in the configuration file is consistent with the rest of the configuration options and requires the whole file name. This is an incompatible change, if you use an old configuration file you might have to append ".action" to your "actionsfile" directives. - With the configuration file option "enforce-blocks" the "go there anyway" mechanism can be disabled without recompiling Privoxy. - More precise error messages in case of incorrect acl syntax. - Logs a warning if filtering is enabled but impossible due to lack of zlib support or use of the prevent-compression action. - Less noisy handling of Cookie:" and "Connection:" headers. - Improved error messages in case of connection problems. - Fix a command-line-parsing bug that was introduced before 3.0.5 beta and caused Privoxy to treat the last argument as configuration file if no configuration file was specified. - Treat unknown command line options as fatal errors instead of silently ignoring them. - Use string functions with length checks more often. - Don't log CONNECT requests twice. - Allow to log the source address for ACL-related connection drops. - Don't ignore applying filters if the server didn't specify a Content-Type. Bug reported by Amuro Namie. - Rejected CONNECT requests are logged with log level info (enabled by default) and the reason for the block. - New command line option "--pre-chroot-nslookup hostname" to intialize the resolver library before chroot'ing. On some systems this reduces the number of files that must be copied into the chroot tree. (Patch provided by Stephen Gildea) - Fix a long-standing memory corruption bug that could cause Privoxy to overwrite a single byte in memory it didn't explicitly allocate (but that probably was allocated anyway due to bucket size). - Send template-based CGI pages as HTTP/1.1 unless the client asked for HTTP/1.0. - Let the first line in connection established responses end in \r\n as required by RFC1945. Reported by Bert van Leeuwen. - If no log file has been specified, disable logging instead of logging to stderr. - Don't block stderr when in daemon mode. - Ignore missing zero-chunks when filtering chunk-encoded content. Earlier Privoxy versions would buffer and then forward the content unmodified which caused some browsers to simply show empty pages. - Fix double free in cgi_edit_actions_list(). Reported by Venustech AD-LAB. - The code to add X-Forwarded-For headers when the hide-forwarded-for-headers action isn't being used has been removed. - Fixed trustfile feature which previously didn't work without FEATURE_TOGGLE. Reported by Lee. - Minor code clean-ups, filter and action file updates. (Some of them reported by Davide Alberani, Markus Elfring, Stefan Huehner and Adam Piggott) *** Version 3.0.6 *** - New content filters: no-ping, google, msn, yahoo and blogspot. - New header filters: x-httpd-php-to-html, html-to-xml, xml-to-html and hide-tor-exit-notation. - The special header "X-Filter: No" now disables header filtering as well. - Improved the filters img-reorder, js-annoyances, webbugs, banners-by-size, banners-by-link and ie-exploits to make them less likely to break anything. - Removed outdated URL patterns in default.action and added new ones. - Added redirection from http://p.p/user-manual to http://p.p/user-manual/ - Changed webinterface default values for hide-user-agent, hide-referrer and set-image-blocker. *** Version 3.0.5 beta *** - Windows version can be installed/started as a service. - Windows icon stays blue when Privoxy is idle, green when busy. - Integrated Fabian Keil's extensive patch. See: http://www.fabiankeil.de/sourcecode/privoxy/. Includes the following new or significantly improved actions (among many other improvements): content-type-overwrite{} crunch-client-header{string} crunch-if-none-match crunch-server-header{string} fast-redirects{check-decoded-url} filter-client-headers filter-server-headers force-text-mode handle-as-empty-document hide-accept-language{} hide-content-disposition{} hide-if-modified-since hide-referrer{conditional-block} overwrite-last-modified{} redirect{URL} treat-forbidden-connects-like-blocks - Standard-compliant clients are prevented from displaying cached copies of Privoxy's error messages after the cause of the problem has gone. - Improved DNS error handling. - Multiple filter files can now be specified in config. - Added jpeg filtering to defend against MS jpeg vulnerability MS04-028 with the new inspect-jpegs action. - Removed the "arbitrary" 1000 filter limit - addresses tracker #911950 - Thanks to Jindrich Makovicka for a race condition fix for the log file. The race condition remains for non-pthread implementations. Reference patch #1175720. Various other logging enhancements. - A pile of assorted bug fixes, memory leaks, enhancements, etc. - Moved Actions file reporting mechanism to SF tracker. - Two new options for config: enable-remote-http-toggle and forwarded-connect-retries. - Trap unsupported FTP requests. - Let text/xml be filtered. - Numerous updates to default.action - Increase the compiled in limit of trusted referrers from 64 to 512 (for trustfile users). *** Version 3.0.3 *** - Fixed yet another two memory leaks. Process growth seems stopped now. - Further tightened security against malicious toggle-off links. - Excluded text/plain MIME types from filtering. This fixes a couple of client-crashing, download corruption and Privoxy performance issues, whose root cause lies in web servers labelling content of unknown type as text/plain. - Assorted fixes for POSIX compliance, signal handling, graceful termination, compiler warnings, OSX support, Win32 systray, error logging, hostname wildcards, correct detection of NetBSD. - Workarounds for client (iTunes etc) and server (PHP < 4.2.3) bugs including the notorious "blank page" problem. - Various filter improvements; most notably the unsolicited-popups filter became less destructive - Major revamp of the actions file *** Version 3.0.2 *** - Fixed two memory leaks, one serious - Fixed bug in pcrs which could cause crashes with user-defined filters - Fixed bug in domain name matching - Assorted small fixes (Win32 menu, CGI URL editor, ..) - Added basic support for the OPTIONS and TRACE http methods - Added workaround for Bug in Mac OSX that made Privoxy crash occasionally - Refined the default action file through >400 items of user feedback - Filter changes: - Assorted refinements, optimizations and fixes in the js-annoyances, img-reorder, banners-by-size, banners-by-link, webbugs, refresh-tags, html-annoyances, content-cookies and fun filters - Replaced filter "popups" by choice between two modes: - "unsolicited-popups" tries to catch only the unsolicited ones - "all-popups" tries to kill them all (as before) - New filter "tiny-textforms" Help those tiny or hard-wrap textareas. - New filter "jumping-windows" that prevents windows from resizing and moving themselves - New filter "demoronizer" which fixes MS's abuse of std charsets (common cases anyway). - Replaced "nimda" with more general "ie-exploits" filter in which all filters for exploits shall be collected - Improved cookie logging - Rewrote make install target. Added uninstall and install-strip targets. - Fixed a potential (application-level, NOT OS-level!) security problem involving remote toggling and action file manipulation by mailicious websites. - Added ability to chroot (thanks to Sviatoslav Sviridov) - Added more action aliases for prehistoric action names - Add Slackware support to Makefile. *** Version 3.0 *** - Fixed Windows startmenu items, log window and tray icon menus. - Added warning for bogus install target - Added quicktime-kioskmode filter and improved frameset-borders - Updated default.action based on latest feedback - New PDF doc build process - Add a user contrib module to cvs: http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/ijbswa/contrib/ *** Version 2.9.18 *** - Added workaround for IE bug that broke CGI interface - Bugfix: String actions now reliably editable through CGI interface - Three filters fixed (again!) - Assorted small fixes and doc enhancements *** Version 2.9.16 *** - Major revamp of default.action to get rid of years of cruft. - Same for default.filter - Re-design and major improvements to the CGI editor interface. - Address spurious 'out of memory' error due to incorrect file permissions. - Impose buffer limits while reading client and server headers. - Better memory and CPU optimization. - Add Conectiva Linux package. - user-manual directive added to config for help links from within CGI editor. - Multiple actions files can now be specified in config. - Actions files are changed to: default.action, standard.action, and user.action. user.action is for personal/local configuration. - The usual many small and miscellaneous bug and security fixes. *** Version 2.9.14 beta *** - Fix Solaris compile problem (gateway.h and filters.h) - Makefile fixes for Solaris, FreeBSD (?) - Fix build failure where certain features were disabled. - 'blocked-compact' template is removed. Various CGI improvements, including an adaptive 'blocked' template. - Various tweaks for actions file to get ready for stable 3.0 - Included a 'Bookmarklet' and PHP scripts for reporting actions file problems via web interface at privoxy.org. Accessed via internal CGIs. - Include cgi-style.css for templates. - #include mechansim for common text in templates - Various other minor fixes. *** Version 2.9.13 beta *** - *NEWS*: The project has been renamed to Privoxy! The new name is reflected throughout (file locations, etc). - ijb.action is now default.action. re_filterfile is now default.filter. - http://i.j.b/ is now http://p.p/ - The 'logo' option for replacing ad iamges is removed now. 'Pattern' (checkerboard) is now the default. - RPM spec file make over. *** Version 2.9.12 beta *** - **READ**: The default listening PORT is NOW 8118!!! Changed from 8000 due to conflict with NAS (Network Audio Server, whatever that is.) - More CGI actions editor fixes and improvements. - Win32 command line fix ups. - re_filterfile now has modular sections that can be activated on a per site basis. Some new goodies there too. - +filter now takes arguments to match FILTER sections in re_filterfile for even more flexibility. - Added a new image blocker option: +image-blocker{pattern}, which displays a checkerboard patthern and scales better than the logo. - PNG images will be used in place of GIF for JB built-in images if configured with --enable-no-gif. - Clean up compiler warnings (mostly). - Improved handling of failed DNS lookups & diagnostics for failed bind to listen socket - Made --no-daemon mode log to tty instead of logfile. - Various spec file and init script cleanups and improvements (Redhat and SuSE). - CGI Editor works on OS/2 now. - Fix restart failure where sockets were in TIME_WAIT. - Fixes for actions cgi editor, make sure we have right file. - A --pidfile command line option now, in addition to --help, --version, --no-daemon, --user and configfile. --no-daemon replaces the former -d option and _DEBUG define. --user will drop privileges to the specified user. - Signal handling cleanups (*nix). - CGI actions editor improvements and fixes. - Error handling improvements, especially out of memory. - Default re_filterfile fix that caused spurious IJB logos (instead of 'blank'). - configure.in threading fixes for Solaris. - Various other minor fixes. *** Version 2.9.11 beta Changes *** - Add "session" cookie concept where cookies exist for the life of that browser session only (ie never goes to disk). - Checks for correct header length. - Fix user:pass@host.domain.com auth bug. - Better signal handling on *nix. - Fix CFLAGS hard-coded in configure.in - Fix threading bug re: gethostbyname() that caused random URLs to fail in some cases. *** Version 2.9.11 Alpha Changes *** - A web-based editor for the actions file is included (go to http://i.j.b/). - Web-based toggle IJB on/off support. - Cookie handling has changed - the new +no-cookies-keep feature is now the default. - actionsfile is renamed to ijb.action. - junkbstr.txt is now config.txt on Win32. - Support for running IJB as a UNIX daemon process has improved. - Unix daemon now returns error code on failed start. - Timestamps in logfile and jarfile now. - Fix for the Netscape bug reintroduced in 2.9.9. - make should now abort if gmake (GNU make) not present. - Many other minor bugfixes - Start a ChangeLog :) *** Version 2.9.3 pre-Alpha Changes *** - Amiga support (completely untested by me - I don't have an Amiga) - "tinygif 3" support (redirects blocked images to a specified URL, so the browser doesn't have to load and cache many copies of the same image). - one case where there were both local and global "referrer" variables (yuck!) clarified by renaming the local one to "refer". - Fixed some places where close() was used instead of close_socket(). Thanks to Jörg Strohmayer (joergs at users.sourceforge.net) for these. - Temporary hack to get FORCE_LOAD to work with IE. I just lowercased the FORCE_LOAD_PREFIX. Needs fixing properly. - Most URLs hardcoded into Junkbuster were changed to go through a script e.g. http://ijbswa.sourceforge.net/redirect.php?v=2.9.3&to=faq The only other URLs left are the GNU GPL: http://www.fsf.org/copyleft/gpl.html and the home page: http://ijbswa.sourceforge.net/ ... and various URLs which will be intercepted by Junkbuster anyway. TODO: Still need to do something with the URLs in Junkbuster Corp's copyright/trademark notice on the bottom of the show-proxy-args page. - PCRE or GNU Regex is now a #define option. *** Version 2.9.2 pre-Alpha Changes *** - Andreas applied the latest version of the FORCE patch. *** Version 2.9.1 pre-Alpha Changes *** - in parsers.c, fixed two #ifdef FORCE to #ifdef FORCE_LOAD (BTW: I think FORCE is precise enough, since loading remote data is the whole purpose of a proxy..) - Set the FORCE_PREFIX (back) to 'IJB-FORCE-LOAD-'. While 'noijb.' is more elegant and looks like a hostname in the URL, it doesn't make clear to the inexperienced user that the proxy is bypassed. It also has a higher name collision risk. - Filled in the function header templates for my functions in parsers.c (again). They obviously got lost in our current patch war ;-) - Cut the credit for the §-referrer-option from the config file, that Stefan had placed there. - Improved the re_filterfile *** Version 2.9.0 pre-Alpha Changes *** - Now use PCRE, not GNU REGEX. I have not yet had chance to check the syntax of the block/image/cookie file to ensure that they match what is expected - however they seem to work. - Replaced "configure" script with one generated by "autoconf". Also use a header "config.h" (was ijbconfig.h in my previous release) for the #defines. "config.h" is now generated with "autoheader" from "acconfig.h" and "configure.in". (Note that to install you do not need autoconf or autoheader - just run "./configure".) To see command-line options, run "./configure --help". This is my first ever autoconf script, so it has some rough edges (how PCRE is handled is the roughest). - Error logging code replaced with new module errlog.c, based on the one from JunkbusterMT (but with the threading code removed). - Most of Rodney's 0.21 and 0.21A patches applied. (Marked *). I did not apply all of these, since I had already independently done conditional popup file, conditional image file, and integration of popup code. - ACL, Jar and trust files conditionally compiled. - New source file headers. - Various cosmetic changes. (But I have not consistently ordered the config files - I think that's worthwhile, but it's 1am and I want to get this released!) - RCS tags on .h files. - RCS tags are const char[] rather than const char *. (Saves 4 bytes per tag ;-) - VC++ project files renamed to vc_junkbuster.*. - show-proxy-args now shows status of all conditionals, not just REGEX - Various functions moved around. Most notably all the system-specific sockets code which was spread between jcc.c, bind.c, and connect.c, has been moved to "jbsockets.c". The non-system-specific code from connect.c and socks4.c has been movet to "gateway.c". Also, the config file loader and the global variables it writes to have been moved to "loadcfg.c". (Maybe this should go into loaders.c?) And candidate for the "worst filename ever" award is "miscutil.c", which contains, well, miscellaneous utility functions like zalloc. (Suggestions for a better name for this file are welcome!) - Loaders now use a common function to read a line and skip comments, and this function also stores the proxy_args. - Added ./junkbuster --help (Not for Win32 GUI) - Added ./junkbuster --version (Not for Win32 GUI) - Win32 resources are now all marked as "U.S. English", rather than being a mix of "U.S. English", "U.K. English" and "Irish English". - Version number changes to 2.9.0 ---------------------------------------------------------------------- Copyright : Written by and Copyright (C) 2001-2016 the Privoxy team. https://www.privoxy.org/ Based on the Internet Junkbuster originally written by and Copyright (C) 1997 Anonymous Coders and Junkbusters Corporation. http://www.junkbusters.com/ This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. The GNU General Public License should be included with this file. If not, you can view it at http://www.gnu.org/copyleft/gpl.html or write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.