Privoxy 3.0.11 User Manual
Prev		Next

3. What's New in this Release

There are many improvements and new features since Privoxy 3.0.8, the last stable release:

Added SOCKS5 support (with address resolution done by - the SOCKS5 server). Patch provided by Eric M. Hopper. -
The "blocked" CGI pages include a block reason that was - provided as argument to the last-applying block action. -
If enable-edit-actions is disabled (the default since 3.0.7 beta) - the show-status page hides the edit buttons and explains why. - Previously the user would get the "this feature has been disabled" - message after using the edit button. -
Forbidden CONNECT requests are treated like blocks by default. - The now-pointless treat-forbidden-connects-like-blocks action - has been removed. -
Not enabling limit-connect now allows CONNECT requests to all ports. - In previous versions it would only allow CONNECT requests to port 443. - Use +limit-connect{443} if you think you need the old default behaviour. -
The CGI editor gets turned off after three edit requests with invalid - file modification timestamps. This makes life harder for attackers - who can leverage browser bugs to send fake Referers and intend to - brute-force edit URLs. -
Action settings for multiple patterns in the same section are - shared in memory. As a result these sections take up less space - (and are loaded slightly faster). Problem reported by Franz Schwartau. -
Linear white space in HTTP headers will be normalized to single - spaces before parsing the header's content, headers split across - multiple lines get merged first. -
Host information is gathered outside the main thread so it's less - likely to delay other incoming connections if the host is misconfigured. -
New config option "hostname" to use a hostname other than - the one returned by the operating system. Useful to speed-up responses - for CGI requests on misconfigured systems. Requested by Max Khon. -
The CGI editor supports the "disable all filters of this type" - directives "-client-header-filter", "-server-header-filter", - "-client-header-tagger" and "-server-header-tagger". -
Fixed false-positives with the link-by-url filter and URLs that - contain the pattern "/jump/". -
The less-download-windows filter no longer messes - "Content-Type: application/x-shockwave-flash" headers up. -
In the show-url-info page's "Final results" section active and - inactive actions are listed separately. Patch provided by Lee. -
The GNUmakefile supports the DESTDIR variable. Patch for - the install target submitted by Radoslaw Zielinski. -
Embedding the content of configuration files in the show-status - page is significantly faster now. For a largish action file (1 MB) - a speedup of about 2450 times has been measured. This is mostly - interesting if you are using large action files or regularly use - Privoxy-Regression-Test while running Privoxy through Valgrind, - for stock configuration files it doesn't really matter. -
If zlib support is unavailable and there are content - filters active but the prevent-compression action is disabled, - the show-url-info page includes a warning that compression - might prevent filtering. -
The show-url-info page provides an OpenSearch Description that - allows to access the page through browser search plugins. -
The obsolete kill-popups action has been removed as the - PCRS-based popup filters can do the same and are slightly - less unreliable. -
The inspect-jpegs action has been removed. -
The send-wafer and send-vanilla-wafer actions have been removed. - They weren't particular useful and their behaviour could be emulated - with add-header anyway. -
Privoxy-Regression-Test has been significantly improved. -
Most sections in the default.action file contain tests for - Privoxy-Regression-Test to verify that they are working as intended. -
Parts of Privoxy have been refactored to increase maintainability. -
Building with zlib (if available) is done by default. -
Ordinary configuration file changes no longer cause program - termination on OS/2 if the name of the logfile hasn't been - changed as well. This regression probably crept in with the - logging improvements in 3.0.7. Reported by Maynard. -
The img-reorder filter is less likely to mess up JavaScript code in - img tags. Problem and solution reported by Glenn Washburn in #2014552. -
The source tar ball now includes Privoxy-Log-Parser, - a syntax-highlighter for Privoxy logs. Documentation is available - through perldoc(1), for fancy screenshots see: - http://www.fabiankeil.de/sourcecode/privoxy-log-parser/. -

For a more detailed list of changes please have a look at the ChangeLog.

3.1. Note to Upgraders

A quick list of things to be aware of before upgrading from earlier - versions of Privoxy:

The recommended way to upgrade Privoxy is to backup your old - configuration files, install the new ones, verify that Privoxy - is working correctly and finally merge back your changes using - diff and maybe patch. -
There are a number of new features in each Privoxy release and - most of them have to be explicitly enabled in the configuration - files. Old configuration files obviously don't do that and due - to syntax changes using old configuration files with a new - Privoxy isn't always possible anyway. -
- Note that some installers remove earlier versions completely, - including configuration files, therefore you should really save - any important configuration files! -
- On the other hand, other installers don't overwrite existing configuration - files, thinking you will want to do that yourself. -
- standard.action now only includes the enabled actions. - Not all actions as before. -
In the default configuration only fatal errors are logged now. - You can change that in the debug section - of the configuration file. You may also want to enable more verbose - logging until you verified that the new Privoxy version is working - as expected. -
Three other config file settings are now off by default: - enable-remote-toggle, - enable-remote-http-toggle, - and enable-edit-actions. - If you use or want these, you will need to explicitly enable them, and - be aware of the security issues involved. -
The "filter-client-headers" and - "filter-server-headers" actions that were introduced with - Privoxy 3.0.5 to apply content filters to - the headers have been removed and replaced with new actions. - See the What's New section above. -

Privoxy 3.0.20 User Manual
Prev		Next

3. What's New in this + Release

+ +

Privoxy 3.0.20 is a beta release. The + changes since 3.0.19 stable are:

+ +

+
Bug fixes:
+ +
- +
  Client sockets are now properly shutdown and drained before + being closed. This fixes page truncation issues with clients that + aggressively pipeline data on platforms that otherwise discard + already written data. The issue mainly affected Opera users and + was initially reported by Kevin in #3464439, szotsaki provided + additional information to track down the cause.
  +
- +
  Fix latency calculation for shared connections (disabled by + default). It was broken since their introduction in 2009. The + calculated latency for most connections would be 0 in which case + the timeout detection failed to account for the real latency.
  +
- +
  Reject URLs with invalid port. Previously they were parsed + incorrectly and characters between the port number and the first + slash were silently dropped as shown by curl test 187.
  +
- +
  The default-server-timeout and socket-timeout directives + accept 0 as valid value.
  +
- +
  Fix a race condition on Windows that could cause Privoxy to + become unresponsive after toggling it on or off through the + taskbar icon. Reported by Tim H. in #3525694.
  +
- +
  Fix the compilation on Windows when configured without IPv6 + support.
  +
- +
  Fix an assertion that could cause debug builds to abort() in + case of socks5 connection failures with "debug 2" enabled.
  +
- +
  Fix an assertion that could cause debug builds to abort() if a + filter contained nul bytes in the replacement text.
  +
+
+
General improvements:
+ +
- +
  Significantly improved keep-alive support for both client and + server connections.
  +
- +
  New debug log level 65536 which logs all actions that were + applied to the request.
  +
- +
  New directive client-header-order to forward client headers in + a different order than the one in which they arrived.
  +
- +
  New directive tolerate-pipelining to allow client-side + pipelining. If enabled (3.0.20 beta enables it by default), + Privoxy will keep pipelined client requests around to deal with + them once the current request has been served.
  +
- +
  New --config-test option to let Privoxy exit after checking + whether or not the configuration seems valid. The limitations + noted in TODO #22 and #23 still apply. Based on a patch by + Ramkumar Chinchani.
  +
- +
  New limit-cookie-lifetime{} action to let cookies expire + before the end of the session. Suggested by Rick Sykes in + #1049575.
  +
- +
  Increase the hard-coded maximum number of actions and filter + files from 10 to 30 (each). It doesn't significantly affect + Privoxy's memory usage and recompiling wasn't an option for all + Privoxy users that reached the limit.
  +
- +
  Add support for chunk-encoded client request bodies. + Previously chunk-encoded request bodies weren't guaranteed to be + forwarded correctly, so this can also be considered a bug fix + although chunk-encoded request bodies aren't commonly used in the + real world.
  +
- +
  Add support for Tor's optimistic-data SOCKS extension, which + can reduce the latency for requests on newly created connections. + Currently only the headers are sent optimistically and only if + the client request has already been read completely which rules + out requests with large bodies.
  +
- +
  After preventing the client from pipelining, don't signal + keep-alive intentions. When looking at the response headers + alone, it previously wasn't obvious from the client's perspective + that no additional responses should be expected.
  +
- +
  Stop considering client sockets tainted after receving a + request with body. It hasn't been necessary for a while now and + unnecessarily causes test failures when using curl's test + suite.
  +
- +
  Allow HTTP/1.0 clients to signal interest in keep-alive + through the Proxy-Connection header. While such client are rare + in the real world, it doesn't hurt and couple of curl tests rely + on it.
  +
- +
  Only remove duplicated Content-Type headers when filters are + enabled. If they are not it doesn't cause ill effects and the + user might not want it. Downgrade the removal message to + LOG_LEVEL_HEADER to clarify that it's not an error in Privoxy and + is unlikely to cause any problems in general. Anonymously + reported in #3599335.
  +
- +
  Set the socket option SO_LINGER for the client socket.
  +
- +
  Move several variable declarations to the beginning of their + code block. It's required when compiling with gcc 2.95 which is + still used on some platforms. Initial patch submitted by Simon + South in #3564815.
  +
- +
  Optionally try to sanity-check strptime() results before + trusting them. Broken strptime() implementations have caused + problems in the past and the most recent offender seems to be + FreeBSD's libc (standards/173421).
  +
- +
  When filtering is enabled, let Range headers pass if the range + starts at the beginning. This should work around (or at least + reduce ) the video playback issues with various Apple clients as + reported by Duc in #3426305.
  +
- +
  Do not confuse a client hanging up with a connection time out. + If a client closes its side of the connection without sending a + request line, do not send the CLIENT_CONNECTION_TIMEOUT_RESPONSE, + but report the condition properly.
  +
- +
  Allow closing curly braces as part of action values as long as + they are escaped.
  +
- +
  On Windows, the logfile is now written before showing the GUI + error message which blocks until the user acknowledges it. + Reported by Adriaan in #3593603.
  +
- +
  Remove an unreasonable parameter limit in the CGI interface. + The new parameter limit depends on the memory available and is + currently unlikely to be reachable, due to other limits in both + Privoxy and common clients. Reported by Andrew on + ijbswa-users@.
  +
- +
  Decrease the chances of parse failures after requests with + unsupported methods were sent to the CGI interface.
  +
+
+
Action file improvements:
+ +
- +
  Remove the comment that indicated that updated default.action + versions are released on their own.
  +
- +
  Block 'optimize.indieclick.com/' and + 'optimized-by.rubiconproject.com/'
  +
- +
  Unblock 'adjamblog.wordpress.com/' and + 'adjamblog.files.wordpress.com/'. Reported by Ryan Farmer in + #3496116.
  +
- +
  Unblock '/.*Bugtracker'. Reported by pwhk in #3522341.
  +
- +
  Add test URLs for '.freebsd.org' and '.watson.org'.
  +
- +
  Unblock '.urbandictionary.com/popular'.
  +
- +
  Block '.adnxs.com/'.
  +
- +
  Block 'farm.plista.com/widgetdata.php'.
  +
- +
  Block 'rotation.linuxnewmedia.com/'.
  +
- +
  Block 'reklamy.sfd.pl/'. Reported by kacperdominik in + #3399948.
  +
- +
  Block 'g.adspeed.net/'.
  +
- +
  Unblock 'websupport.wdc.com/'. Reported by Adam Piggot in + #3577851.
  +
- +
  Block '/openx/www/delivery/'.
  +
- +
  Disable fast-redirects for '.googleapis.com/'.
  +
- +
  Block 'imp.double.net/'. Reported by David Bo in #3070411.
  +
- +
  Block 'gm-link.com/' whis is used for email tracking. Reported + by David Bo in #1812733.
  +
- +
  Verify that requests to "bwp." are blocked. URL taken from + #1736879 submitted by Francois Marier.
  +
- +
  Block '/.*bannerid='. Reported by Adam Piggott in + #2975779.
  +
- +
  Block 'cltomedia.info/delivery/' and '.adexprt.com/'. + Anonymously reported in #2965254.
  +
- +
  Block 'de17a.com/'. Reported by David Bo in #3061472.
  +
- +
  Block 'oskar.tradera.com/'. Reported by David Bo in + #3060596.
  +
- +
  Block '/scripts/webtrends\.js'. Reported by johnd16 in + #3002729.
  +
- +
  Block requests for 'pool.*.adhese.com/'. Reported by johnd16 + in #3002716.
  +
- +
  Update path pattern for Coremetrics and add tests. Pattern and + URLs submitted by Adam Piggott #3168443.
  +
- +
  Enable +fast-redirects{check-decoded-url} for 'tr.anp.se/'. + Reported by David Bo in #3268832.
  +
- +
  Unblock '.conrad.se/newsletter/banners/'. Reported by David Bo + in #3413824.
  +
- +
  Block '.tynt.com/'. Reported by Dan Stahlke in #3421767.
  +
- +
  Unblock '.bbci.co.uk/radio/'. Reported by Adam Piggott in + #3569603.
  +
- +
  Block requests to 'service.maxymiser.net/'. Reported by + johnd16 in #3118401 (with a previous URL).
  +
- +
  Disable fast-redirects for Google's "let's pretend your + computer is infected" page.
  +
- +
  Unblock '/.*download' to resolve actionsfile feedback + #3498129. Submitted by Steven Kolins (soundcloud.com not + working).
  +
- +
  Unblock '.wlxrs.com/' which is required by hotmail.com. Fixes + #3413827 submitted by David Bo.
  +
- +
  Add two unblock patterns for popup radio and TV players. + Submitted by Adam Piggott in #3596089.
  +
+
+
Filter file improvements & bug fixes:
+ +
- +
  Add a referer tagger.
  +
- +
  Reduce the likelihood that the google filter messes up + HTML-generating JavaScript. Reported by Zeno Kugy in + #3520260.
  +
+
+
Documentation improvements:
+ +
- +
  Revised all OS X sections due to new packaging module + (OSXPackageBuilder).
  +
- +
  Update the list of supported operating systems to clarify that + all Windows versions after 95 are expected to work and note that + the platform-specific code for AmigaOS and QNX currently isn't + maintained.
  +
- +
  Update 'Signals' section, the only explicitly handled signals + are SIGINT, SIGTERM and SIGHUP.
  +
- +
  Add Haiku to the list of operating systems on which Privoxy is + known to run.
  +
- +
  Add DragonFly to the list of BSDs on which Privoxy is known to + run.
  +
- +
  Removed references to redhat-specific documentation set since + it no longer exists.
  +
- +
  Removed references to building PDFs since we no longer do + so.
  +
- +
  Multiple listen-address directives are supported since 3.0.18, + correct the documentation to say so.
  +
- +
  Remove bogus section about long and short being preferable to + int.
  +
- +
  Corrected some Internet JunkBuster references to Privoxy.
  +
- +
  Removed references to www.junkbusters.com since it is no + longer maintained. Reported by Angelina Matson.
  +
- +
  Various grammar and spelling corrections
  +
- +
  Add a client-header-tagger{} example for disabling filtering + for range requests.
  +
- +
  Correct a URL in the "Privoxy with Tor" FAQ.
  +
- +
  Spell 'refresh-tags' correctly. Reported by Don in + #3571927.
  +
- +
  Sort manpage options alphabetically.
  +
- +
  Remove an incorrect sentence in the toggle section. The toggle + state doesn't affect whether or not the Windows version uses the + tray icon. Reported by Zeno Kugy in #3596395.
  +
- +
  Add new contributors since 3.0.19.
  +
+
+
Log message improvements:
+ +
- +
  When stopping to watch a client socket due to pipelining, + additionally log the socket number.
  +
- +
  Log the client socket and its condition before closing it. + This makes it more obvious that the socket actually gets closed + and should help when diagnosing problems like #3464439.
  +
- +
  In case of SOCKS5 failures, do not explicitly log the server's + response. It hasn't helped so far and the response can already be + logged by enabling "debug 32768" anyway. This reverts v1.81 and + the follow-up bug fix v1.84.
  +
- +
  Relocate the connection-accepted message from listen_loop() to + serve(). This way it's printed by the thread that is actually + serving the connection which is nice when grepping for thread ids + in log files.
  +
+
+
Code cleanups:
+ +
- +
  Remove compatibility layer for versions prior to 3.0 since it + has been obsolete for more than 10 years now.
  +
- +
  Remove the ijb_isupper() and ijb_tolower() macros from + parsers.c since they aren't used in this file.
  +
- +
  Removed the 'Functions declared include:' comment sections + since they tend to be incomplete, incorrect and out of date and + the benefit seems questionable.
  +
- +
  Various comment grammar and comprehensibility + improvements.
  +
- +
  Remove a pointless fflush() call in chat(). Flushing all + streams pretty much all the time for no obvious reason is + ridiculous.
  +
- +
  Relocate ijb_isupper()'s definition to project.h and get the + ijb_tolower() definition from there, too.
  +
- +
  Relocate ijb_isdigit()'s definition to project.h.
  +
- +
  Rename ijb_foo macros to privoxy_foo.
  +
- +
  Add malloc_or_die() which will allow to simplify code paths + where malloc() failures don't need to be handled gracefully.
  +
- +
  Add strdup_or_die() which will allow to simplify code paths + where strdup() failures don't need to be handled gracefully.
  +
- +
  Replace strdup() calls with strdup_or_die() calls where it's + safe and simplifies the code.
  +
- +
  Fix white-space around parentheses.
  +
- +
  Add missing white-space behind if's and the following + parentheses.
  +
- +
  Unwrap a memcpy() call in resolve_hostname_to_ip().
  +
- +
  Declare pcrs_get_delimiter()'s delimiters[] static const.
  +
- +
  Various optimisations to remove dead code and merge + inefficient code structures for improved clarity, performance or + code compactness.
  +
- +
  Various data type corrections.
  +
- +
  Change visibility of several code segments when compiling + without FEATURE_CONNECTION_KEEP_ALIVE enabled for clarity.
  +
- +
  In pcrs_get_delimiter(), do not use delimiters ouside the + ASCII range. Fixes a clang complaint.
  +
- +
  Fix an error message in get_last_url() nobody is supposed to + see. Reported by Matthew Fischer in #3507301.
  +
- +
  Fix a typo in the no-zlib-support complaint. Patch submitted + by Matthew Fischer in #3507304.
  +
- +
  Shorten ssplit()'s prototype by removing the last two + arguments. We always want to skip empty fields and ignore leading + delimiters, so having parameters for this only complicates the + API.
  +
- +
  Use an enum for the type of the action value.
  +
- +
  Rename action_name's member takes_value to value_type as it + isn't used as boolean.
  +
- +
  Turn family mismatches in match_sockaddr() into fatal + errors.
  +
- +
  Let enlist_unique_header() verify that the caller didn't pass + a header containing either \r or \n.
  +
- +
  Change the hashes used in load_config() to unsigned int. + That's what hash_string() actually returns and using a + potentiallly larger type is at best useless.
  +
- +
  Use privoxy_tolower() instead of vanilla tolower() with manual + casting of the argument.
  +
- +
  Catch ssplit() failures in parse_cgi_parameters().
  +
+
+
Privoxy-Regression-Test:
+ +
- +
  Add an 'Overwrite condition' directive to skip any matching + tests before it. As it has a global scope, using it is more + convenient than clowning around with the Ignore directive.
  +
- +
  Log to STDOUT instead of STDERR.
  +
- +
  Include the Privoxy version in the output.
  +
- +
  Various grammar and spelling corrections in documentation and + code.
  +
- +
  Additional tests for range requests with filtering + enabled.
  +
- +
  Tests with mostly invalid range request.
  +
- +
  Add a couple of hide-if-modified-since{} tests with different + date formats.
  +
- +
  Cleaned up the format of the regression-tests.action file to + match the format of default.action.
  +
- +
  Remove the "Copyright" line from print_version(). When using + --help, every line of screen space matters and thus shouldn't be + wasted on things the user doesn't care about.
  +
+
+
Privoxy-Log-Parser:
+ +
- +
  Improve the --statistics performance by skipping sanity checks + for input that shouldn't affect the results anyway. Add a + --strict-checks option that enables some of the checks again, + just in case anybody cares.
  +
- +
  The distribution of client requests per connection is included + in the --statistic output.
  +
- +
  The --accept-unknown-messages option has been removed and the + behavior is now the default.
  +
- +
  Accept and (mostly) highlight new log messages introduced with + Privoxy 3.0.20.
  +
+
+
uagen:
+ +
- +
  Bump generated Firefox version to 17.
  +
+
+
GNUmakefile improvements:
+ +
- +
  The dok-tidy target no longer taints documents with a + tidy-mark
  +
- +
  Change RA_MODE from 0664 to 0644. Suggested by Markus Dittrich + in #3505445.
  +
- +
  Remove tidy's clean flag as it changes the scope of + attributes. Link-specific colors end up being applied to all + text. Reported by Adam Piggott in #3569551.
  +
- +
  Leave it up to the user whether or not smart tags are + inserted.
  +
- +
  Let w3m itself do the line wrapping for the config file. It + works better than fmt as it can honour pre tags causing less + unintentional line breaks.
  +
- +
  Ditch a pointless '-r' passed to rm to delete files.
  +
- +
  The config-file target now requires less manual intervention + and updates the original config.
  +
- +
  Change WDUMP to generate ASCII. Add WDUMP_UTF8 to allow UTF-8 + in the AUTHORS file so the names are right.
  +
- +
  Stop pretending that lynx and links are supported for the + documentation.
  +
+
+
configure improvements:
+ +
- +
  On Haiku, do not pass -lpthread to the compiler. Haiku's + pthreads implementation is contained in its system library, + libroot, so no additional library needs to be searched. Patch + submitted by Simon South in #3564815.
  +
- +
  Additional Haiku-specific improvements. Disable checks + intended for multi-user systems as Haiku is presently + single-user. Group Haiku-specific settings in their own section, + following the pattern for Solaris, OS/2 and AmigaOS. Add + additional library-related settings to remove the need for + providing configure with custom LDFLAGS. Submitted by Simon South + in #3574538. *** Version 3.0.19 Stable ***
  +
+
+
Bug fixes:
+ +
- +
  Prevent a segmentation fault when de-chunking buffered + content. It could be triggered by malicious web servers if + Privoxy was configured to filter the content and running on a + platform where SIZE_T_MAX isn't larger than UINT_MAX, which + probably includes most 32-bit systems. On those platforms, all + Privoxy versions before 3.0.19 appear to be affected. To be on + the safe side, this bug should be presumed to allow code + execution as proving that it doesn't seems unrealistic.
  +
- +
  Do not expect a response from the SOCKS4/4A server until it + got something to respond to. This regression was introduced in + 3.0.18 and prevented the SOCKS4/4A negotiation from working. + Reported by qqqqqw in #3459781.
  +
+
+
General improvements:
+ +
- +
  Fix an off-by-one in an error message about connect + failures.
  +
- +
  Use a GNUMakefile variable for the webserver root directory + and update the path. Sourceforge changed it which broke various + web-related targets.
  +
- +
  Update the CODE_STATUS description.
  +
+

+ +

3.1. Note + to Upgraders

+ +

A quick list of things to be aware of before upgrading from earlier + versions of Privoxy:

+ +

+
The recommended way to upgrade Privoxy is to backup your old configuration + files, install the new ones, verify that Privoxy is working correctly and finally merge + back your changes using diff and + maybe patch.
+ +
There are a number of new features in each Privoxy release and most of them have to be + explicitly enabled in the configuration files. Old configuration + files obviously don't do that and due to syntax changes using old + configuration files with a new Privoxy isn't always possible anyway.
+
+
Note that some installers remove earlier versions completely, + including configuration files, therefore you should really save any + important configuration files!
+
+
On the other hand, other installers don't overwrite existing + configuration files, thinking you will want to do that + yourself.
+
+
standard.action has been merged into + the default.action file.
+
+
In the default configuration only fatal errors are logged now. + You can change that in the debug + section of the configuration file. You may also want to enable + more verbose logging until you verified that the new Privoxy version is working as expected.
+
+
Three other config file settings are now off by default: + enable-remote-toggle, + enable-remote-http-toggle, + and enable-edit-actions. If you + use or want these, you will need to explicitly enable them, and be + aware of the security issues involved.
+

Prev	Home	Next
Installation		Quickstart to Using Privoxy

Prev	Home	Next
Installation		Quickstart to Using + Privoxy