Privoxy 3.0.4 User Manual
Prev		Next

Privoxy 3.0.18 User Manual
Prev		Next

3. What's New in this + Release

+ +

Privoxy 3.0.18 is a stable release. + The changes since 3.0.17 stable are:

+ +

+
Bug fixes:
+ +
- +
  Fix a logic bug that could cause Privoxy to reuse a tainted + server socket. It could happen for server sockets that got + tainted by a server-header-tagger-induced block, in which case + Privoxy doesn't necessarily read the whole server response. If + keep-alive was enabled and the request following the blocked one + was to the same host and using the same forwarding settings, + Privoxy would send it on the tainted server socket. While the + server would simply treat it as a pipelined request, Privoxy + would later on fail to properly parse the server's response as it + would try to parse the unread data from the first response as + server headers for the second one. Regression introduced in + 3.0.17.
  +
- +
  When implying keep-alive in client_connection(), remember that + the client didn't Fixes a regression introduced in 3.0.13 that + would cause Privoxy to wait for additional client requests after + receiving a HTTP/1.1 request with "Connection: close" set and + connection sharing enabled. With clients like curl which + terminates the client connection after detecting that the whole + body has been received it doesn't really matter, but with clients + like FreeBSD's fetch the client connection would be kept open + until it timed out.
  +
- +
  Fix a subtle race condition between + prepare_csp_for_next_request() and sweep() A thread preparing + itself for the next client request could briefly appear to be + inactive. If all other threads were already using more recent + files, the thread could get its files swept away under its feet. + I've only seen it while stress testing in valgrind while touching + action files in a loop. It's unlikely to have caused any actual + problems in the real world.
  +
+
+
General improvements:
+ +
- +
  Privoxy can (re)compress buffered content before delivering it + to the client. Disabled by default as most users wouldn't benefit + from it.
  +
- +
  The +fast-redirects{check-decoded-url} action checks URL + segments separately. If there are other parameters behind the + redirect URL, this makes it unnecessary to cut them of by + additionally using a +redirect{} pcrs command. Initial patch + submitted by Jamie Zawinski in #3429848.
  +
- +
  Properly deal with FEATURE_TOGGLE being disabled
  +
- +
  Adjust url_code_map[] so spaces are replaced with %20 instead + of '+' While '+' can be used by client's submitting form data, + this is not actually what Privoxy is using the lookups for. This + is more of a cosmetic issue and doesn't fix any actual problems + I'm aware of.
  +
- +
  When compiled without FEATURE_FAST_REDIRECTS, do not silently + ignore +fast-redirect{} directives
  +
- +
  Added a workaround for GNU libc's strptime() reporting + negative year values when the parsed year is only specified with + two digits. On affected systems cookies with such a date would + not be turned into session cookies by the +session-cookies-only + action. Reported by Vaeinoe in #3403560
  +
- +
  When loading action sections, verify that the referenced + filters exist Currently missing filters only result in an error + message, but eventually the severity will be upgraded to + fatal.
  +
- +
  Allow to bind to multiple separate addresses. Patch set + submitted by Petr Pisar in #3354485.
  +
- +
  Set socket_error to errno if connecting fails in + rfc2553_connect_to() Previously rejected direct connections could + be incorrectly reported as DNS issues.
  +
- +
  Fixed bind failures with certain GNU libc versions if no + non-loopback IP address has been configured on the system. This + is mainly an issue if the system is using DHCP and Privoxy is + started before the network is completely configured. Reported by + Raphael Marichez in #3349356. Additional insight from Petr + Pisar.
  +
- +
  Disable filters if SDCH compression is used unless filtering + is forced. If SDCH was combined with a supported compression + algorithm, we'd previously try to decompress it, when successful + apply the enabled filters and ditch the Content-Encoding header + even though the SDCH compression wasn't removed. Reported by + zebul666 in #3225863.
  +
- +
  Privoxy log messages now use the ISO 8601 date format + %Y-%m-%d. It's only slightly longer than the old format, but + contains the full date including the year and allows sorting by + date (when grepping in multiple log files) without hassle.
  +
- +
  Make a copy of the --user value and only mess with that when + splitting user and group. On some operating systems modifying the + value directly is reflected in the output of ps and friends and + can be misleading. Reported by zepard in #3292710.
  +
- +
  If forwarded-connect-retries is set, only retry if the we are + actually forwarding the request. Previously direct connections + would be retried as well.
  +
- +
  Fixed a small memory leak when retrying connection
  +
- +
  Remove an incorrect assertion in + compile_dynamic_pcrs_job_list() It could be triggered by a pcrs + job with an invalid pcre pattern (for example one that contains a + lone quantifier).
  +
+
+
Action file improvements:
+ +
- +
  Moved the site-specific block pattern section below the one + for the generic patterns so for requests that are matched in + both, the block reason for the domain is shown which is usually + more useful than showing the one for the generic pattern.
  +
- +
  Add a (disabled) section to block various Facebook tracking + URLs Reported by Dan Stahlke in #3421764.
  +
- +
  Add a (disabled) section to rewrite and redirect + click-tracking URLs used on news.google.com Reported by Dan + Stahlke in #3421755.
  +
- +
  Unblock linuxcounter.net/ Reported by Dan Stahlke in + #3422612.
  +
- +
  Block 'www91.intel.com/' which is used by Omniture. Reported + by Adam Piggott in #3167370.
  +
- +
  Disable the handle-as-empty-doc-returns-ok option and mark it + as deprecated. Reminded by tceverling in #2790091.
  +
- +
  Add ".ivwbox.de/" to the "Cross-site user tracking" section. + Reported by Nettozahler in #3172525.
  +
- +
  Unblock and fast-redirect ".awin1.com/.*=http://" Reported by + Adam Piggott in #3170921.
  +
- +
  Block "b.collective-media.net/".
  +
- +
  Widen the Debian popcon exception to "qa.debian.org/popcon". + Seen in Debian's 05_default_action.dpatch by Roland + Rosenfeld.
  +
- +
  Block ".gemius.pl/" which only seems to be used for user + tracking. Reported by johnd16 in #3002731. Additional input from + Lee and movax.
  +
- +
  Disable banners-by-size filters for '.thinkgeek.com/' The + filter only seems to catch pictures of the inventory.
  +
- +
  Block requests for 'go.idmnet.bbelements.com/please/showit/' + Reported by kacperdominik in #3372959.
  +
- +
  Unblock adainitiative.org/
  +
- +
  Add a fast-redirects exception for + '.googleusercontent.com/.*=cache'
  +
- +
  Add a fast-redirects exception for + webcache.googleusercontent.com/
  +
- +
  Remove -prevent-compression from the fragile alias It's no + longer used anywhere by default and isn't known to break stuff + anyway.
  +
- +
  Unblock http://adassier.wordpress.com/ and + http://adassier.files.wordpress.com/
  +
+
+
Filter file improvements:
+ +
- +
  Let the yahoo filter hide '.ads'
  +
- +
  Let the msn filter hide overlay ads for Facebook 'likes' in + search results.
  +
- +
  Let the msn filter hide elements with the id 's_notf_div'. + They only seem to be used to advertise site 'enhancements'.
  +
- +
  Let the js-events filter additionally disarm setInterval() + Suggested by dg1727 in #3423775.
  +
+
+
Documentation improvements:
+ +
- +
  Clarify the effect of compiling Privoxy with zlib support + Suggested by dg1727 in #3423782.
  +
- +
  Point out that the SourceForge messaging system works like a + blackhole and should thus not be used
  +
- +
  Mention some of the problems one can experience when not + explicitly configuring an IP addresses as listen address.
  +
- +
  Explicitly mention that hostnames can be used instead of IP + addresses for the listen-address, that only the first address + returned will be used and what happens if the address is invalid. + Requested by Calestyo in #3302213.
  +
+
+
Log message improvements:
+ +
- +
  If only the server connection is kept alive, do not pretent to + wait for a new client request.
  +
- +
  Remove a superfluos log message in forget_connection()
  +
- +
  In chat(), properly report missing server responses as such + instead of calling them empty
  +
- +
  In forwarded_connect(), fix a log message nobody should ever + see
  +
- +
  Fix a log message in socks5_connect(), a failed write + operation was logged as failed read operation
  +
- +
  Let load_one_actions_file() properly complain about a missing + '{' at the beginning of the file Simply stating that a line is + invalid isn't particularly helpful.
  +
- +
  Do not claim to listen on a socket until we actually do. Patch + submitted by Petr Pisar #3354485
  +
- +
  Prevent a duplicated LOG_LEVEL_CLF message when sending out + the "no-server-data" response
  +
- +
  Also log the client socket when dropping a connection.
  +
- +
  Include the destination host in the 'Request ... marked for + blocking. limit-connect{...} doesn't allow CONNECT ...' message + Patch submitted by Saperski in #3296250.
  +
- +
  Prevent a duplicated log message if none of the resolved IP + addresses were reachable
  +
- +
  In connect_to(), do not pretend to retry if + forwarded-connect-retries is zero or unset.
  +
- +
  When a specified user or group can't be found, put the name in + single-quotes when logging it.
  +
- +
  In rfc2553_connect_to(), explain getnameinfo() errors + differently.
  +
- +
  Remove a useless log message in chat()
  +
- +
  When retrying to connect, also log the maximum number of + connection attempts
  +
- +
  Rephrase a log message in compile_dynamic_pcrs_job_list() + Divide the error code and its meaning with a colon. Call the pcrs + job dynamic and not the filter. Filters may contain dynamic and + non-dynamic pcrs jobs at the same time. Only mention the name of + the filter or tagger, but don't claim it's a filter when it could + be a tagger.
  +
- +
  In a fatal error message in load_one_actions_file(), cover + both URL and TAG patterns
  +
- +
  In pcrs_strerror(), properly report unknown positive error + code values as unknown. Previously they were handled like 0 (no + error).
  +
- +
  In compile_dynamic_pcrs_job_list(), also log the actual error + code as pcrs_strerror() doesn't handle all errors reported by + pcre
  +
- +
  Don't bother trying to continue chatting if the client didn't + ask for it. Reduces log noise a bit.
  +
- +
  Make two fatal error message in load_one_actions_file() more + descriptive
  +
- +
  In cgi_send_user_manual(), log when rejecting a file name due + to '/' or '..'
  +
- +
  In load_file(), log a message if opening a file failed The CGI + error message alone isn't too helpful.
  +
- +
  In connection_destination_matches(), improve two log messages + to help understand why the destinations don't match
  +
- +
  Rephrase a log message in serve(). Client request arrival + should be differentiated from closed client connections now.
  +
- +
  In serve(), log if a client connection isn't reused due to a + configuration file change.
  +
- +
  Let mark_server_socket_tainted() always mark the server socket + tainted, just don't talk about it in cases where it has no + effect. It doesn't change Privoxy's behaviour, but makes + understanding the log file easier.
  +
+
+
Miscellaneous Privoxy improvements:
+ +
- +
  In get_last_url(), do not bother trying to decode URLs that do + not contain at least one '%' sign. It reduces the log noise and a + number of unnecessary memory allocations.
  +
- +
  If the --user argument user[.group] contains a dot, always + bail out if no group has been specified. Previously the intended, + but undocumented (and apparently untested), behaviour was to try + interpreting the whole argument as user name, but the detection + was flawed and checked for '0' isntead of '\0', thus merely + preventing group names beginning with a zero.
  +
- +
  Simplify the signal setup in main()
  +
- +
  Streamline socks5_connect() slightly
  +
- +
  In case of SOCKS5 failures, dump the socks response
  +
- +
  In socks5_connect(), require a complete socks response from + the server Previously we didn't care how much data the server + response contained as long as the first two bytes contained the + expected values. While at it, shrink the buffer size so we can't + read more than a whole socks response. This is required to + support Tor's optimistic data extension.
  +
- +
  In chat(), do not bother to generate a client request in case + of direct CONNECT requests
  +
- +
  Reduce server_last_modified()'s stack size
  +
- +
  Shorten get_http_time() by using strftime()
  +
- +
  Constify the known_http_methods pointers in + unknown_method()
  +
- +
  Constify the time_formats pointers in parse_header_time()
  +
- +
  Constify the formerly_valid_actions pointers in + action_used_to_be_valid()
  +
- +
  In html_code_map[], use a numeric character reference instead + of ' which wasn't standardized before XHTML 1.0
  +
- +
  Introduce a MAN_PAGE variable that defaults to privoxy.1. The + Debian package uses section 8 for the man page and this should + simplify the patch.
  +
- +
  Deduplicate the INADDR_NONE definition for Solaris by moving + it to jbsockets.h
  +
- +
  In block_url(), ditch the obsolete workaround for ancient + Netscape versions that supposedly couldn't properly deal with + status code 403.
  +
- +
  Remove a useless NULL pointer check in load_trustfile()
  +
- +
  Remove two useless NULL pointer checks in + load_one_re_filterfile().
  +
- +
  Change url_code_map[] from an array of pointers to an array of + arrays It removes an unnecessary layer of indirection and on + 64bit system reduces the size of the binary a bit.
  +
- +
  Fix various typos. Fixes taken from Debian's 29_typos.dpatch + by Roland Rosenfeld.
  +
- +
  Add a dok-tidy GNUMakefile target to clean up the messy HTML + generated by the other dok targets.
  +
- +
  GNUisms in the GNUMakefile have been removed.
  +
- +
  Change the HTTP version in static responses to 1.1
  +
- +
  Synced config.sub and config.guess with upstream + 2011-11-11/386c7218162c145f5f9e1ff7f558a3fbb66c37c5.
  +
- +
  Add a dedicated function to parse the values of toggles + Reduces duplicated code in load_config() and provides better + error handling. Invalid or missing toggle values are now a fatal + error instead of being silently ignored.
  +
- +
  Terminate HTML lines in static error messages with \n instead + of \r\n.
  +
- +
  Simplify cgi_error_unknown() a bit.
  +
- +
  In LogPutString(), don't bother looking at pszText when not + actually logging anything
  +
- +
  Change ssplit()'s fourth parameter from int to size_t. Fixes a + clang complaint.
  +
- +
  Add a warning that the statistics currently can't be trusted. + Mention Privoxy-Log-Parser's --statistics option as an + alternative for the time being.
  +
- +
  In rfc2553_connect_to(), start setting cgi->error_message + on error
  +
- +
  Change the expected status code returned for http://p.p/die + depending on whether or not FEATURE_GRACEFUL_TERMINATION is + available.
  +
- +
  In cgi_die(), mark the client connection for closing. If the + client will fetch the style sheet through another connection it + gets the main thread out of the accept() state and should thus + trigger the actual shutdown.
  +
- +
  Add a proper CGI message for cgi_die().
  +
- +
  Fix an invalid free when compiled with + FEATURE_GRACEFUL_TERMINATION and shut down through + http://config.privoxy.org/die
  +
- +
  Don't enforce a logical line length limit in + read_config_line()
  +
- +
  Slightly refactor server_last_modified() to remove useless + gmtime*() calls
  +
- +
  In get_content_type(), also recognize '.jpeg' as JPEG + extension
  +
- +
  Add '.png' to the list of recognized file extenstions in + get_content_type()
  +
- +
  In block_url(), consistently use the block reason "Request + blocked by Privoxy" In two places the reason was "Request for + blocked URL" which hides the fact that the request got blocked by + Privoxy and isn't necessarly correct as the block may be due to + tags.
  +
- +
  In get_actions(), fix the "temporary" backwards compatibility + hack to accept block actions without reason. It also covered + other actions that should be rejected as invalid. Reported by + Billy Crook.
  +
- +
  In listen_loop(), reload the configuration files after + accepting a new connection instead of before. Previously the + first connection that arrived after a configuration change would + still be handled with the old configuration.
  +
- +
  In chat()'s receive-data loop, skip a client socket check if + the socket will be written to right away anyway. This can + increase the transfer speed for unfiltered content on fast + network connections.
  +
- +
  The socket timeout is used for SOCKS negotiation as well.
  +
- +
  Don't keep the client connection alive if any configuration + file changed since the time the connection came in. This is + closer to Privoxy's behaviour before keep-alive support for + client connection has been added and also less confusing in + general.
  +
- +
  Treat all Content-Type header values containing the pattern + 'script' as a sign of text. Reported by pribog in #3134970.
  +
+
+
configure:
+ +
- +
  Added a --disable-ipv6-support switch for platforms where + support is detected but doesn't actually work.
  +
- +
  Do not check for the existence of strerror() and memmove() + twice
  +
- +
  Remove a useless test for setpgrp(2). Privoxy doesn't need it + and it can cause problems when cross-compiling
  +
- +
  Rename the --disable-acl-files switch to --disable-acl-support + Since about 2001, ACL directives are specified in the standard + config file.
  +
- +
  Update the URL of the 'Removing outdated PCRE version after + the next stable release' posting. The old URL stopped working + after one of SF's recent layout pessimizations. Reported by Han + Liu.
  +
+
+
Privoxy-Regression-Test:
+ +
- +
  Added --shuffle-tests option to increase the chances of + detection race conditions
  +
- +
  Added a --local-test-file option that allows to use + Privoxy-Regression-Test without Privoxy
  +
- +
  Added tests for missing socks4 and socks4a forwarders
  +
- +
  The --privoxy-address option now works with IPv6 addresses + containing brackets, too
  +
- +
  Perform limited sanity checks for parameters that are supposed + to have numerical values.
  +
- +
  Added a --sleep-time option to specify a number of seconds to + sleep between tests, defaults to 0.
  +
- +
  Disable the range-requests tagger for tests that break if it's + enabled
  +
- +
  Log messages use the ISO 8601 date format %Y-%m-%d.
  +
- +
  Fix spelling in two error messages.
  +
- +
  In the --help output, include a list of supported tests and + their default levels.
  +
+
+
Privoxy-Log-Parser:
+ +
- +
  Perform limited sanity checks for parameters that are supposed + to have numerical values.
  +
- +
  Implement a --unbreak-lines-only option to try to revert MUA + breakage.
  +
- +
  Accept and highlight: Added header: Content-Encoding: + deflate
  +
- +
  Accept and highlight: Compressed content from 29258 to 8630 + bytes.
  +
- +
  Accept and highlight: Client request arrived in time on socket + 21.
  +
- +
  Highlight: Didn't receive data in time: a.fsdn.com:443
  +
- +
  Accept log messages with ISO 8601 time stamps, too
  +
+
+
uagen:
+ +
- +
  Bump generated Firefox version to 9.0
  +
- +
  Only randomize the release date if the new + --randomize-release-date option is enabled. Firefox versions + after 4 use a fixed date string without meaning.
  +
+

+ +

3.1. Note + to Upgraders

+ +

A quick list of things to be aware of before upgrading from earlier + versions of Privoxy:

+ +

+
The recommended way to upgrade Privoxy is to backup your old configuration + files, install the new ones, verify that Privoxy is working correctly and finally merge + back your changes using diff and + maybe patch.
+ +
There are a number of new features in each Privoxy release and most of them have to be + explicitly enabled in the configuration files. Old configuration + files obviously don't do that and due to syntax changes using old + configuration files with a new Privoxy isn't always possible anyway.
+
+
Note that some installers remove earlier versions completely, + including configuration files, therefore you should really save any + important configuration files!
+
+
On the other hand, other installers don't overwrite existing + configuration files, thinking you will want to do that + yourself.
+
+
standard.action has been merged into + the default.action file.
+
+
In the default configuration only fatal errors are logged now. + You can change that in the debug + section of the configuration file. You may also want to enable + more verbose logging until you verified that the new Privoxy version is working as expected.
+
+
Three other config file settings are now off by default: + enable-remote-toggle, + enable-remote-http-toggle, + and enable-edit-actions. If you + use or want these, you will need to explicitly enable them, and be + aware of the security issues involved.
+

Prev	Home	Next
Installation		Quickstart to Using Privoxy

Prev	Home	Next
Installation		Quickstart to Using + Privoxy

3. What's New in this Release

3.1. Note to Upgraders

3. What's New in this + Release

3.1. Note + to Upgraders