X-Git-Url: http://www.privoxy.org/gitweb/?p=privoxy.git;a=blobdiff_plain;f=doc%2Fwebserver%2Fuser-manual%2Fwhatsnew.html;h=776982d6e63947f51743f79cb0fdefb946259ccb;hp=fc2176896a9aafa1ef1fa7676d9450ff7d55ef7e;hb=84a6d458b85203e8cc651d7a360b2bf18e458e9f;hpb=e4446b36c844acb82cc754737b739fd0cdc402ed diff --git a/doc/webserver/user-manual/whatsnew.html b/doc/webserver/user-manual/whatsnew.html index fc217689..776982d6 100644 --- a/doc/webserver/user-manual/whatsnew.html +++ b/doc/webserver/user-manual/whatsnew.html @@ -1,482 +1,1003 @@ - -
There are many improvements and new features since Privoxy 3.0.8, the last stable release:
Added SOCKS5 support (with address resolution done by - the SOCKS5 server). Patch provided by Eric M. Hopper. -
The "blocked" CGI pages include a block reason that was - provided as argument to the last-applying block action. -
If enable-edit-actions is disabled (the default since 3.0.7 beta) - the show-status page hides the edit buttons and explains why. - Previously the user would get the "this feature has been disabled" - message after using the edit button. -
Forbidden CONNECT requests are treated like blocks by default. - The now-pointless treat-forbidden-connects-like-blocks action - has been removed. -
Not enabling limit-connect now allows CONNECT requests to all ports. - In previous versions it would only allow CONNECT requests to port 443. - Use +limit-connect{443} if you think you need the old default behaviour. -
The CGI editor gets turned off after three edit requests with invalid - file modification timestamps. This makes life harder for attackers - who can leverage browser bugs to send fake Referers and intend to - brute-force edit URLs. -
Action settings for multiple patterns in the same section are - shared in memory. As a result these sections take up less space - (and are loaded slightly faster). Problem reported by Franz Schwartau. -
Linear white space in HTTP headers will be normalized to single - spaces before parsing the header's content, headers split across - multiple lines get merged first. -
Host information is gathered outside the main thread so it's less - likely to delay other incoming connections if the host is misconfigured. -
New config option "hostname" to use a hostname other than - the one returned by the operating system. Useful to speed-up responses - for CGI requests on misconfigured systems. Requested by Max Khon. -
The CGI editor supports the "disable all filters of this type" - directives "-client-header-filter", "-server-header-filter", - "-client-header-tagger" and "-server-header-tagger". -
Fixed false-positives with the link-by-url filter and URLs that - contain the pattern "/jump/". -
The less-download-windows filter no longer messes - "Content-Type: application/x-shockwave-flash" headers up. -
In the show-url-info page's "Final results" section active and - inactive actions are listed separately. Patch provided by Lee. -
The GNUmakefile supports the DESTDIR variable. Patch for - the install target submitted by Radoslaw Zielinski. -
Embedding the content of configuration files in the show-status - page is significantly faster now. For a largish action file (1 MB) - a speedup of about 2450 times has been measured. This is mostly - interesting if you are using large action files or regularly use - Privoxy-Regression-Test while running Privoxy through Valgrind, - for stock configuration files it doesn't really matter. -
If zlib support is unavailable and there are content - filters active but the prevent-compression action is disabled, - the show-url-info page includes a warning that compression - might prevent filtering. -
The show-url-info page provides an OpenSearch Description that - allows to access the page through browser search plugins. -
The obsolete kill-popups action has been removed as the - PCRS-based popup filters can do the same and are slightly - less unreliable. -
The inspect-jpegs action has been removed. -
The send-wafer and send-vanilla-wafer actions have been removed. - They weren't particular useful and their behaviour could be emulated - with add-header anyway. -
Privoxy-Regression-Test has been significantly improved. -
Most sections in the default.action file contain tests for - Privoxy-Regression-Test to verify that they are working as intended. -
Parts of Privoxy have been refactored to increase maintainability. -
Building with zlib (if available) is done by default. -
Ordinary configuration file changes no longer cause program - termination on OS/2 if the name of the logfile hasn't been - changed as well. This regression probably crept in with the - logging improvements in 3.0.7. Reported by Maynard. -
The img-reorder filter is less likely to mess up JavaScript code in - img tags. Problem and solution reported by Glenn Washburn in #2014552. -
The source tar ball now includes Privoxy-Log-Parser, - a syntax-highlighter for Privoxy logs. Documentation is available - through perldoc(1), for fancy screenshots see: - http://www.fabiankeil.de/sourcecode/privoxy-log-parser/. -
For a more detailed list of changes please have a look at the ChangeLog.
A quick list of things to be aware of before upgrading from earlier - versions of Privoxy:
The recommended way to upgrade Privoxy is to backup your old - configuration files, install the new ones, verify that Privoxy - is working correctly and finally merge back your changes using - diff and maybe patch. -
There are a number of new features in each Privoxy release and - most of them have to be explicitly enabled in the configuration - files. Old configuration files obviously don't do that and due - to syntax changes using old configuration files with a new - Privoxy isn't always possible anyway. -
- Note that some installers remove earlier versions completely, - including configuration files, therefore you should really save - any important configuration files! -
- On the other hand, other installers don't overwrite existing configuration - files, thinking you will want to do that yourself. -
- standard.action now only includes the enabled actions. - Not all actions as before. -
In the default configuration only fatal errors are logged now. - You can change that in the debug section - of the configuration file. You may also want to enable more verbose - logging until you verified that the new Privoxy version is working - as expected. -
Three other config file settings are now off by default: - enable-remote-toggle, - enable-remote-http-toggle, - and enable-edit-actions. - If you use or want these, you will need to explicitly enable them, and - be aware of the security issues involved. -
The "filter-client-headers" and - "filter-server-headers" actions that were introduced with - Privoxy 3.0.5 to apply content filters to - the headers have been removed and replaced with new actions. - See the What's New section above. -
Privoxy 3.0.18 is a stable release. + The changes since 3.0.17 stable are:
+ +Bug fixes:
+ +Fix a logic bug that could cause Privoxy to reuse a server + socket after it got tainted by a server-header-tagger-induced + block that was triggered before the whole server response had + been read. If keep-alive was enabled and the request following + the blocked one was to the same host and using the same + forwarding settings, Privoxy would send it on the tainted server + socket. While the server would simply treat it as a pipelined + request, Privoxy would later on fail to properly parse the + server's response as it would try to parse the unread data from + the first response as server headers for the second one. + Regression introduced in 3.0.17.
+When implying keep-alive in client_connection(), remember that + the client didn't. Fixes a regression introduced in 3.0.13 that + would cause Privoxy to wait for additional client requests after + receiving a HTTP/1.1 request with "Connection: close" set and + connection sharing enabled. With clients which terminates the + client connection after detecting that the whole body has been + received it doesn't really matter, but with clients that don't + the connection would be kept open until it timed out.
+Fix a subtle race condition between + prepare_csp_for_next_request() and sweep() A thread preparing + itself for the next client request could briefly appear to be + inactive. If all other threads were already using more recent + files, the thread could get its files swept away under its feet. + So far this has only been reproduced while stress testing in + valgrind while touching action files in a loop. It's unlikely to + have caused any actual problems in the real world.
+Disable filters if SDCH compression is used unless filtering + is forced. If SDCH was combined with a supported compression + algorithm, Privoxy previously could try to decompress it and + ditch the Content-Encoding header even though the SDCH + compression wasn't dealt with. Reported by zebul666 in + #3225863.
+Make a copy of the --user value and only mess with that when + splitting user and group. On some operating systems modifying the + value directly is reflected in the output of ps and friends and + can be misleading. Reported by zepard in #3292710.
+If forwarded-connect-retries is set, only retry if Privoxy is + actually forwarding the request. Previously direct connections + would be retried as well.
+Fixed a small memory leak when retrying connections with IPv6 + support enabled.
+Remove an incorrect assertion in + compile_dynamic_pcrs_job_list() It could be triggered by a pcrs + job with an invalid pcre pattern (for example one that contains a + lone quantifier).
+If the --user argument user[.group] contains a dot, always + bail out if no group has been specified. Previously the intended, + but undocumented (and apparently untested), behaviour was to try + interpreting the whole argument as user name, but the detection + was flawed and checked for '0' instead of '\0', thus merely + preventing group names beginning with a zero.
+In html_code_map[], use a numeric character reference instead + of ' which wasn't standardized before XHTML 1.0.
+Fix an invalid free when compiled with + FEATURE_GRACEFUL_TERMINATION and shut down through + http://config.privoxy.org/die
+In get_actions(), fix the "temporary" backwards compatibility + hack to accept block actions without reason. It also covered + other actions that should be rejected as invalid. Reported by + Billy Crook.
+General improvements:
+ +Privoxy can (re)compress buffered content before delivering it + to the client. Disabled by default as most users wouldn't benefit + from it.
+The +fast-redirects{check-decoded-url} action checks URL + segments separately. If there are other parameters behind the + redirect URL, this makes it unnecessary to cut them off by + additionally using a +redirect{} pcrs command. Initial patch + submitted by Jamie Zawinski in #3429848.
+When loading action sections, verify that the referenced + filters exist. Currently missing filters only result in an error + message, but eventually the severity will be upgraded to + fatal.
+Allow to bind to multiple separate addresses. Patch set + submitted by Petr Pisar in #3354485.
+Set socket_error to errno if connecting fails in + rfc2553_connect_to() Previously rejected direct connections could + be incorrectly reported as DNS issues if Privoxy was compiled + with IPv6 support.
+Adjust url_code_map[] so spaces are replaced with %20 instead + of '+' While '+' can be used by client's submitting form data, + this is not actually what Privoxy is using the lookups for. This + is more of a cosmetic issue and doesn't fix any known + problems.
+When compiled without FEATURE_FAST_REDIRECTS, do not silently + ignore +fast-redirect{} directives
+Added a workaround for GNU libc's strptime() reporting + negative year values when the parsed year is only specified with + two digits. On affected systems cookies with such a date would + not be turned into session cookies by the +session-cookies-only + action. Reported by Vaeinoe in #3403560
+Fixed bind failures with certain GNU libc versions if no + non-loopback IP address has been configured on the system. This + is mainly an issue if the system is using DHCP and Privoxy is + started before the network is completely configured. Reported by + Raphael Marichez in #3349356. Additional insight from Petr + Pisar.
+Privoxy log messages now use the ISO 8601 date format + %Y-%m-%d. It's only slightly longer than the old format, but + contains the full date including the year and allows sorting by + date (when grepping in multiple log files) without hassle.
+In get_last_url(), do not bother trying to decode URLs that do + not contain at least one '%' sign. It reduces the log noise and a + number of unnecessary memory allocations.
+In case of SOCKS5 failures, dump the socks response in the log + message.
+Simplify the signal setup in main()
+Streamline socks5_connect() slightly
+In socks5_connect(), require a complete socks response from + the server Previously Privoxy didn't care how much data the + server response contained as long as the first two bytes + contained the expected values. While at it, shrink the buffer + size so Privoxy can't read more than a whole socks response.
+In chat(), do not bother to generate a client request in case + of direct CONNECT requests. It will not be used anyway.
+Reduce server_last_modified()'s stack size.
+Shorten get_http_time() by using strftime().
+Constify the known_http_methods pointers in + unknown_method().
+Constify the time_formats pointers in parse_header_time().
+Constify the formerly_valid_actions pointers in + action_used_to_be_valid().
+Introduce a GNUMakefile MAN_PAGE variable that defaults to + privoxy.1. The Debian package uses section 8 for the man page and + this should simplify the patch.
+Deduplicate the INADDR_NONE definition for Solaris by moving + it to jbsockets.h
+In block_url(), ditch the obsolete workaround for ancient + Netscape versions that supposedly couldn't properly deal with + status code 403.
+Remove a useless NULL pointer check in load_trustfile().
+Remove two useless NULL pointer checks in + load_one_re_filterfile().
+Change url_code_map[] from an array of pointers to an array of + arrays It removes an unnecessary layer of indirection and on + 64bit system reduces the size of the binary a bit.
+Fix various typos. Fixes taken from Debian's 29_typos.dpatch + by Roland Rosenfeld.
+Add a dok-tidy GNUMakefile target to clean up the messy HTML + generated by the other dok targets.
+GNUisms in the GNUMakefile have been removed.
+Change the HTTP version in static responses to 1.1
+Synced config.sub and config.guess with upstream + 2011-11-11/386c7218162c145f5f9e1ff7f558a3fbb66c37c5.
+Add a dedicated function to parse the values of toggles. + Reduces duplicated code in load_config() and provides better + error handling. Invalid or missing toggle values are now a fatal + error instead of being silently ignored.
+Terminate HTML lines in static error messages with \n instead + of \r\n.
+Simplify cgi_error_unknown() a bit.
+In LogPutString(), don't bother looking at pszText when not + actually logging anything.
+Change ssplit()'s fourth parameter from int to size_t. Fixes a + clang complaint.
+Add a warning that the statistics currently can't be trusted. + Mention Privoxy-Log-Parser's --statistics option as an + alternative for the time being.
+In rfc2553_connect_to(), start setting cgi->error_message + on error
+Change the expected status code returned for http://p.p/die + depending on whether or not FEATURE_GRACEFUL_TERMINATION is + available.
+In cgi_die(), mark the client connection for closing. If the + client will fetch the style sheet through another connection it + gets the main thread out of the accept() state and should thus + trigger the actual shutdown.
+Add a proper CGI message for cgi_die().
+Don't enforce a logical line length limit in + read_config_line()
+Slightly refactor server_last_modified() to remove useless + gmtime*() calls
+In get_content_type(), also recognize '.jpeg' as JPEG + extension
+Add '.png' to the list of recognized file extenstions in + get_content_type()
+In block_url(), consistently use the block reason "Request + blocked by Privoxy" In two places the reason was "Request for + blocked URL" which hides the fact that the request got blocked by + Privoxy and isn't necessarily correct as the block may be due to + tags.
+In listen_loop(), reload the configuration files after + accepting a new connection instead of before. Previously the + first connection that arrived after a configuration change would + still be handled with the old configuration.
+In chat()'s receive-data loop, skip a client socket check if + the socket will be written to right away anyway. This can + increase the transfer speed for unfiltered content on fast + network connections.
+The socket timeout is used for SOCKS negotiations as well + which previously couldn't timeout.
+Don't keep the client connection alive if any configuration + file changed since the time the connection came in. This is + closer to Privoxy's behaviour before keep-alive support for + client connection has been added and also less confusing in + general.
+Treat all Content-Type header values containing the pattern + 'script' as a sign of text. Reported by pribog in #3134970.
+Action file improvements:
+ +Moved the site-specific block pattern section below the one + for the generic patterns so for requests that are matched in + both, the block reason for the domain is shown which is usually + more useful than showing the one for the generic pattern.
+Remove -prevent-compression from the fragile alias It's no + longer used anywhere by default and isn't known to break stuff + anyway.
+Add a (disabled) section to block various Facebook tracking + URLs Reported by Dan Stahlke in #3421764.
+Add a (disabled) section to rewrite and redirect + click-tracking URLs used on news.google.com Reported by Dan + Stahlke in #3421755.
+Unblock linuxcounter.net/ Reported by Dan Stahlke in + #3422612.
+Block 'www91.intel.com/' which is used by Omniture. Reported + by Adam Piggott in #3167370.
+Disable the handle-as-empty-doc-returns-ok option and mark it + as deprecated. Reminded by tceverling in #2790091.
+Add ".ivwbox.de/" to the "Cross-site user tracking" section. + Reported by Nettozahler in #3172525.
+Unblock and fast-redirect ".awin1.com/.*=http://" Reported by + Adam Piggott in #3170921.
+Block "b.collective-media.net/".
+Widen the Debian popcon exception to "qa.debian.org/popcon". + Seen in Debian's 05_default_action.dpatch by Roland + Rosenfeld.
+Block ".gemius.pl/" which only seems to be used for user + tracking. Reported by johnd16 in #3002731. Additional input from + Lee and movax.
+Disable banners-by-size filters for '.thinkgeek.com/' The + filter only seems to catch pictures of the inventory.
+Block requests for 'go.idmnet.bbelements.com/please/showit/' + Reported by kacperdominik in #3372959.
+Unblock adainitiative.org/
+Add a fast-redirects exception for + '.googleusercontent.com/.*=cache'
+Add a fast-redirects exception for + webcache.googleusercontent.com/
+Unblock http://adassier.wordpress.com/ and + http://adassier.files.wordpress.com/
+Filter file improvements:
+ +Let the yahoo filter hide '.ads'
+Let the msn filter hide overlay ads for Facebook 'likes' in + search results and elements with the id 's_notf_div'. They only + seem to be used to advertise site 'enhancements'.
+Let the js-events filter additionally disarm setInterval() + Suggested by dg1727 in #3423775.
+Documentation improvements:
+ +Clarify the effect of compiling Privoxy with zlib support + Suggested by dg1727 in #3423782.
+Point out that the SourceForge messaging system works like a + black hole and should thus not be used to contact individual + developers.
+Mention some of the problems one can experience when not + explicitly configuring an IP addresses as listen address.
+Explicitly mention that hostnames can be used instead of IP + addresses for the listen-address, that only the first address + returned will be used and what happens if the address is invalid. + Requested by Calestyo in #3302213.
+Log message improvements:
+ +If only the server connection is kept alive, do not pretent to + wait for a new client request.
+Remove a superfluos log message in forget_connection()
+In chat(), properly report missing server responses as such + instead of calling them empty
+In forwarded_connect(), fix a log message nobody should ever + see
+Fix a log message in socks5_connect(), a failed write + operation was logged as failed read operation
+Let load_one_actions_file() properly complain about a missing + '{' at the beginning of the file Simply stating that a line is + invalid isn't particularly helpful.
+Do not claim to listen on a socket until Privoxy actually + does. Patch submitted by Petr Pisar #3354485
+Prevent a duplicated LOG_LEVEL_CLF message when sending out + the "no-server-data" response
+Also log the client socket when dropping a connection.
+Include the destination host in the 'Request ... marked for + blocking. limit-connect{...} doesn't allow CONNECT ...' message + Patch submitted by Saperski in #3296250.
+Prevent a duplicated log message if none of the resolved IP + addresses were reachable
+In connect_to(), do not pretend to retry if + forwarded-connect-retries is zero or unset.
+When a specified user or group can't be found, put the name in + single-quotes when logging it.
+In rfc2553_connect_to(), explain getnameinfo() errors + better.
+Remove a useless log message in chat()
+When retrying to connect, also log the maximum number of + connection attempts
+Rephrase a log message in compile_dynamic_pcrs_job_list(). + Divide the error code and its meaning with a colon. Call the pcrs + job dynamic and not the filter. Filters may contain dynamic and + non-dynamic pcrs jobs at the same time. Only mention the name of + the filter or tagger, but don't claim it's a filter when it could + be a tagger.
+In a fatal error message in load_one_actions_file(), cover + both URL and TAG patterns.
+In pcrs_strerror(), properly report unknown positive error + code values as such. Previously they were handled like 0 (no + error).
+In compile_dynamic_pcrs_job_list(), also log the actual error + code as pcrs_strerror() doesn't handle all errors reported by + pcre
+Don't bother trying to continue chatting if the client didn't + ask for it. Reduces log noise a bit.
+Make two fatal error message in load_one_actions_file() more + descriptive
+In cgi_send_user_manual(), log when rejecting a file name due + to '/' or '..'
+In load_file(), log a message if opening a file failed The CGI + error message alone isn't too helpful.
+In connection_destination_matches(), improve two log messages + to help understand why the destinations don't match.
+Rephrase a log message in serve(). Client request arrival + should be differentiated from closed client connections now.
+In serve(), log if a client connection isn't reused due to a + configuration file change.
+Let mark_server_socket_tainted() always mark the server socket + tainted, just don't talk about it in cases where it has no + effect. It doesn't change Privoxy's behaviour, but makes + understanding the log file easier.
+configure:
+ +Added a --disable-ipv6-support switch for platforms where + support is detected but doesn't actually work.
+Do not check for the existence of strerror() and memmove() + twice
+Remove a useless test for setpgrp(2). Privoxy doesn't need it + and it can cause problems when cross-compiling.
+Rename the --disable-acl-files switch to + --disable-acl-support. Since about 2001, ACL directives are + specified in the standard config file.
+Update the URL of the 'Removing outdated PCRE version after + the next stable release' posting. The old URL stopped working + after one of SF's recent site "optimizations". Reported by Han + Liu.
+Privoxy-Regression-Test:
+ +Added --shuffle-tests option to increase the chances of + detection race conditions.
+Added a --local-test-file option that allows to use + Privoxy-Regression-Test without Privoxy
+Added tests for missing socks4 and socks4a forwarders
+The --privoxy-address option now works with IPv6 addresses + containing brackets, too
+Perform limited sanity checks for parameters that are supposed + to have numerical values.
+Added a --sleep-time option to specify a number of seconds to + sleep between tests, defaults to 0.
+Disable the range-requests tagger for tests that break if it's + enabled
+Log messages use the ISO 8601 date format %Y-%m-%d.
+Fix spelling in two error messages.
+In the --help output, include a list of supported tests and + their default levels.
+Adjust the tests to properly deal with FEATURE_TOGGLE being + disabled.
+Privoxy-Log-Parser:
+ +Perform limited sanity checks for command line parameters that + are supposed to have numerical values.
+Implement a --unbreak-lines-only option to try to revert MUA + breakage.
+Accept and highlight: Added header: Content-Encoding: + deflate
+Accept and highlight: Compressed content from 29258 to 8630 + bytes.
+Accept and highlight: Client request arrived in time on socket + 21.
+Highlight: Didn't receive data in time: a.fsdn.com:443
+Accept log messages with ISO 8601 time stamps, too
+uagen:
+ +Bump generated Firefox version to 8.0
+Only randomize the release date if the new + --randomize-release-date option is enabled. Firefox versions + after 4 use a fixed date string without meaning.
+A quick list of things to be aware of before upgrading from earlier + versions of Privoxy:
+ +The recommended way to upgrade Privoxy is to backup your old configuration + files, install the new ones, verify that Privoxy is working correctly and finally merge + back your changes using diff and + maybe patch.
+ +There are a number of new features in each Privoxy release and most of them have to be + explicitly enabled in the configuration files. Old configuration + files obviously don't do that and due to syntax changes using old + configuration files with a new Privoxy isn't always possible anyway.
+Note that some installers remove earlier versions completely, + including configuration files, therefore you should really save any + important configuration files!
+On the other hand, other installers don't overwrite existing + configuration files, thinking you will want to do that + yourself.
+standard.action has been merged into + the default.action file.
+In the default configuration only fatal errors are logged now. + You can change that in the debug + section of the configuration file. You may also want to enable + more verbose logging until you verified that the new Privoxy version is working as expected.
+Three other config file settings are now off by default: + enable-remote-toggle, + enable-remote-http-toggle, + and enable-edit-actions. If you + use or want these, you will need to explicitly enable them, and be + aware of the security issues involved.
+