<!entity license SYSTEM "license.sgml">
<!entity p-authors SYSTEM "p-authors.sgml">
<!entity config SYSTEM "p-config.sgml">
-<!entity p-version "3.0.14">
-<!entity p-status "BETA">
+<!entity p-version "3.0.17">
+<!entity p-status "stable">
<!entity % p-authors-formal "INCLUDE"> <!-- include additional text, etc -->
-<!entity % p-not-stable "INCLUDE">
-<!entity % p-stable "IGNORE">
+<!entity % p-not-stable "IGNORE">
+<!entity % p-stable "INCLUDE">
<!entity % p-text "IGNORE"> <!-- define we are not a text only doc -->
<!entity % p-doc "INCLUDE"> <!-- and we are a formal doc -->
<!entity % p-readme "IGNORE">
This file belongs into
ijbswa.sourceforge.net:/home/groups/i/ij/ijbswa/htdocs/
- $Id: user-manual.sgml,v 2.109 2009/07/18 16:24:39 fabiankeil Exp $
+ $Id: user-manual.sgml,v 2.127 2010/11/10 21:48:54 fabiankeil Exp $
- Copyright (C) 2001-2009 Privoxy Developers http://www.privoxy.org/
+ Copyright (C) 2001-2010 Privoxy Developers http://www.privoxy.org/
See LICENSE.
========================================================================
<subscript>
<!-- Completely the wrong markup, but very little is allowed -->
<!-- in this part of an article. FIXME -->
- <link linkend="copyright">Copyright</link> &my-copy; 2001-2009 by
+ <link linkend="copyright">Copyright</link> &my-copy; 2001-2010 by
<ulink url="http://www.privoxy.org/">Privoxy Developers</ulink>
</subscript>
</pubdate>
-<pubdate>$Id: user-manual.sgml,v 2.109 2009/07/18 16:24:39 fabiankeil Exp $</pubdate>
+<pubdate>$Id: user-manual.sgml,v 2.127 2010/11/10 21:48:54 fabiankeil Exp $</pubdate>
<!--
time being is still the comments in the source files and in the individual
configuration files. Development of a new version is currently nearing
completion, and includes significant changes and enhancements over
- earlier versions.]]>.
+ earlier versions]]>.
</para>
<!-- include only in non-stable versions -->
<sect1 id="whatsnew">
<title>What's New in this Release</title>
<para>
- <application>Privoxy 3.0.14 Beta</application> is a bugfix-release
- for the previous beta which introduced IPv6 support, improved keep-alive
- support and a bunch of minor improvements. The changes since 3.0.12:
+ <application>Privoxy 3.0.17</application> is a stable release.
+ The changes since 3.0.16 stable are:
</para>
<para>
<itemizedlist>
- <listitem>
- <para>
- Added IPv6 support. Thanks to Petr Pisar who not only provided
- the initial patch but also helped a lot with the integration.
- </para>
- </listitem>
- <listitem>
- <para>
- Added client-side keep-alive support. This should also allow
- NTLM authentication through Privoxy, but this hasn't been
- confirmed yet.
- </para>
- </listitem>
- <listitem>
- <para>
- The connection sharing code is only used if the connection-sharing
- option is enabled.
- </para>
- </listitem>
- <listitem>
- <para>
- The latency is taken into account when evaluating whether or not to
- reuse a connection. This should significantly reduce the number of
- connections problems several users reported.
- </para>
- </listitem>
- <listitem>
- <para>
- The max-client-connections option has been added to restrict
- the number of client connections below a value enforced by
- the operating system.
- </para>
- </listitem>
- <listitem>
- <para>
- If the server doesn't specify how long the connection stays alive,
- Privoxy errs on the safe side of caution and assumes it's only a second.
- </para>
- </listitem>
- <listitem>
- <para>
- Setting keep-alive-timeout to 0 disables keep-alive support. Previously
- Privoxy would claim to allow persistence but not reuse the connection.
- </para>
- </listitem>
- <listitem>
- <para>
- Pipelined requests are less likely to be mistaken for the request
- body of the previous request. Note that Privoxy still has no real
- pipeline support and will either serialize pipelined requests or
- drop them in which case the client has to resent them.
- </para>
- </listitem>
- <listitem>
- <para>
- Fixed a crash on some Windows versions when header randomization
- is enabled and the date couldn't be parsed.
- </para>
- </listitem>
- <listitem>
- <para>
- Privoxy's keep-alive timeout for the current connection is reduced
- to the one specified in the client's Keep-Alive header.
- </para>
- </listitem>
- <listitem>
+ <listitem>
<para>
- For HTTP/1.1 requests, Privoxy implies keep-alive support by not
- setting any Connection header instead of using 'Connection: keep-alive'.
+ Fixed last-chunk-detection for responses where the content was small
+ enough to be read with the body, causing Privoxy to wait for the
+ end of the content until the server closed the connection.
+ Reported by "Karsten" in #3028326.
</para>
</listitem>
<listitem>
<para>
- If the socket isn't reusable, Privoxy doesn't temporarily waste
- a socket slot to remember the connection.
+ Responses with status code 204 weren't properly detected as body-less
+ like RFC2616 mandates. Like the previous bug, this caused Privoxy
+ to wait for the end of the content until the server closed the connection.
+ Fixes #3022042 and #3025553, reported by a user with no visible name.
+ Most likely also fixes a bunch of other AJAX-related problem reports
+ that got closed in the past due to insufficient information and lack
+ of feedback.
</para>
</listitem>
<listitem>
<para>
- If keep-alive support is disabled but compiled in, the client's
- Keep-Alive header is removed.
+ Fixed an ACL bug that made it impossible to build a blacklist.
+ Usually the ACL directives are used in a whitelist, which worked
+ as expected, but blacklisting is still useful for public proxies
+ where one only needs to deny known abusers access.
</para>
</listitem>
<listitem>
<para>
- Fixed a bug on mingw32 where downloading large files failed if
- keep-alive support was enabled.
+ Added LOG_LEVEL_RECEIVED to log the not-yet-parsed data read from the
+ network. This should make debugging various parsing issues a lot easier.
</para>
</listitem>
<listitem>
<para>
- Fixed a bug that (at least theoretically) could cause log
- timestamps to be occasionally off by about a second.
+ The IPv6 code is enabled by default on Windows versions that support it.
+ Patch submitted by oCameLo in #2942729.
</para>
</listitem>
<listitem>
<para>
- The configure script respects the $PATH variable when searching
- for groups and id.
+ In mingw32 versions, the user.filter file is reachable through the
+ GUI, just like default.filter is. Feature request 3040263.
</para>
</listitem>
<listitem>
<para>
- Compressed content with extra fields couldn't be decompressed
- and would get passed to the client unfiltered. This problem
- has only be detected through statical analysis with clang as
- nobody seems to be using extra fields anyway.
+ Added the configure option --enable-large-file-support to set a few
+ defines that are required by platforms like GNU/Linux to support files
+ larger then 2GB. Mainly interesting for users without proper logfile
+ management.
</para>
</listitem>
<listitem>
<para>
- If the server resets the Connection after sending only the headers
- Privoxy forwards what it got to the client. Previously Privoxy
- would deliver an error message instead.
+ Logging with "debug 16" no longer stops at the first nul byte which is
+ pretty useless. Non-printable characters are replaced with their hex value
+ so the result can't span multiple lines making parsing them harder then
+ necessary.
</para>
</listitem>
<listitem>
<para>
- Error messages in case of connection timeouts use the right
- HTTP status code.
+ Privoxy logs when reading an action, filter or trust file.
</para>
</listitem>
<listitem>
<para>
- If spawning a child to handle a request fails, the client
- gets an error message and Privoxy continues to listen for
- new requests right away.
+ Fixed incorrect regression test markup which caused a test in
+ 3.0.16 to fail while Privoxy itself was working correctly.
+ While Privoxy accepts hide-referer, too, the action name is actually
+ hide-referrer which is also the name used one the final results page,
+ where the test expected the alias.
</para>
</listitem>
<listitem>
<para>
- The error messages in case of server-connection timeouts or
- prematurely closed server connections are now template-based.
+ CGI interface improvements:
+ <itemizedlist>
+ <listitem>
+ <para>
+ In finish_http_response(), continue to add the 'Connection: close'
+ header if the client connection will not be kept alive.
+ Anonymously pointed out in #2987454.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Apostrophes in block messages no longer cause parse errors
+ when the blocked page is viewed with JavaScript enabled.
+ Reported by dg1727 in #3062296.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Fix a bunch of anchors that used underscores instead of dashes.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Allow to keep the client connection alive after crunching the previous request.
+ Already opened server connections can be kept alive, too.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ In cgi_show_url_info(), don't forget to prefix URLs that only contain http:// or https:// in the path.
+ Fixes #2975765 reported by Adam Piggott.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Show the 404 CGI page if cgi_send_user_manual() is called while
+ local user manual delivery is disabled.
+ </para>
+ </itemizedlist>
</para>
</listitem>
<listitem>
<para>
- If zlib support isn't compiled in, Privoxy no longer tries to
- filter compressed content unless explicitly asked to do so.
+ Action file improvements:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Enable user.filter by default. Suggested by David White in #3001830.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Block .sitestat.com/. Reported by johnd16 in #3002725.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Block .atemda.com/. Reported by johnd16 in #3002723.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Block js.adlink.net/. Reported by johnd16 in #3002720.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Block .analytics.yahoo.com/. Reported by johnd16 in #3002713.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Block sb.scorecardresearch.com, too. Reported by dg1727 in #2992652.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Fix problems noticed on Yahoo mail and news pages.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Remove the too broad yahoo section, only keeping the
+ fast-redirects exception as discussed on ijbswa-devel@.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Don't block adesklets.sourceforge.net. Reported in #2974204.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Block chartbeat ping tracking. Reported in #2975895.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Tag CSS and image requests with cautious and medium settings, too.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Don't handle view.atdmt.com as image. It's used for click-throughs
+ so users should be able to "go there anyway".
+ Reported by Adam Piggott in #2975927.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Also let the refresh-tags filter remove invalid refresh tags where
+ the 'url=' part is missing. Anonymously reported in #2986382.
+ While at it, update the description to mention the fact that only
+ refresh tags with refresh times above 9 seconds are covered.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ javascript needs to be blocked with +handle-as-empty-document to
+ work around Firefox bug 492459. So move .js blockers from
+ +block{Might be a web-bug.} -handle-as-empty-document
+ to
+ +block{Might be a web-bug.} +handle-as-empty-document
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ ijbswa-Feature Requests-3006719 - Block 160x578 Banners.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Block another omniture tracking domain
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Added a range-requests tagger.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Added two sections to get Flickr's Ajax interface working with
+ default pre-settings. If you change the configuration to block
+ cookies by default, you'll need additional exceptions.
+ Reported by Mathias Homann in #3101419 and by Patrick on ijbswa-users@
+ </para>
+ </itemizedlist>
</para>
</listitem>
<listitem>
<para>
- In case of connections that are denied based on ACL directives,
- the memory used for the client IP is no longer leaked.
+ Documentation improvements:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Explicitly mention how to match all URLs.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Consistently recommend socks5 in the Tor FAQ entry and mention
+ its advantage compared to socks4a. Reported by David in #2960129.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Slightly improve the explanation of why filtering may appear
+ slower than it is.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Grammar fixes for the ACL section.
+ </para>
+ </itemizedlist>
</para>
</listitem>
<listitem>
<para>
- Fixed another small memory leak if the client request times out
- while waiting for client headers other than the request line.
+ Privoxy-Log-Parser improvements:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Also gather statistics for blocked and redirected requests.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Provide the percentage of keep-alive offers the client accepted.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Add a --url-statistics-threshold option.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Also gather statistics for ressources, methods, and HTTP versions
+ used by the client.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Add a --host-statistics-threshold option to also gather
+ statistics about how many request where made per host.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Fix a bug in handle_loglevel_header() where a 'scan: ' got lost.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Add a --shorten-thread-ids option to replace the thread id with
+ a decimal number.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Accept and ignore: Looks like we got the last chunk together
+ with the server headers. We better stop reading.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Accept and ignore: Continue hack in da house.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Accept and higlight: Rejecting connection from 10.0.0.2.
+ Maximum number of connections reached.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Accept and highlight: Loading actions file: /usr/local/etc/privoxy/default.action
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Accept and highlight: Loading filter file: /usr/local/etc/privoxy/default.filter
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Accept and highlight: Killed all-caps Host header line: HOST: bestproxydb.com
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Accept and highlight: Reducing expected bytes to 0. Marking
+ the server socket tainted after throwing 4 bytes away.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Accept: Merged multiple header lines to: 'X-FORWARDED-PROTO: http X-HOST: 127.0.0.1'
+ </para>
+ </itemizedlist>
</para>
</listitem>
<listitem>
<para>
- The client socket is kept open until the server socket has
- been marked as unused. This should increase the chances that
- the still-open connection will be reused for the client's next
- request to the same destination. Note that this only matters
- if connection-sharing is enabled.
+ Code cleanups:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Remove the next member from the client_state struct. Only the main
+ thread needs access to all client states so give it its own struct.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Garbage-collect request_contains_null_bytes().
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Ditch redundant code in unload_configfile().
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Ditch LogGetURLUnderCursor() which doesn't seem to be used anywhere.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ In write_socket(), remove the write-only variable write_len in
+ an ifdef __OS2__ block. Spotted by cppcheck.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ In connect_to(), don't declare the variable 'flags' on OS/2 where
+ it isn't used. Spotted by cppcheck.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Limit the scope of various variables. Spotted by cppcheck.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ In add_to_iob(), turn an interestingly looking for loop into a
+ boring while loop.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Code cleanup in preparation for external filters.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ In listen_loop(), mention the socket on which we accepted the
+ connection, not just the source IP address.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ In write_socket(), also log the socket we're writing to.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ In log_error(), assert that escaped characters get logged
+ completely or not at all.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ In log_error(), assert that ival and sval have reasonable values.
+ There's no reason not to abort() if they don't.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Remove an incorrect cgi_error_unknown() call in a
+ cannnot-happen-situation in send_crunch_response().
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Clean up white-space in http_response definition and
+ move the crunch_reason to the beginning.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Turn http_response.reason into an enum and rename it
+ to http_response.crunch_reason.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Silence a 'gcc (Debian 4.3.2-1.1) 4.3.2' warning on i686 GNU/Linux.
+ </para>
+ </itemizedlist>
</para>
</listitem>
<listitem>
<para>
- A TODO list has been added to the source tarballs to give potential
- volunteers a better idea of what the current goals are. Donations
- are still welcome too: http://www.privoxy.org/faq/general.html#DONATE
+ GNUmakefile improvements:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Use $(SSH) instead of ssh, so one only needs to specify a username once.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Removed references to the action feedback thingy that hasn't been
+ working for years.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Consistently use shell.sourceforge.net instead of shell.sf.net so
+ one doesn't need to check server fingerprints twice.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Removed GNUisms in the webserver and webactions targets so they
+ work with standard tar.
+ </para>
+ </itemizedlist>
</para>
</listitem>
</itemizedlist>
</para>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>/</literal></term>
+ <listitem>
+ <para>
+ Matches any URL because there's no requirement for either the
+ domain or the path to match anything.
+ </para>
+ </listitem>
+ </varlistentry>
<varlistentry>
<term><literal>:8000/</literal></term>
<listitem>
<quote>HTTP headers</quote> are, you definitely don't need to worry about this
one.
</para>
+ <para>
+ Headers added by this action are not modified by other actions.
+ </para>
</listitem>
</varlistentry>
<para>
Filtering requires buffering the page content, which may appear to
slow down page rendering since nothing is displayed until all content has
- passed the filters. (It does not really take longer, but seems that way
- since the page is not incrementally displayed.) This effect will be more
- noticeable on slower connections.
+ passed the filters. (The total time until the page is completely rendered
+ doesn't change much, but it may be perceived as slower since the page is
+ not incrementally displayed.)
+ This effect will be more noticeable on slower connections.
</para>
<para>
<quote>Rolling your own</quote>
USA
$Log: user-manual.sgml,v $
+ Revision 2.127 2010/11/10 21:48:54 fabiankeil
+ Update the "What's New" section.
+
+ Revision 2.126 2010/11/06 12:55:48 fabiankeil
+ Set p-version to 3.0.17
+
+ Revision 2.125 2010/09/03 17:39:37 fabiankeil
+ Slightly improve the explanation of why filtering may appear slower than it is.
+
+ Revision 2.124 2010/05/01 18:21:30 fabiankeil
+ Explicitly mention how to match any URL.
+
+ Revision 2.123 2010/02/19 16:00:38 fabiankeil
+ Even more fixes.
+
+ Revision 2.122 2010/02/19 15:22:47 fabiankeil
+ Add missing word.
+
+ Revision 2.121 2010/02/15 15:30:13 fabiankeil
+ Mention the use of the no-such-domain template for DNS problems with FEATURE_IPV6_SUPPORT enabled.
+
+ Revision 2.120 2010/02/13 17:38:39 fabiankeil
+ Update entities for 3.0.16 stable.
+
+ Revision 2.119 2010/02/13 16:37:37 fabiankeil
+ Update 'What's new?' section.
+
+ Revision 2.118 2010/02/11 13:59:48 fabiankeil
+ Mention that the headers added by the add-header action aren't modified by other actions.
+
+ Revision 2.117 2010/01/11 12:56:04 fabiankeil
+ Bump copyright range as p-config.sgml's copyright line is only used in the config file.
+
+ Revision 2.116 2009/11/15 14:24:12 fabiankeil
+ Prepare to generate docs for 3.0.16 UNRELEASED.
+
+ Revision 2.115 2009/10/10 06:19:34 fabiankeil
+ Ditch a duplicated 'since'.
+
+ Revision 2.114 2009/10/10 05:51:48 fabiankeil
+ Update "What's new" section.
+
+ Revision 2.113 2009/10/10 05:48:55 fabiankeil
+ Prepare for 3.0.15 beta.
+
+ Revision 2.112 2009/07/24 12:20:30 fabiankeil
+ Remove duplicated period.
+
+ Revision 2.111 2009/07/18 18:11:11 fabiankeil
+ Don't claim that NTLM should work when there are multiple reports that it doesn't.
+
+ Revision 2.110 2009/07/18 16:25:17 fabiankeil
+ Fix trailing whitespace.
+
Revision 2.109 2009/07/18 16:24:39 fabiankeil
Bump entities for 3.0.14 beta.