<!entity license SYSTEM "license.sgml">
<!entity p-authors SYSTEM "p-authors.sgml">
<!entity config SYSTEM "p-config.sgml">
-<!entity p-version "3.0.14">
-<!entity p-status "BETA">
+<!entity p-version "3.0.17">
+<!entity p-status "stable">
<!entity % p-authors-formal "INCLUDE"> <!-- include additional text, etc -->
-<!entity % p-not-stable "INCLUDE">
-<!entity % p-stable "IGNORE">
+<!entity % p-not-stable "IGNORE">
+<!entity % p-stable "INCLUDE">
<!entity % p-text "IGNORE"> <!-- define we are not a text only doc -->
<!entity % p-doc "INCLUDE"> <!-- and we are a formal doc -->
<!entity % p-readme "IGNORE">
This file belongs into
ijbswa.sourceforge.net:/home/groups/i/ij/ijbswa/htdocs/
- $Id: user-manual.sgml,v 2.108 2009/07/18 15:49:23 fabiankeil Exp $
+ $Id: user-manual.sgml,v 2.127 2010/11/10 21:48:54 fabiankeil Exp $
- Copyright (C) 2001-2009 Privoxy Developers http://www.privoxy.org/
+ Copyright (C) 2001-2010 Privoxy Developers http://www.privoxy.org/
See LICENSE.
========================================================================
<subscript>
<!-- Completely the wrong markup, but very little is allowed -->
<!-- in this part of an article. FIXME -->
- <link linkend="copyright">Copyright</link> &my-copy; 2001-2009 by
+ <link linkend="copyright">Copyright</link> &my-copy; 2001-2010 by
<ulink url="http://www.privoxy.org/">Privoxy Developers</ulink>
</subscript>
</pubdate>
-<pubdate>$Id: user-manual.sgml,v 2.108 2009/07/18 15:49:23 fabiankeil Exp $</pubdate>
+<pubdate>$Id: user-manual.sgml,v 2.127 2010/11/10 21:48:54 fabiankeil Exp $</pubdate>
<!--
<sect1 label="1" id="introduction"><title>Introduction</title>
<para>
This documentation is included with the current &p-status; version of
- <application>Privoxy</application>, v.&p-version;<![%p-not-stable;[,
+ <application>Privoxy</application>, v.&p-version;<![%p-not-stable;[,
and is mostly complete at this point. The most up to date reference for the
time being is still the comments in the source files and in the individual
configuration files. Development of a new version is currently nearing
completion, and includes significant changes and enhancements over
- earlier versions. ]]>.
+ earlier versions]]>.
</para>
<!-- include only in non-stable versions -->
<sect1 id="whatsnew">
<title>What's New in this Release</title>
<para>
- <application>Privoxy 3.0.14 Beta</application> is a bugfix-release
- for the previous beta which introduced IPv6 support, improved keep-alive
- support and a bunch of minor improvements. The changes since 3.0.12:
+ <application>Privoxy 3.0.17</application> is a stable release.
+ The changes since 3.0.16 stable are:
</para>
<para>
<itemizedlist>
- <listitem>
- <para>
- Added IPv6 support. Thanks to Petr Pisar who not only provided
- the initial patch but also helped a lot with the integration.
- </para>
- </listitem>
- <listitem>
- <para>
- Added client-side keep-alive support. This should also allow
- NTLM authentication through Privoxy, but this hasn't been
- confirmed yet.
- </para>
- </listitem>
- <listitem>
- <para>
- The connection sharing code is only used if the connection-sharing
- option is enabled.
- </para>
- </listitem>
- <listitem>
- <para>
- The latency is taken into account when evaluating whether or not to
- reuse a connection. This should significantly reduce the number of
- connections problems several users reported.
- </para>
- </listitem>
- <listitem>
- <para>
- The max-client-connections option has been added to restrict
- the number of client connections below a value enforced by
- the operating system.
- </para>
- </listitem>
- <listitem>
- <para>
- If the server doesn't specify how long the connection stays alive,
- Privoxy errs on the safe side of caution and assumes it's only a second.
- </para>
- </listitem>
- <listitem>
- <para>
- Setting keep-alive-timeout to 0 disables keep-alive support. Previously
- Privoxy would claim to allow persistence but not reuse the connection.
- </para>
- </listitem>
- <listitem>
- <para>
- Pipelined requests are less likely to be mistaken for the request
- body of the previous request. Note that Privoxy still has no real
- pipeline support and will either serialize pipelined requests or
- drop them in which case the client has to resent them.
- </para>
- </listitem>
- <listitem>
- <para>
- Fixed a crash on some Windows versions when header randomization
- is enabled and the date couldn't be parsed.
- </para>
- </listitem>
- <listitem>
- <para>
- Privoxy's keep-alive timeout for the current connection is reduced
- to the one specified in the client's Keep-Alive header.
- </para>
- </listitem>
- <listitem>
+ <listitem>
<para>
- For HTTP/1.1 requests, Privoxy implies keep-alive support by not
- setting any Connection header instead of using 'Connection: keep-alive'.
+ Fixed last-chunk-detection for responses where the content was small
+ enough to be read with the body, causing Privoxy to wait for the
+ end of the content until the server closed the connection.
+ Reported by "Karsten" in #3028326.
</para>
</listitem>
<listitem>
<para>
- If the socket isn't reusable, Privoxy doesn't temporarily waste
- a socket slot to remember the connection.
+ Responses with status code 204 weren't properly detected as body-less
+ like RFC2616 mandates. Like the previous bug, this caused Privoxy
+ to wait for the end of the content until the server closed the connection.
+ Fixes #3022042 and #3025553, reported by a user with no visible name.
+ Most likely also fixes a bunch of other AJAX-related problem reports
+ that got closed in the past due to insufficient information and lack
+ of feedback.
</para>
</listitem>
<listitem>
<para>
- If keep-alive support is disabled but compiled in, the client's
- Keep-Alive header is removed.
+ Fixed an ACL bug that made it impossible to build a blacklist.
+ Usually the ACL directives are used in a whitelist, which worked
+ as expected, but blacklisting is still useful for public proxies
+ where one only needs to deny known abusers access.
</para>
</listitem>
<listitem>
<para>
- Fixed a bug on mingw32 where downloading large files failed if
- keep-alive support was enabled.
+ Added LOG_LEVEL_RECEIVED to log the not-yet-parsed data read from the
+ network. This should make debugging various parsing issues a lot easier.
</para>
</listitem>
<listitem>
<para>
- Fixed a bug that (at least theoretically) could cause log
- timestamps to be occasionally off by about a second.
+ The IPv6 code is enabled by default on Windows versions that support it.
+ Patch submitted by oCameLo in #2942729.
</para>
</listitem>
<listitem>
<para>
- The configure script respects the $PATH variable when searching
- for groups and id.
+ In mingw32 versions, the user.filter file is reachable through the
+ GUI, just like default.filter is. Feature request 3040263.
</para>
</listitem>
<listitem>
<para>
- Compressed content with extra fields couldn't be decompressed
- and would get passed to the client unfiltered. This problem
- has only be detected through statical analysis with clang as
- nobody seems to be using extra fields anyway.
+ Added the configure option --enable-large-file-support to set a few
+ defines that are required by platforms like GNU/Linux to support files
+ larger then 2GB. Mainly interesting for users without proper logfile
+ management.
</para>
</listitem>
<listitem>
<para>
- If the server resets the Connection after sending only the headers
- Privoxy forwards what it got to the client. Previously Privoxy
- would deliver an error message instead.
+ Logging with "debug 16" no longer stops at the first nul byte which is
+ pretty useless. Non-printable characters are replaced with their hex value
+ so the result can't span multiple lines making parsing them harder then
+ necessary.
</para>
</listitem>
<listitem>
<para>
- Error messages in case of connection timeouts use the right
- HTTP status code.
+ Privoxy logs when reading an action, filter or trust file.
</para>
</listitem>
<listitem>
<para>
- If spawning a child to handle a request fails, the client
- gets an error message and Privoxy continues to listen for
- new requests right away.
+ Fixed incorrect regression test markup which caused a test in
+ 3.0.16 to fail while Privoxy itself was working correctly.
+ While Privoxy accepts hide-referer, too, the action name is actually
+ hide-referrer which is also the name used one the final results page,
+ where the test expected the alias.
</para>
</listitem>
<listitem>
<para>
- The error messages in case of server-connection timeouts or
- prematurely closed server connections are now template-based.
+ CGI interface improvements:
+ <itemizedlist>
+ <listitem>
+ <para>
+ In finish_http_response(), continue to add the 'Connection: close'
+ header if the client connection will not be kept alive.
+ Anonymously pointed out in #2987454.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Apostrophes in block messages no longer cause parse errors
+ when the blocked page is viewed with JavaScript enabled.
+ Reported by dg1727 in #3062296.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Fix a bunch of anchors that used underscores instead of dashes.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Allow to keep the client connection alive after crunching the previous request.
+ Already opened server connections can be kept alive, too.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ In cgi_show_url_info(), don't forget to prefix URLs that only contain http:// or https:// in the path.
+ Fixes #2975765 reported by Adam Piggott.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Show the 404 CGI page if cgi_send_user_manual() is called while
+ local user manual delivery is disabled.
+ </para>
+ </itemizedlist>
</para>
</listitem>
<listitem>
<para>
- If zlib support isn't compiled in, Privoxy no longer tries to
- filter compressed content unless explicitly asked to do so.
+ Action file improvements:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Enable user.filter by default. Suggested by David White in #3001830.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Block .sitestat.com/. Reported by johnd16 in #3002725.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Block .atemda.com/. Reported by johnd16 in #3002723.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Block js.adlink.net/. Reported by johnd16 in #3002720.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Block .analytics.yahoo.com/. Reported by johnd16 in #3002713.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Block sb.scorecardresearch.com, too. Reported by dg1727 in #2992652.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Fix problems noticed on Yahoo mail and news pages.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Remove the too broad yahoo section, only keeping the
+ fast-redirects exception as discussed on ijbswa-devel@.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Don't block adesklets.sourceforge.net. Reported in #2974204.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Block chartbeat ping tracking. Reported in #2975895.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Tag CSS and image requests with cautious and medium settings, too.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Don't handle view.atdmt.com as image. It's used for click-throughs
+ so users should be able to "go there anyway".
+ Reported by Adam Piggott in #2975927.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Also let the refresh-tags filter remove invalid refresh tags where
+ the 'url=' part is missing. Anonymously reported in #2986382.
+ While at it, update the description to mention the fact that only
+ refresh tags with refresh times above 9 seconds are covered.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ javascript needs to be blocked with +handle-as-empty-document to
+ work around Firefox bug 492459. So move .js blockers from
+ +block{Might be a web-bug.} -handle-as-empty-document
+ to
+ +block{Might be a web-bug.} +handle-as-empty-document
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ ijbswa-Feature Requests-3006719 - Block 160x578 Banners.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Block another omniture tracking domain
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Added a range-requests tagger.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Added two sections to get Flickr's Ajax interface working with
+ default pre-settings. If you change the configuration to block
+ cookies by default, you'll need additional exceptions.
+ Reported by Mathias Homann in #3101419 and by Patrick on ijbswa-users@
+ </para>
+ </itemizedlist>
</para>
</listitem>
<listitem>
<para>
- In case of connections that are denied based on ACL directives,
- the memory used for the client IP is no longer leaked.
+ Documentation improvements:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Explicitly mention how to match all URLs.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Consistently recommend socks5 in the Tor FAQ entry and mention
+ its advantage compared to socks4a. Reported by David in #2960129.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Slightly improve the explanation of why filtering may appear
+ slower than it is.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Grammar fixes for the ACL section.
+ </para>
+ </itemizedlist>
</para>
</listitem>
<listitem>
<para>
- Fixed another small memory leak if the client request times out
- while waiting for client headers other than the request line.
+ Privoxy-Log-Parser improvements:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Also gather statistics for blocked and redirected requests.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Provide the percentage of keep-alive offers the client accepted.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Add a --url-statistics-threshold option.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Also gather statistics for ressources, methods, and HTTP versions
+ used by the client.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Add a --host-statistics-threshold option to also gather
+ statistics about how many request where made per host.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Fix a bug in handle_loglevel_header() where a 'scan: ' got lost.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Add a --shorten-thread-ids option to replace the thread id with
+ a decimal number.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Accept and ignore: Looks like we got the last chunk together
+ with the server headers. We better stop reading.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Accept and ignore: Continue hack in da house.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Accept and higlight: Rejecting connection from 10.0.0.2.
+ Maximum number of connections reached.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Accept and highlight: Loading actions file: /usr/local/etc/privoxy/default.action
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Accept and highlight: Loading filter file: /usr/local/etc/privoxy/default.filter
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Accept and highlight: Killed all-caps Host header line: HOST: bestproxydb.com
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Accept and highlight: Reducing expected bytes to 0. Marking
+ the server socket tainted after throwing 4 bytes away.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Accept: Merged multiple header lines to: 'X-FORWARDED-PROTO: http X-HOST: 127.0.0.1'
+ </para>
+ </itemizedlist>
</para>
</listitem>
<listitem>
<para>
- The client socket is kept open until the server socket has
- been marked as unused. This should increase the chances that
- the still-open connection will be reused for the client's next
- request to the same destination. Note that this only matters
- if connection-sharing is enabled.
+ Code cleanups:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Remove the next member from the client_state struct. Only the main
+ thread needs access to all client states so give it its own struct.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Garbage-collect request_contains_null_bytes().
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Ditch redundant code in unload_configfile().
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Ditch LogGetURLUnderCursor() which doesn't seem to be used anywhere.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ In write_socket(), remove the write-only variable write_len in
+ an ifdef __OS2__ block. Spotted by cppcheck.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ In connect_to(), don't declare the variable 'flags' on OS/2 where
+ it isn't used. Spotted by cppcheck.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Limit the scope of various variables. Spotted by cppcheck.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ In add_to_iob(), turn an interestingly looking for loop into a
+ boring while loop.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Code cleanup in preparation for external filters.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ In listen_loop(), mention the socket on which we accepted the
+ connection, not just the source IP address.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ In write_socket(), also log the socket we're writing to.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ In log_error(), assert that escaped characters get logged
+ completely or not at all.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ In log_error(), assert that ival and sval have reasonable values.
+ There's no reason not to abort() if they don't.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Remove an incorrect cgi_error_unknown() call in a
+ cannnot-happen-situation in send_crunch_response().
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Clean up white-space in http_response definition and
+ move the crunch_reason to the beginning.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Turn http_response.reason into an enum and rename it
+ to http_response.crunch_reason.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Silence a 'gcc (Debian 4.3.2-1.1) 4.3.2' warning on i686 GNU/Linux.
+ </para>
+ </itemizedlist>
</para>
</listitem>
<listitem>
<para>
- A TODO list has been added to the source tarballs to give potential
- volunteers a better idea of what the current goals are. Donations
- are still welcome too: http://www.privoxy.org/faq/general.html#DONATE
+ GNUmakefile improvements:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Use $(SSH) instead of ssh, so one only needs to specify a username once.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Removed references to the action feedback thingy that hasn't been
+ working for years.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Consistently use shell.sourceforge.net instead of shell.sf.net so
+ one doesn't need to check server fingerprints twice.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Removed GNUisms in the webserver and webactions targets so they
+ work with standard tar.
+ </para>
+ </itemizedlist>
</para>
</listitem>
</itemizedlist>
</para>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>/</literal></term>
+ <listitem>
+ <para>
+ Matches any URL because there's no requirement for either the
+ domain or the path to match anything.
+ </para>
+ </listitem>
+ </varlistentry>
<varlistentry>
<term><literal>:8000/</literal></term>
<listitem>
<quote>HTTP headers</quote> are, you definitely don't need to worry about this
one.
</para>
+ <para>
+ Headers added by this action are not modified by other actions.
+ </para>
</listitem>
</varlistentry>
<para>
Filtering requires buffering the page content, which may appear to
slow down page rendering since nothing is displayed until all content has
- passed the filters. (It does not really take longer, but seems that way
- since the page is not incrementally displayed.) This effect will be more
- noticeable on slower connections.
+ passed the filters. (The total time until the page is completely rendered
+ doesn't change much, but it may be perceived as slower since the page is
+ not incrementally displayed.)
+ This effect will be more noticeable on slower connections.
</para>
<para>
<quote>Rolling your own</quote>
USA
$Log: user-manual.sgml,v $
+ Revision 2.127 2010/11/10 21:48:54 fabiankeil
+ Update the "What's New" section.
+
+ Revision 2.126 2010/11/06 12:55:48 fabiankeil
+ Set p-version to 3.0.17
+
+ Revision 2.125 2010/09/03 17:39:37 fabiankeil
+ Slightly improve the explanation of why filtering may appear slower than it is.
+
+ Revision 2.124 2010/05/01 18:21:30 fabiankeil
+ Explicitly mention how to match any URL.
+
+ Revision 2.123 2010/02/19 16:00:38 fabiankeil
+ Even more fixes.
+
+ Revision 2.122 2010/02/19 15:22:47 fabiankeil
+ Add missing word.
+
+ Revision 2.121 2010/02/15 15:30:13 fabiankeil
+ Mention the use of the no-such-domain template for DNS problems with FEATURE_IPV6_SUPPORT enabled.
+
+ Revision 2.120 2010/02/13 17:38:39 fabiankeil
+ Update entities for 3.0.16 stable.
+
+ Revision 2.119 2010/02/13 16:37:37 fabiankeil
+ Update 'What's new?' section.
+
+ Revision 2.118 2010/02/11 13:59:48 fabiankeil
+ Mention that the headers added by the add-header action aren't modified by other actions.
+
+ Revision 2.117 2010/01/11 12:56:04 fabiankeil
+ Bump copyright range as p-config.sgml's copyright line is only used in the config file.
+
+ Revision 2.116 2009/11/15 14:24:12 fabiankeil
+ Prepare to generate docs for 3.0.16 UNRELEASED.
+
+ Revision 2.115 2009/10/10 06:19:34 fabiankeil
+ Ditch a duplicated 'since'.
+
+ Revision 2.114 2009/10/10 05:51:48 fabiankeil
+ Update "What's new" section.
+
+ Revision 2.113 2009/10/10 05:48:55 fabiankeil
+ Prepare for 3.0.15 beta.
+
+ Revision 2.112 2009/07/24 12:20:30 fabiankeil
+ Remove duplicated period.
+
+ Revision 2.111 2009/07/18 18:11:11 fabiankeil
+ Don't claim that NTLM should work when there are multiple reports that it doesn't.
+
+ Revision 2.110 2009/07/18 16:25:17 fabiankeil
+ Fix trailing whitespace.
+
+ Revision 2.109 2009/07/18 16:24:39 fabiankeil
+ Bump entities for 3.0.14 beta.
+
Revision 2.108 2009/07/18 15:49:23 fabiankeil
Add most of the changes in 3.0.14 to the "What's New" section.
projects / privoxy.git / blobdiff