<!entity license SYSTEM "license.sgml">
<!entity p-authors SYSTEM "p-authors.sgml">
<!entity config SYSTEM "p-config.sgml">
-<!entity p-version "3.0.11">
+<!entity p-version "3.0.17">
<!entity p-status "stable">
<!entity % p-authors-formal "INCLUDE"> <!-- include additional text, etc -->
<!entity % p-not-stable "IGNORE">
This file belongs into
ijbswa.sourceforge.net:/home/groups/i/ij/ijbswa/htdocs/
- $Id: user-manual.sgml,v 2.103 2009/03/21 10:49:05 fabiankeil Exp $
+ $Id: user-manual.sgml,v 2.130 2010/12/01 19:28:28 fabiankeil Exp $
- Copyright (C) 2001-2009 Privoxy Developers http://www.privoxy.org/
+ Copyright (C) 2001-2010 Privoxy Developers http://www.privoxy.org/
See LICENSE.
========================================================================
<subscript>
<!-- Completely the wrong markup, but very little is allowed -->
<!-- in this part of an article. FIXME -->
- <link linkend="copyright">Copyright</link> &my-copy; 2001-2009 by
+ <link linkend="copyright">Copyright</link> &my-copy; 2001-2010 by
<ulink url="http://www.privoxy.org/">Privoxy Developers</ulink>
</subscript>
</pubdate>
-<pubdate>$Id: user-manual.sgml,v 2.103 2009/03/21 10:49:05 fabiankeil Exp $</pubdate>
+<pubdate>$Id: user-manual.sgml,v 2.130 2010/12/01 19:28:28 fabiankeil Exp $</pubdate>
<!--
<sect1 label="1" id="introduction"><title>Introduction</title>
<para>
This documentation is included with the current &p-status; version of
- <application>Privoxy</application>, v.&p-version;<![%p-not-stable;[,
+ <application>Privoxy</application>, v.&p-version;<![%p-not-stable;[,
and is mostly complete at this point. The most up to date reference for the
time being is still the comments in the source files and in the individual
configuration files. Development of a new version is currently nearing
completion, and includes significant changes and enhancements over
- earlier versions. ]]>.
+ earlier versions]]>.
</para>
<!-- include only in non-stable versions -->
<sect1 id="whatsnew">
<title>What's New in this Release</title>
<para>
- <application>Privoxy 3.0.12</application> is mainly a bugfix release:
+ <application>Privoxy 3.0.17</application> is a stable release.
+ The changes since 3.0.16 stable are:
</para>
<para>
<itemizedlist>
<listitem>
<para>
- The socket-timeout option now also works on platforms whose
- select() implementation modifies the timeout structure.
- Previously the timeout was triggered even if the connection
- didn't stall. Reported by cyberpatrol.
+ Fixed last-chunk-detection for responses where the content was small
+ enough to be read with the body, causing Privoxy to wait for the
+ end of the content until the server closed the connection or the
+ request timed out. Reported by "Karsten" in #3028326.
</para>
</listitem>
<listitem>
<para>
- The Connection: keep-alive code properly deals with files
- larger than 2GB. Previously the connection was closed too
- early.
+ Responses with status code 204 weren't properly detected as body-less
+ like RFC2616 mandates. Like the previous bug, this caused Privoxy to
+ wait for the end of the content until the server closed the connection
+ or the request timed out. Fixes #3022042 and #3025553, reported by a
+ user with no visible name. Most likely also fixes a bunch of other
+ AJAX-related problem reports that got closed in the past due to
+ insufficient information and lack of feedback.
</para>
</listitem>
<listitem>
<para>
- The content length for files above 2GB is logged correctly.
+ Fixed an ACL bug that made it impossible to build a blacklist.
+ Usually the ACL directives are used in a whitelist, which worked
+ as expected, but blacklisting is still useful for public proxies
+ where one only needs to deny known abusers access.
</para>
</listitem>
<listitem>
<para>
- The user-manual directive on the show-status page links to
- the documentation location specified with the directive,
- not to the Privoxy website.
+ Added LOG_LEVEL_RECEIVED to log the not-yet-parsed data read from the
+ network. This should make debugging various parsing issues a lot easier.
</para>
</listitem>
<listitem>
<para>
- When running in daemon mode, Privoxy doesn't log anything
- to the console unless there are errors before the logfile
- has been opened.
+ The IPv6 code is enabled by default on Windows versions that support it.
+ Patch submitted by oCameLo in #2942729.
</para>
</listitem>
<listitem>
<para>
- The show-status page prints warnings about invalid directives
- on the same line as the directives themselves.
+ In mingw32 versions, the user.filter file is reachable through the
+ GUI, just like default.filter is. Feature request 3040263.
</para>
</listitem>
<listitem>
<para>
- Fixed several justified (but harmless) compiler warnings,
- mostly on 64 bit platforms.
+ Added the configure option --enable-large-file-support to set a few
+ defines that are required by platforms like GNU/Linux to support files
+ larger then 2GB. Mainly interesting for users without proper logfile
+ management.
</para>
</listitem>
<listitem>
<para>
- The mingw32 version explicitly requests the default charset
- to prevent display problems with some fonts available on more
- recent Windows versions. Patch by Burberry.
+ Logging with "debug 16" no longer stops at the first nul byte which is
+ pretty useless. Non-printable characters are replaced with their hex value
+ so the result can't span multiple lines making parsing them harder then
+ necessary.
</para>
</listitem>
<listitem>
<para>
- The mingw32 version uses the Privoxy icon in the alt-tab
- windows. Patch by Burberry.
+ Privoxy logs when reading an action, filter or trust file.
</para>
</listitem>
<listitem>
<para>
- The timestamp and the thread id is omitted in the "Fatal error"
- message box on mingw32.
+ Fixed incorrect regression test markup which caused a test in
+ 3.0.16 to fail while Privoxy itself was working correctly.
+ While Privoxy accepts hide-referer, too, the action name is actually
+ hide-referrer which is also the name used one the final results page,
+ where the test expected the alias.
</para>
</listitem>
<listitem>
<para>
- Fixed two related mingw32-only buffer overflows. Triggering
- them required control over the configuration file, therefore
- this isn't seen as a security issue.
+ CGI interface improvements:
+ <itemizedlist>
+ <listitem>
+ <para>
+ In finish_http_response(), continue to add the 'Connection: close'
+ header if the client connection will not be kept alive.
+ Anonymously pointed out in #2987454.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Apostrophes in block messages no longer cause parse errors
+ when the blocked page is viewed with JavaScript enabled.
+ Reported by dg1727 in #3062296.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Fix a bunch of anchors that used underscores instead of dashes.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Allow to keep the client connection alive after crunching the previous request.
+ Already opened server connections can be kept alive, too.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ In cgi_show_url_info(), don't forget to prefix URLs that only contain
+ http:// or https:// in the path. Fixes #2975765 reported by Adam Piggott.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Show the 404 CGI page if cgi_send_user_manual() is called while
+ local user manual delivery is disabled.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Action file improvements:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Enable user.filter by default. Suggested by David White in #3001830.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Block .sitestat.com/. Reported by johnd16 in #3002725.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Block .atemda.com/. Reported by johnd16 in #3002723.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Block js.adlink.net/. Reported by johnd16 in #3002720.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Block .analytics.yahoo.com/. Reported by johnd16 in #3002713.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Block sb.scorecardresearch.com, too. Reported by dg1727 in #2992652.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Fix problems noticed on Yahoo mail and news pages.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Remove the too broad yahoo section, only keeping the
+ fast-redirects exception as discussed on ijbswa-devel@.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Don't block adesklets.sourceforge.net. Reported in #2974204.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Block chartbeat ping tracking. Reported in #2975895.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Tag CSS and image requests with cautious and medium settings, too.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Don't handle view.atdmt.com as image. It's used for click-throughs
+ so users should be able to "go there anyway".
+ Reported by Adam Piggott in #2975927.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Also let the refresh-tags filter remove invalid refresh tags where
+ the 'url=' part is missing. Anonymously reported in #2986382.
+ While at it, update the description to mention the fact that only
+ refresh tags with refresh times above 9 seconds are covered.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ javascript needs to be blocked with +handle-as-empty-document to
+ work around Firefox bug 492459. So move .js blockers from
+ +block{Might be a web-bug.} -handle-as-empty-document to
+ +block{Might be a web-bug.} +handle-as-empty-document.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ ijbswa-Feature Requests-3006719 - Block 160x578 Banners.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Block another omniture tracking domain.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Added a range-requests tagger.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Added two sections to get Flickr's Ajax interface working with
+ default pre-settings. If you change the configuration to block
+ cookies by default, you'll need additional exceptions.
+ Reported by Mathias Homann in #3101419 and by Patrick on ijbswa-users@.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Documentation improvements:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Explicitly mention how to match all URLs.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Consistently recommend socks5 in the Tor FAQ entry and mention
+ its advantage compared to socks4a. Reported by David in #2960129.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Slightly improve the explanation of why filtering may appear
+ slower than it is.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Grammar fixes for the ACL section.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Fixed a link to the 'intercepting' entry and add another one.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Rename the 'Other' section to 'Mailing Lists' and reword it
+ to make it clear that nobody is forced to use the trackers
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Note that 'anonymously' posting on the trackers may not always
+ be possible.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Suggest to enable debug 32768 when suspecting parsing problems.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Privoxy-Log-Parser improvements:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Gather statistics for ressources, methods, and HTTP versions
+ used by the client.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Also gather statistics for blocked and redirected requests.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Provide the percentage of keep-alive offers the client accepted.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Add a --url-statistics-threshold option.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Add a --host-statistics-threshold option to also gather
+ statistics about how many request where made per host.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Fix a bug in handle_loglevel_header() where a 'scan: ' got lost.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Add a --shorten-thread-ids option to replace the thread id with
+ a decimal number.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Accept and ignore: Looks like we got the last chunk together
+ with the server headers. We better stop reading.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Accept and ignore: Continue hack in da house.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Accept and higlight: Rejecting connection from 10.0.0.2.
+ Maximum number of connections reached.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Accept and highlight: Loading actions file: /usr/local/etc/privoxy/default.action
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Accept and highlight: Loading filter file: /usr/local/etc/privoxy/default.filter
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Accept and highlight: Killed all-caps Host header line: HOST: bestproxydb.com
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Accept and highlight: Reducing expected bytes to 0. Marking
+ the server socket tainted after throwing 4 bytes away.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Accept: Merged multiple header lines to: 'X-FORWARDED-PROTO: http X-HOST: 127.0.0.1'
+ </para>
+ </listitem>
+ </itemizedlist>
</para>
</listitem>
<listitem>
<para>
- In verbose mode, or if the new option --show-skipped-tests
- is used, Privoxy-Regression-Test logs skipped tests and the
- skip reason.
+ Code cleanups:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Remove the next member from the client_state struct. Only the main
+ thread needs access to all client states so give it its own struct.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Garbage-collect request_contains_null_bytes().
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Ditch redundant code in unload_configfile().
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Ditch LogGetURLUnderCursor() which doesn't seem to be used anywhere.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ In write_socket(), remove the write-only variable write_len in
+ an ifdef __OS2__ block. Spotted by cppcheck.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ In connect_to(), don't declare the variable 'flags' on OS/2 where
+ it isn't used. Spotted by cppcheck.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Limit the scope of various variables. Spotted by cppcheck.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ In add_to_iob(), turn an interestingly looking for loop into a
+ boring while loop.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Code cleanup in preparation for external filters.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ In listen_loop(), mention the socket on which we accepted the
+ connection, not just the source IP address.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ In write_socket(), also log the socket we're writing to.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ In log_error(), assert that escaped characters get logged
+ completely or not at all.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ In log_error(), assert that ival and sval have reasonable values.
+ There's no reason not to abort() if they don't.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Remove an incorrect cgi_error_unknown() call in a
+ cannot-happen-situation in send_crunch_response().
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Clean up white-space in http_response definition and
+ move the crunch_reason to the beginning.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Turn http_response.reason into an enum and rename it
+ to http_response.crunch_reason.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Silence a 'gcc (Debian 4.3.2-1.1) 4.3.2' warning on i686 GNU/Linux.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Fix white-space in a log message in remove_chunked_transfer_coding().
+ While at it, add a note that the message doesn't seem to
+ be entirely correct and should be improved later on.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ GNUmakefile improvements:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Use $(SSH) instead of ssh, so one only needs to specify a username once.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Removed references to the action feedback thingy that hasn't been
+ working for years.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Consistently use shell.sourceforge.net instead of shell.sf.net so
+ one doesn't need to check server fingerprints twice.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Removed GNUisms in the webserver and webactions targets so they
+ work with standard tar.
+ </para>
+ </listitem>
+ </itemizedlist>
</para>
</listitem>
</itemizedlist>
<para>
Generally, an URL pattern has the form
- <literal><domain><port>/<path></literal>, where both the
- <literal><domain></literal> and <literal><port></literal>
- and <literal><path></literal> are optional. (This is why the special
+ <literal><domain><port>/<path></literal>, where the
+ <literal><domain></literal>, the <literal><port></literal>
+ and the <literal><path></literal> are optional. (This is why the special
<literal>/</literal> pattern matches all URLs). Note that the protocol
portion of the URL pattern (e.g. <literal>http://</literal>) should
<emphasis>not</emphasis> be included in the pattern. This is assumed already!
Expressions</quote></ulink> (POSIX 1003.2).
</para>
<para>
- The port part of pattern is decimal port number preceeded by a colon
- (<literal>:</literal>). If domain part contains numeric IPv6 address, you
- will need to quote the domain part by angle brackets
+ The port part of a pattern is a decimal port number preceded by a colon
+ (<literal>:</literal>). If the domain part contains a numerical IPv6 address,
+ it has to be put into angle brackets
(<literal><</literal>, <literal>></literal>).
</para>
</para>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>/</literal></term>
+ <listitem>
+ <para>
+ Matches any URL because there's no requirement for either the
+ domain or the path to match anything.
+ </para>
+ </listitem>
+ </varlistentry>
<varlistentry>
<term><literal>:8000/</literal></term>
<listitem>
<term><literal><2001:db8::1>/</literal></term>
<listitem>
<para>
- Matches any URL having <literal>2001:db8::1</literal> as a domain.
- (Note that real URL uses plain brackets, not an angle brackets.)
+ Matches any URL with the host address <literal>2001:db8::1</literal>.
+ (Note that the real URL uses plain brackets, not angle brackets.)
</para>
</listitem>
</varlistentry>
<quote>HTTP headers</quote> are, you definitely don't need to worry about this
one.
</para>
+ <para>
+ Headers added by this action are not modified by other actions.
+ </para>
</listitem>
</varlistentry>
<para>
Filtering requires buffering the page content, which may appear to
slow down page rendering since nothing is displayed until all content has
- passed the filters. (It does not really take longer, but seems that way
- since the page is not incrementally displayed.) This effect will be more
- noticeable on slower connections.
+ passed the filters. (The total time until the page is completely rendered
+ doesn't change much, but it may be perceived as slower since the page is
+ not incrementally displayed.)
+ This effect will be more noticeable on slower connections.
</para>
<para>
<quote>Rolling your own</quote>
USA
$Log: user-manual.sgml,v $
+ Revision 2.130 2010/12/01 19:28:28 fabiankeil
+ Hopefully unbreak the dok target when using some kind of jade.
+
+ Reported by Lee.
+
+ Revision 2.129 2010/11/13 20:17:11 fabiankeil
+ Merge ChangeLog updates
+
+ Revision 2.128 2010/11/10 22:00:13 fabiankeil
+ Update the first paragraph of the 'What's New' section.
+
+ Revision 2.127 2010/11/10 21:48:54 fabiankeil
+ Update the "What's New" section.
+
+ Revision 2.126 2010/11/06 12:55:48 fabiankeil
+ Set p-version to 3.0.17
+
+ Revision 2.125 2010/09/03 17:39:37 fabiankeil
+ Slightly improve the explanation of why filtering may appear slower than it is.
+
+ Revision 2.124 2010/05/01 18:21:30 fabiankeil
+ Explicitly mention how to match any URL.
+
+ Revision 2.123 2010/02/19 16:00:38 fabiankeil
+ Even more fixes.
+
+ Revision 2.122 2010/02/19 15:22:47 fabiankeil
+ Add missing word.
+
+ Revision 2.121 2010/02/15 15:30:13 fabiankeil
+ Mention the use of the no-such-domain template for DNS problems with FEATURE_IPV6_SUPPORT enabled.
+
+ Revision 2.120 2010/02/13 17:38:39 fabiankeil
+ Update entities for 3.0.16 stable.
+
+ Revision 2.119 2010/02/13 16:37:37 fabiankeil
+ Update 'What's new?' section.
+
+ Revision 2.118 2010/02/11 13:59:48 fabiankeil
+ Mention that the headers added by the add-header action aren't modified by other actions.
+
+ Revision 2.117 2010/01/11 12:56:04 fabiankeil
+ Bump copyright range as p-config.sgml's copyright line is only used in the config file.
+
+ Revision 2.116 2009/11/15 14:24:12 fabiankeil
+ Prepare to generate docs for 3.0.16 UNRELEASED.
+
+ Revision 2.115 2009/10/10 06:19:34 fabiankeil
+ Ditch a duplicated 'since'.
+
+ Revision 2.114 2009/10/10 05:51:48 fabiankeil
+ Update "What's new" section.
+
+ Revision 2.113 2009/10/10 05:48:55 fabiankeil
+ Prepare for 3.0.15 beta.
+
+ Revision 2.112 2009/07/24 12:20:30 fabiankeil
+ Remove duplicated period.
+
+ Revision 2.111 2009/07/18 18:11:11 fabiankeil
+ Don't claim that NTLM should work when there are multiple reports that it doesn't.
+
+ Revision 2.110 2009/07/18 16:25:17 fabiankeil
+ Fix trailing whitespace.
+
+ Revision 2.109 2009/07/18 16:24:39 fabiankeil
+ Bump entities for 3.0.14 beta.
+
+ Revision 2.108 2009/07/18 15:49:23 fabiankeil
+ Add most of the changes in 3.0.14 to the "What's New" section.
+
+ Revision 2.107 2009/06/12 14:30:58 fabiankeil
+ Update entities for 3.0.13 beta.
+
+ Revision 2.106 2009/06/12 11:04:13 fabiankeil
+ Import ChangeLog for 3.0.13 beta.
+
+ Revision 2.105 2009/04/17 11:32:57 fabiankeil
+ Grammar and spelling fixes.
+
+ Revision 2.104 2009/04/17 11:27:49 fabiankeil
+ Petr Pisar's privoxy-3.0.12-ipv6-3.diff.
+
Revision 2.103 2009/03/21 10:49:05 fabiankeil
Merge updated ChangeLog.
projects / privoxy.git / blobdiff