<!entity license SYSTEM "license.sgml">
<!entity p-authors SYSTEM "p-authors.sgml">
<!entity config SYSTEM "p-config.sgml">
-<!entity p-version "3.0.6">
-<!entity p-status "UNRELEASED">
+<!entity p-version "3.0.7">
+<!entity p-status "stable">
<!entity % p-authors-formal "INCLUDE"> <!-- include additional text, etc -->
-<!entity % p-not-stable "INCLUDE">
-<!entity % p-stable "IGNORE">
+<!entity % p-not-stable "IGNORE">
+<!entity % p-stable "INCLUDE">
<!entity % p-text "IGNORE"> <!-- define we are not a text only doc -->
<!entity % p-doc "INCLUDE"> <!-- and we are a formal doc -->
<!entity % p-readme "IGNORE">
This file belongs into
ijbswa.sourceforge.net:/home/groups/i/ij/ijbswa/htdocs/
- $Id: user-manual.sgml,v 2.23 2006/10/02 22:43:53 hal9 Exp $
+ $Id: user-manual.sgml,v 2.31 2007/06/02 14:01:37 fabiankeil Exp $
- Copyright (C) 2001- 2006 Privoxy Developers http://www.privoxy.org
+ Copyright (C) 2001-2007 Privoxy Developers http://www.privoxy.org/
See LICENSE.
========================================================================
<subscript>
<!-- Completely the wrong markup, but very little is allowed -->
<!-- in this part of an article. FIXME -->
- <link linkend="copyright">Copyright</link> &my-copy; 2001 - 2006 by
+ <link linkend="copyright">Copyright</link> &my-copy; 2001 - 2007 by
<ulink url="http://www.privoxy.org/">Privoxy Developers</ulink>
</subscript>
</pubdate>
-<pubdate>$Id: user-manual.sgml,v 2.23 2006/10/02 22:43:53 hal9 Exp $</pubdate>
+<pubdate>$Id: user-manual.sgml,v 2.31 2007/06/02 14:01:37 fabiankeil Exp $</pubdate>
<!--
How to install the binary packages depends on your operating system:
</para>
+<!-- XXX: The installation sections should be sorted -->
+
<!-- ~~~~~ New section ~~~~~ -->
-<sect3 id="installation-pack-rpm"><title>Red Hat, SuSE and Conectiva RPMs</title>
+<sect3 id="installation-pack-rpm"><title>Red Hat and Fedora RPMs</title>
<para>
RPMs can be installed with <literal>rpm -Uvh privoxy-&p-version;-1.rpm</literal>,
</sect3>
<!-- ~~~~~ New section ~~~~~ -->
-<sect3 id="installation-pack-bintgz"><title>Solaris, NetBSD, FreeBSD, HP-UX</title>
+<sect3 id="installation-pack-bintgz"><title>Solaris, NetBSD, HP-UX</title>
<para>
Create a new directory, <literal>cd</literal> to it, then unzip and
</para>
</sect3>
+<!-- ~~~~~ New section ~~~~~ -->
+<sect3 id="installation-tbz"><title>FreeBSD</title>
+
+<para>
+ Privoxy is part of FreeBSD's Ports Collection, you can build and install
+ it with <literal>cd /usr/ports/www/privoxy; make install clean</literal>.
+</para>
+<para>
+ If you don't use the ports, you can fetch and install
+ the package with <literal>pkg_add -r privoxy</literal>.
+</para>
+<para>
+ The port skeleton and the package can also be downloaded from the
+ <ulink url="https://sourceforge.net/project/showfiles.php?group_id=11118">File Release
+ Page</ulink>, but if you're interested in stable releases only you don't
+ gain anything by using them.
+</para>
+</sect3>
+
<!-- ~~~~~ New section ~~~~~ -->
<sect3 id="installattion-gentoo"><title>Gentoo</title>
<para>
<para>
The most convenient way to obtain the <application>Privoxy</application> sources
- is to download the source tarball from our <ulink url="http://sf.net/projects/ijbswa/">project
+ is to download the source tarball from our
+ <ulink url="http://sourceforge.net/project/showfiles.php?group_id=11118&package_id=10571">project download
page</ulink>.
</para>
<sect1 id="whatsnew">
<title>What's New in this Release</title>
<para>
- There are many improvements and new features since the last <application>Privoxy</application> stable release:
+ There are many improvements and new features since <application>Privoxy 3.0.6</application>, the last stable release:
</para>
<para>
<itemizedlist>
<listitem>
<para>
- Multiple <link linkend="filter-file">filter files</link> can now be specified in <filename>config</filename>. This allows for
- locally defined filters that can be maintained separately from the filters as
- supplied by the developers, i.e. <filename>default.filter</filename>.
+ Header filtering can be done with dedicated header filters now. As a result
+ the actions <quote>filter-client-headers</quote> and <quote>filter-server-headers</quote>
+ that were introduced with <application>Privoxy 3.0.5</application> to apply
+ the content filters to the headers as, well have been removed again.
</para>
</listitem>
-
+
+<!-- pre-3.0.6 changes:
<listitem>
<para>
There are a number of new <link linkend="actions-file">actions</link>:
<para>
In addition, there are numerous bug fixes and significant enhancements,
including error pages should no longer be cached if the problem is fixed,
- much better DNS error handling, and various logging improvements.
- </para>
- </listitem>
-
- <listitem>
- <para>
- The default actions setting is now <literal>Cautious</literal>. Previous
- releases had a default setting of <literal>Medium</literal>. Experienced
- users may want to adjust this, as it is fairly conservative by &my-app;
- standards and past practices. See <ulink
- url="http://config.privoxy.org/edit-actions-list?f=default">
- http://config.privoxy.org/edit-actions-list?f=default</ulink>. New users
- should try the default settings for a while before turning up the volume.
+ much better DNS error handling, various logging improvements, and
+ configuration updates for better ad blocking and junk elimination.
</para>
</listitem>
+-->
</itemizedlist>
</para>
default. This is primarily a matter of emphasis, but some features
you may have been used to, may now be <quote>off</quote> by default.
There are also a number of new actions and filters you may want to
- consider, most of which are not incorporated into the default settings as
- yet (see above).
+ consider, most of which are not fully incorporated into the default
+ settings as yet (see above).
</para>
</listitem>
-
+
+ <listitem>
+ <para>
+ The default actions setting is now <literal>Cautious</literal>. Previous
+ releases had a default setting of <literal>Medium</literal>. Experienced
+ users may want to adjust this, as it is fairly conservative by &my-app;
+ standards and past practices. See <ulink
+ url="http://config.privoxy.org/edit-actions-list?f=default">
+ http://config.privoxy.org/edit-actions-list?f=default</ulink>. New users
+ should try the default settings for a while before turning up the volume.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The default setting has filtering turned <emphasis>off</emphasis>, which
+ subsequently means that compression is <emphasis>on</emphasis>. Remember
+ that filtering does not work on compressed pages, so if you use, or want to
+ use, filtering, you will need to force compression off. Example:
+ </para>
+ <para>
+ <screen>
+ { +<link linkend="filter">filter</link>{google} +<link linkend="prevent-compression">prevent-compression</link> }
+ .google.</screen>
+ </para>
+ <para>
+ Or if you use a number of filters, or filter many sites, you may just want
+ to turn off compression for all sites in
+ <filename>default.action</filename> (or
+ <filename>user.action</filename>).
+ </para>
+
+ </listitem>
+
+ <listitem>
+ <para>
+ Also, <link linkend="SESSION-COOKIES-ONLY">session-cookies-only</link> is
+ off by default now. If you've liked this feature in the past, you may want
+ to turn it back on in <filename>user.action</filename> now.
+ </para>
+ </listitem>
+
+
<listitem>
<para>
<!-- I think it is best to keep this somewhat vague, in case -->
</para>
</listitem>
+
</itemizedlist>
</para>
</sect2>
<para>
First a bit of a warning ... blocking ads is much like blocking SPAM: the
more aggressive you are about it, the more likely you are to block
- things that were not intended. So there is a trade off here. If you want
+ things that were not intended. And the more likely that some things
+ may not work as intended. So there is a trade off here. If you want
extreme ad free browsing, be prepared to deal with more
<quote>problem</quote> sites, and to spend more time adjusting the
configuration to solve these unintended consequences. In short, there is
Secondly, a brief explanation of <application>Privoxy's </application>
<quote>actions</quote>. <quote>Actions</quote> in this context, are
the directives we use to tell <application>Privoxy</application> to perform
- some task relating to HTTP transactions (i.e. web browsing). We tell
+ some task relating to WWW transactions (i.e. web browsing). We tell
<application>Privoxy</application> to take some <quote>action</quote>. Each
action has a unique name and function. While there are many potential
<application>actions</application> in <application>Privoxy's</application>
original page's HTML content. An ad image for instance, is just an URL
embedded in the page somewhere. The image itself may be on the same server,
or a server somewhere else on the Internet. Complex web pages will have many
- such embedded URLs.
+ such embedded URLs. &my-app; can deal with each URL individually, so, for
+ instance, the main page text is not touched, but images from such-and-such
+ server are blocked.
</para>
<para>
- The actions we need to know about for ad blocking are: <literal><link
+ The most important actions for basic ad blocking are: <literal><link
linkend="block">block</link></literal>, <literal><link
- linkend="handle-as-image">handle-as-image</link></literal>, and
+ linkend="handle-as-image">handle-as-image</link></literal>,
+ <literal><link
+ linkend="handle-as-empty-document">handle-as-empty-document</link></literal>,and
<literal><link linkend="set-image-blocker">set-image-blocker</link></literal>:
</para>
<listitem>
<para>
- <literal><link linkend="block">block</link></literal> - this action stops
- any contact between your browser and any URL patterns that match this
- action's configuration. It can be used for blocking ads, but also anything
- that is determined to be unwanted. By itself, it simply stops any
- communication with the remote server and sends <application>Privoxy</application>'s
- own built-in BLOCKED page instead to let you now what has happened.
+ <literal><link linkend="block">block</link></literal> - this is perhaps
+ the single most used action, and is particularly important for ad blocking.
+ This action stops any contact between your browser and any URL patterns
+ that match this action's configuration. It can be used for blocking ads,
+ but also anything that is determined to be unwanted. By itself, it simply
+ stops any communication with the remote server and sends
+ <application>Privoxy</application>'s own built-in BLOCKED page instead to
+ let you now what has happened (with some exceptions, see below).
</para>
</listitem>
</para>
</listitem>
+ <listitem>
+ <para>
+ <literal><link linkend="handle-as-empty-document">handle-as-empty-document</link></literal> -
+ sends an empty document instead of <application>Privoxy's</application>
+ normal BLOCKED HTML page. This is useful for file types that are neither
+ HTML nor images, such as blocking JavaScript files.
+ </para>
+ </listitem>
+
<listitem>
<para>
<literal><link
to now go to the <link linkend="act-examples">Actions Files Tutorial</link>.
The ideas explained therein also apply to the web-based editor.
</para>
+<para>
+ There are also various
+ <link linkend="filter">filters</link> that can be used for ad blocking
+ (filters are a special subset of actions). These
+ fall into the <quote>advanced</quote> usage category, and are explained in
+ depth in later sections.
+</para>
</sect2>
<para>
- With <application>Firefox</application>, this can be set under:
+ With <application>Firefox</application>, this is typically set under:
</para>
<literallayout>
</literallayout>
+<para>
+ Or optionally on some platforms:
+</para>
+
+<literallayout>
+ <guibutton>Edit</guibutton> -> <guibutton>Preferences</guibutton> -> <guibutton>General</guibutton> -> <guibutton>Connection Settings</guibutton> -> <guibutton>Manual Proxy Configuration</guibutton>
+
+</literallayout>
+
<para>
With <application>Netscape</application> (and
</para>
<sect2 id="start-redhat">
-<title>Red Hat, Fedora and Conectiva</title>
+<title>Red Hat and Fedora</title>
<para>
A default Red Hat installation may not start &my-app; upon boot. It will use
the file <filename>/etc/privoxy/config</filename> as its main configuration
</para>
</sect2>
+<!--
+ omitting 10/31/06 HB
+
<sect2 id="start-suse">
<title>SuSE</title>
<para>
</screen>
</para>
</sect2>
-
+-->
<sect2 id="start-windows">
<title>Windows</title>
<para>
<filename>default.action</filename> (which you will most probably want
to define sooner or later) are probably best applied in
<filename>user.action</filename>, where you can preserve them across
- upgrades. <filename>standard.action</filename> is for
+ upgrades. <filename>standard.action</filename> is only for
<application>Privoxy's</application> internal use.
</para>
<para>
</listitem>
<listitem>
<para>
- <filename>standard.action</filename> - is used by the web based editor
+ <filename>standard.action</filename> - is used only by the web based editor
at <ulink url="http://config.privoxy.org/edit-actions-list?f=default">
http://config.privoxy.org/edit-actions-list?f=default</ulink>,
to set various pre-defined sets of rules for the default actions section
The <guibutton>Edit</guibutton> button allows you to turn each
action on/off individually for fine-tuning. The <guibutton>Cautious</guibutton>
button changes the actions list to low/safe settings which will activate
- a minimal set of &my-app;'s features, and subsequently there will be
- less of a chance for accidental problems. The <guibutton>Medium</guibutton>
- button sets the list to a medium level of ad blocking and a low level set of
- privacy features. The <guibutton>Advanced</guibutton> button
- sets the list to a high level of ad blocking and medium level of
- privacy. See the chart below. The latter three buttons over-ride
- any changes via with the <guibutton>Edit</guibutton> button. More
- fine-tuning can be done in the lower sections of this internal page.
+ ad blocking and a minimal set of &my-app;'s features, and subsequently
+ there will be less of a chance for accidental problems. The
+ <guibutton>Medium</guibutton> button sets the list to a medium level of
+ other features and a low level set of privacy features. The
+ <guibutton>Advanced</guibutton> button sets the list to a high level of
+ ad blocking and medium level of privacy. See the chart below. The latter
+ three buttons over-ride any changes via with the
+ <guibutton>Edit</guibutton> button. More fine-tuning can be done in the
+ lower sections of this internal page.
</para>
<para>
It is not recommend to edit the <filename>standard.action</filename> file
<para>
The list of actions files to be used are defined in the main configuration
file, and are processed in the order they are defined (e.g.
- <filename>default.action</filename> is typically process before
+ <filename>default.action</filename> is typically processed before
<filename>user.action</filename>). The content of these can all be viewed and
edited from <ulink
url="http://config.privoxy.org/show-status">http://config.privoxy.org/show-status</ulink>.
<sect2 id="actions-apply">
-<title>How Actions are Applied to URLs</title>
+<title>How Actions are Applied to Requests</title>
<para>
Actions files are divided into sections. There are special sections,
like the <quote><link linkend="aliases">alias</link></quote> sections which will
be discussed later. For now let's concentrate on regular sections: They have a
heading line (often split up to multiple lines for readability) which consist
of a list of actions, separated by whitespace and enclosed in curly braces.
- Below that, there is a list of URL patterns, each on a separate line.
+ Below that, there is a list of URL and tag patterns, each on a separate line.
</para>
<para>
To determine which actions apply to a request, the URL of the request is
- compared to all patterns in each <quote>action file</quote> file. Every time it matches, the list of
- applicable actions for the URL is incrementally updated, using the heading
- of the section in which the pattern is located. If multiple matches for
- the same URL set the same action differently, the last match wins. If not,
- the effects are aggregated. E.g. a URL might match a regular section with
- a heading line of <literal>{
+ compared to all URL patterns in each <quote>action file</quote>.
+ Every time it matches, the list of applicable actions for the request is
+ incrementally updated, using the heading of the section in which the
+ pattern is located. The same is done again for tags and tag patterns later on.
+</para>
+
+<para>
+ If multiple applying sections set the same action differently,
+ the last match wins. If not, the effects are aggregated.
+ E.g. a URL might match a regular section with a heading line of <literal>{
+<link linkend="handle-as-image">handle-as-image</link> }</literal>,
then later another one with just <literal>{
+<link linkend="block">block</link> }</literal>, resulting
</para>
<para>
- You can trace this process for any given URL by visiting <ulink
+ You can trace this process for
URL patterns and any given URL by visiting <ulink
url="http://config.privoxy.org/show-url-info">http://config.privoxy.org/show-url-info</ulink>.
</para>
</para>
<para>
- Generally, a <application>Privoxy</application> pattern has the form
+ Generally, a URL pattern has the form
<literal><domain>/<path></literal>, where both the
<literal><domain></literal> and <literal><path></literal> are
optional. (This is why the special <literal>/</literal> pattern matches all
</variablelist>
<para>
- While flexibile, this is not the sophistication of full regular expression based syntax.
+ While flexible, this is not the sophistication of full regular expression based syntax.
</para>
</sect3>
</sect3>
+<!-- ~ End section ~ -->
+
+
+<!-- ~~~~~ New section ~~~~~ -->
+<sect3 id="tag-pattern"><title>The Tag Pattern</title>
+
+<para>
+ Tag patterns are used to change the applying actions based on the
+ request's tags. Tags can be created with either the
+ <link linkend="CLIENT-HEADER-FILTER">client-header-tagger</link>
+ or the <link linkend="SERVER-HEADER-FILTER">server-header-tagger</link> action.
+</para>
+
+<para>
+ Tag patterns have to start with <quote>TAG:</quote>, so &my-app;
+ can tell them apart from URL patterns. Everything after the colon
+ including white space, is interpreted as a regular expression with
+ path patterns syntax, except that tag patterns aren't left-anchored
+ automatically (Privoxy doesn't silently add a <quote>^</quote>,
+ you have to do it yourself if you need it).
+</para>
+
+<para>
+ To match all requests that are tagged with <quote>foo</quote>
+ your pattern line should be <quote>TAG:^foo$</quote>,
+ <quote>TAG:foo</quote> would work as well, but it would also
+ match requests whose tags contain <quote>foo</quote> somewhere.
+</para>
+
+<para>
+ Sections can contain URL and tag patterns at the same time,
+ but tag patterns are checked after the URL patterns and thus
+ always overrule them, even if they are located before the URL patterns.
+</para>
+
+<para>
+ Once a new tag is added, Privoxy checks right away if it's matched by one
+ of the tag patterns and updates the action settings accordingly. As a result
+ tags can be used to activate other tagger actions, as long as these other
+ taggers look for headers that haven't already be parsed.
+</para>
+
+<para>
+ For example you could tag client requests which use the POST method,
+ use this tag to activate another tagger that adds a tag if cookies
+ are send, and then block based on the cookie tag. However if you'd
+ reverse the position of the described taggers, and activated the method
+ tagger based on the cookie tagger, no method tags would be created.
+ The method tagger would look for the request line, but at the time
+ the cookie tag is created the request line has already been parsed.
+</para>
+
+<para>
+ While this is a limitation you should be aware of, this kind of
+ indirection is seldom needed anyway and even the example doesn't
+ make too much sense.
+</para>
+
+</sect3>
+
</sect2>
<!-- ~ End section ~ -->
</sect3>
+<!-- ~~~~~ New section ~~~~~ -->
+<sect3 renderas="sect4" id="client-header-filter">
+<title>client-header-filter</title>
+
+<variablelist>
+ <varlistentry>
+ <term>Typical use:</term>
+ <listitem>
+ <para>
+ Rewrite or remove single client headers.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Effect:</term>
+ <listitem>
+ <para>
+ All client headers to which this action applies are filtered on-the-fly through
+ the specified regular expression based substitutions.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type:</term>
+ <!-- boolean, parameterized, Multi-value -->
+ <listitem>
+ <para>Parameterized.</para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Parameter:</term>
+ <listitem>
+ <para>
+ The name of a client-header filter, as defined in one of the
+ <link linkend="filter-file">filter files</link>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Notes:</term>
+ <listitem>
+ <para>
+ Client-header filters are applied to each header on its own, not to
+ all at once. This makes it easier to diagnose problems, but on the downside
+ you can't write filters that only change header x if header y's value is z.
+ You can do that by using tags though.
+ </para>
+ <para>
+ Client-header filters are executed after the other header actions have finished
+ and use their output as input.
+ </para>
+ <para>
+ Please refer to the <link linkend="filter-file">filter file chapter</link>
+ to learn which client-header filters are available by default, and how to
+ create your own.
+ </para>
+
+ </varlistentry>
+
+ <varlistentry>
+ <term>Example usage (section):</term>
+ <listitem>
+ <para>
+ <screen>
+{+client-header-filter{hide-tor-exit-notation}}
+.exit/
+ </screen>
+ </para>
+ </listitem>
+ </varlistentry>
+
+</variablelist>
+</sect3>
+
+
+<!-- ~~~~~ New section ~~~~~ -->
+<sect3 renderas="sect4" id="client-header-tagger">
+<title>client-header-tagger</title>
+
+<variablelist>
+ <varlistentry>
+ <term>Typical use:</term>
+ <listitem>
+ <para>
+ Block requests based on their headers.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Effect:</term>
+ <listitem>
+ <para>
+ Client headers to which this action applies are filtered on-the-fly through
+ the specified regular expression based substitutions, the result is used as
+ tag.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type:</term>
+ <!-- boolean, parameterized, Multi-value -->
+ <listitem>
+ <para>Parameterized.</para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Parameter:</term>
+ <listitem>
+ <para>
+ The name of a client-header tagger, as defined in one of the
+ <link linkend="filter-file">filter files</link>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Notes:</term>
+ <listitem>
+ <para>
+ Client-header taggers are applied to each header on its own,
+ and as the header isn't modified, each tagger <quote>sees</quote>
+ the original.
+ </para>
+ <para>
+ Client-header taggers are the first actions that are executed
+ and their tags can be used to control every other action.
+ </para>
+
+ </varlistentry>
+
+ <varlistentry>
+ <term>Example usage (section):</term>
+ <listitem>
+ <para>
+ <screen>
+# Tag every request with the User-Agent header
+{+client-header-filter{user-agent}}
+/
+ </screen>
+ </para>
+ </listitem>
+ </varlistentry>
+
+</variablelist>
+</sect3>
+
+
<!-- ~~~~~ New section ~~~~~ -->
<sect3 renderas="sect4" id="content-type-overwrite">
<!--
This limitation exists for a reason, think twice before circumventing it.
</para>
<para>
- Most of the time it's easier to enable
- <literal><link linkend="filter-server-headers">filter-server-headers</link></literal>
- and replace this action with a custom regular expression. It allows you
- to activate it for every document of a certain site and it will still
+ Most of the time it's easier to replace this action with a custom
+ <literal><link linkend="server-header-filter">server-header filter</link></literal>.
+ It allows you to activate it for every document of a certain site and it will still
only replace the content types you aimed at.
</para>
<para>
<listitem>
<para>
<screen># Check if www.example.net/ really uses valid XHTML
-{+content-type-overwrite {application/xml}}
+{ +content-type-overwrite{application/xml} }
www.example.net/
# but leave the content type unmodified if the URL looks like a style sheet
{-content-type-overwrite}
-www.example.net/*.\.css$
-www.example.net/*.style
+www.example.net/.*\.css$
+www.example.net/.*style
</screen>
</para>
</listitem>
<para>
<literal>crunch-client-header</literal> is only meant for quick tests.
If you have to block several different headers, or only want to modify
- parts of them, you should enable
- <literal><link linkend="filter-client-headers">filter-client-headers</link></literal>
- and create your own filter.
+ parts of them, you should use a
+ <literal><link linkend="client-header-filter">client-header filter</link></literal>.
</para>
<warning>
<para>
<listitem>
<para>
<screen># Block the non-existent "Privacy-Violation:" client header
-{+crunch-client-header {Privacy-Violation:}}
+{ +crunch-client-header{Privacy-Violation:} }
/
</screen>
</para>
<listitem>
<para>
<screen># Let the browser revalidate cached documents without being tracked across sessions
-{+hide-if-modified-since {-60} \
-+overwrite-last-modified {randomize} \
-+crunch-if-none-match}
+{ +hide-if-modified-since{-60} \
+ +overwrite-last-modified{randomize} \
+ +crunch-if-none-match}
/ </screen>
</para>
</listitem>
<para>
<literal>crunch-server-header</literal> is only meant for quick tests.
If you have to block several different headers, or only want to modify
- parts of them, you should enable
- <literal><link linkend="filter-server-headers">filter-server-headers</link></literal>
- and create your own filter.
+ parts of them, you should use a custom
+ <literal><link linkend="server-header-filter">server-header filter</link></literal>.
</para>
<warning>
<para>
<listitem>
<para>
<screen># Crunch server headers that try to prevent caching
-{+crunch-server-header {no-cache}}
+{ +crunch-server-header{no-cache} }
/ </screen>
</para>
</listitem>
followed by another parameter. <literal>fast-redirects</literal> doesn't know that
and will cause a redirect to <quote>http://www.example.net/&foo=bar</quote>.
Depending on the target server configuration, the parameter will be silently ignored
- or lead to a <quote>page not found</quote> error. It is possible to fix these redirected
- requests with <literal><link linkend="filter-client-headers">filter-client-headers</link></literal>
- but it requires a little effort.
+ or lead to a <quote>page not found</quote> error. You can prevent this problem by
+ first using the <literal><link linkend="redirect">redirect</link></literal> action
+ to remove the last part of the URL, but it requires a little effort.
</para>
<para>
To detect a redirection URL, <literal>fast-redirects</literal> only
<term>Effect:</term>
<listitem>
<para>
- All files of text-based type, most notably HTML and
- JavaScript, to which this action applies, can be filtered on-the-fly
- through the specified regular expression based substitutions. (Note: as of
- version 3.0.3 plain text documents are exempted from filtering, because
- web servers often use the <literal>text/plain</literal> MIME type for all
- files whose type they don't know.) By default, filtering works only on the
- raw document content itself (that which can be seen with <literal>View
- Source</literal>),
- not the headers.
+ All instances of text-based type, most notably HTML and JavaScript, to which
+ this action applies, can be filtered on-the-fly through the specified regular
+ expression based substitutions. (Note: as of version 3.0.3 plain text documents
+ are exempted from filtering, because web servers often use the
+ <literal>text/plain</literal> MIME type for all files whose type they don't know.)
</para>
</listitem>
</varlistentry>
<term>Parameter:</term>
<listitem>
<para>
- The name of a filter, as defined in the <link linkend="filter-file">filter file</link>.
+ The name of a content filter, as defined in the <link linkend="filter-file">filter file</link>.
Filters can be defined in one or more files as defined by the
<literal><link linkend="filterfile">filterfile</link></literal>
option in the <link linkend="config">config file</link>.
by defining appropriate <literal>-filter</literal> exceptions.
</para>
<para>
- At this time, <application>Privoxy</application> cannot uncompress compressed
- documents. If you want filtering to work on all documents, even those that
- would normally be sent compressed, you must use the
- <literal><link linkend="prevent-compression">prevent-compression</link></literal>
+ Compressed content can't be filtered either, unless &my-app;
+ is compiled with zlib support (requires at least &my-app; 3.0.7),
+ in which case &my-app; will decompress the content before filtering
+ it.
+ </para>
+ <para>
+ If you use a &my-app; version without zlib support, but want filtering to work on
+ as much documents as possible, even those that would normally be sent compressed,
+ you must use the <literal><link linkend="prevent-compression">prevent-compression</link></literal>
action in conjunction with <literal>filter</literal>.
</para>
<para>
- Filtering can achieve some of the same effects as the
+ Content filtering can achieve some of the same effects as the
<literal><link linkend="block">block</link></literal>
action, i.e. it can be used to block ads and banners. But the mechanism
works quite differently. One effective use, is to block ad banners
<screen>+filter{blogspot} # Cleans up Blogspot blogs</screen>
</para>
<para>
- <anchor id="filter-html-to-xml">
- <screen>+filter{html-to-xml} # Header filter to change the Content-Type from html to xml</screen>
- </para>
- <para>
- <anchor id="filter-xml-to-html">
- <screen>+filter{xml-to-html} # Header filter to change the Content-Type from xml to html</screen>
+ <anchor id="filter-no-ping">
+ <screen>+filter{no-ping} # Removes non-standard ping attributes from anchor and area tags</screen>
</para>
</listitem>
</varlistentry>
<!-- ~~~~~ New section ~~~~~ -->
-<sect3 renderas="sect4" id="filter-client-headers">
-<title>filter-client-headers</title>
-
-<variablelist>
- <varlistentry>
- <term>Typical use:</term>
- <listitem>
- <para>
- To apply filtering to the client's (browser's) headers
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>Effect:</term>
- <listitem>
- <para>
- By default, <application>Privoxy's</application> filters only apply
- to the document content itself. This will extend those filters to
- include the client's headers as well.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>Type:</term>
- <!-- boolean, parameterized, Multi-value -->
- <listitem>
- <para>Boolean.</para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>Parameter:</term>
- <listitem>
- <para>
- N/A
- </para>
- </listitem>
- </varlistentry>
-
-<varlistentry>
- <term>Notes:</term>
- <listitem>
- <para>
- Regular expressions can be used to filter headers as well. Check your
- filters closely before activating this action, as it can easily lead to broken
- requests.
- </para>
- <para>
- These filters are applied to each header on its own, not to them
- all at once. This makes it easier to diagnose problems, but on the downside
- you can't write filters that only change header x if header y's value is
- z.
- </para>
- <para>
- The filters are used after the other header actions have finished and can
- use their output as input.
- </para>
-
- <para>
- Whenever possible one should specify <literal>^</literal>,
- <literal>$</literal>, the whole header name and the colon, to make sure
- the filter doesn't cause havoc to other headers or the
- page itself. For example if you want to transform
- <application>Galeon</application> User-Agents to
- <application>Firefox</application> User-Agents you
- shouldn't use:
-</para>
-<para>
-<screen>
-s@Galeon/\d\.\d\.\d @@
-</screen>
-</para><para>
- but:
-</para><para>
-<screen>
-s@^(User-Agent:.*) Galeon/\d\.\d\.\d (Firefox/\d\.\d\.\d\.\d)$@$1 $2@
-</screen>
-</para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>Example usage (section):</term>
- <listitem>
- <para>
- <screen>
-{+filter-client-headers +filter{test_filter}}
-problem-host.example.com
- </screen>
- </para>
- </listitem>
- </varlistentry>
-
-</variablelist>
-</sect3>
-
-
-<!-- ~~~~~ New section ~~~~~ -->
-<sect3 renderas="sect4" id="filter-server-headers">
-<title>filter-server-headers</title>
-
+<sect3 renderas="sect4" id="force-text-mode">
+<title>force-text-mode</title>
+<!--
+new action
+-->
<variablelist>
<varlistentry>
<term>Typical use:</term>
<listitem>
- <para>
- To apply filtering to the server's headers
- </para>
+ <para>Force <application>Privoxy</application> to treat a document as if it was in some kind of <emphasis>text</emphasis> format. </para>
</listitem>
</varlistentry>
<term>Effect:</term>
<listitem>
<para>
- By default, <application>Privoxy's</application> filters only apply
- to the document content itself. This will extend those filters to
- include the server's headers as well.
- </para>
+ Declares a document as text, even if the <quote>Content-Type:</quote> isn't detected as such.
+ </para>
</listitem>
</varlistentry>
<varlistentry>
<term>Type:</term>
- <!-- boolean, parameterized, Multi-value -->
+ <!-- Boolean, Parameterized, Multi-value -->
<listitem>
<para>Boolean.</para>
</listitem>
<varlistentry>
<term>Parameter:</term>
- <listitem>
- <para>
- N/A
- </para>
- </listitem>
- </varlistentry>
-
-<varlistentry>
- <term>Notes:</term>
- <listitem>
- <para>
- Similar to <literal>filter-client-headers</literal>, but works on
- the server instead. To filter both server and client, use both.
- </para>
- <para>
- As with <literal>filter-client-headers</literal>, check your
- filters before activating this action, as it can easily lead to broken
- requests.
- </para>
- <para>
- These filters are applied to each header on its own, not to them
- all at once. This makes it easier to diagnose problems, but on the downside
- you can't write filters that only change header x if header y's value is
- z.
- </para>
- <para>
- The filters are used after the other header actions have finished and can
- use their output as input.
- </para>
+ <listitem>
<para>
- Remember too, whenever possible one should specify <literal>^</literal>,
- <literal>$</literal>, the whole header name and the colon, to make sure
- the filter doesn't cause havoc to other headers or the
- page itself. See above for example.
+ N/A
</para>
-
</listitem>
</varlistentry>
<varlistentry>
- <term>Example usage (section):</term>
+ <term>Notes:</term>
<listitem>
+ <para>
+ As explained <literal><link linkend="filter">above</link></literal>,
+ <application>Privoxy</application> tries to only filter files that are
+ in some kind of text format. The same restrictions apply to
+ <literal><link linkend="content-type-overwrite">content-type-overwrite</link></literal>.
+ <literal>force-text-mode</literal> declares a document as text,
+ without looking at the <quote>Content-Type:</quote> first.
+ </para>
+ <warning>
<para>
- <screen>
-{+filter-server-headers +filter{test_filter}}
-problem-host.example.com
- </screen>
+ Think twice before activating this action. Filtering binary data
+ with regular expressions can cause file damage.
</para>
+ </warning>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Example usage:</term>
+ <listitem>
+ <para>
+ <screen>
++force-text-mode
+ </screen>
+ </para>
</listitem>
</varlistentry>
-
</variablelist>
</sect3>
<!-- ~~~~~ New section ~~~~~ -->
-<sect3 renderas="sect4" id="force-text-mode">
-<title>force-text-mode</title>
+<sect3 renderas="sect4" id="forward-override">
+<title>forward-override</title>
<!--
new action
-->
<varlistentry>
<term>Typical use:</term>
<listitem>
- <para>Force <application>Privoxy</application> to treat a document as if it was in some kind of <emphasis>text</emphasis> format. </para>
+ <para>Change the forwarding settings based on User-Agent or request origin</para>
</listitem>
</varlistentry>
<term>Effect:</term>
<listitem>
<para>
- Declares a document as text, even if the <quote>Content-Type:</quote> isn't detected as such.
+ Overrules the forward directives in the configuration files.
</para>
</listitem>
</varlistentry>
<term>Type:</term>
<!-- Boolean, Parameterized, Multi-value -->
<listitem>
- <para>Boolean.</para>
+ <para>Multi-value.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Parameter:</term>
<listitem>
- <para>
- N/A
- </para>
+ <itemizedlist>
+ <listitem>
+ <para><quote>forward .</quote> to use a direct connection without any additional proxies.</para>
+ </listitem>
+ <listitem>
+ <para>
+ <quote>forward 127.0.0.1:8123</quote> to use the HTTP proxy listening at 127.0.0.1 port 8123.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <quote>forward-socks4a 127.0.0.1:9050 .</quote> to use the socks4a proxy listening at 127.0.0.1 port 9050.
+ Replace <quote>forward-socks4a</quote> with <quote>forward-socks4</quote> to use a socks4 connection (with local DNS
+ resolution) instead.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <quote>forward-socks4a 127.0.0.1:9050 proxy.example.org:8000</quote> to use the socks4a proxy
+ listening at 127.0.0.1 port 9050 to reach the HTTP proxy listening at proxy.example.org port 8000.
+ Replace <quote>forward-socks4a</quote> with <quote>forward-socks4</quote> to use a socks4 connection (with local DNS
+ resolution) instead.
+ </para>
+ </listitem>
+ </itemizedlist>
</listitem>
</varlistentry>
<term>Notes:</term>
<listitem>
<para>
- As explained <literal><link linkend="filter">above</link></literal>,
- <application>Privoxy</application> tries to only filter files that are
- in some kind of text format. The same restrictions apply to
- <literal><link linkend="content-type-overwrite">content-type-overwrite</link></literal>.
- <literal>force-text-mode</literal> declares a document as text,
- without looking at the <quote>Content-Type:</quote> first.
+ This action takes parameters similar to the <!-- I hope this link actual works -->
+ <link linkend="forwarding">forward</link> directives in the configuration
+ file, but without the URL pattern. It can be used as replacement, but normally it's only
+ used in cases where matching based on the request URL isn't sufficient.
</para>
<warning>
<para>
- Think twice before activating this action. Filtering binary data
- with regular expressions can cause file damage.
+ Please read the description for the <link linkend="forwarding">forward</link> directives before
+ using this action. Forwarding to the wrong people will reduce your privacy and increase the
+ chances of man-in-the-middle attacks.
+ </para>
+ <para>
+ If the ports are missing or invalid, default values will be used. This might change
+ in the future and you shouldn't rely on it. Otherwise incorrect syntax causes Privoxy
+ to exit.
+ </para>
+ <para>
+ Use the <ulink url="http://config.privoxy.org/show-url-info">show-url-info CGI page</ulink>
+ to verify that your forward settings do what you thought the do.
</para>
</warning>
</listitem>
<listitem>
<para>
<screen>
-+force-text-mode
+# Always use direct connections for requests previously tagged as
+# <quote>User-Agent: fetch libfetch/2.0</quote> and make sure
+# resuming downloads continues to work.
+# This way you can continue to use Tor for your normal browsing,
+# without overloading the Tor network with your FreeBSD ports updates
+# or downloads of bigger files like ISOs.
+{+forward-override{forward .} \
+ -hide-if-modified-since \
+ -overwrite-last-modified \
+}
+TAG:^User-Agent: fetch libfetch/2.0$
</screen>
</para>
</listitem>
<para>
This action alone doesn't do anything noticeable. It just marks URLs.
If the <literal><link linkend="block">block</link></literal> action <emphasis>also applies</emphasis>,
- the presence or absence of this mark decides whether an HTML <quote>blocked</quote>
+ the presence or absence of this mark decides whether an HTML <quote>BLOCKED</quote>
page, or an empty document will be sent to the client as a substitute for the blocked content.
The <emphasis>empty</emphasis> document isn't literally empty, but actually contains a single space.
</para>
Some browsers complain about syntax errors if JavaScript documents
are blocked with <application>Privoxy's</application>
default HTML page; this option can be used to silence them.
+ And of course this action can also be used to eliminate the &my-app;
+ BLOCKED message in frames.
</para>
<para>
The content type for the empty document can be specified with
<listitem>
<para>
<screen># Disarm the download link in Sourceforge's patch tracker
-{-filter\
-+content-type-overwrite {text/plain}\
-+hide-content-disposition {block} }
-.sourceforge.net/tracker/download.php</screen>
+{ -filter \
+ +content-type-overwrite{text/plain}\
+ +hide-content-disposition{block} }
+ .sourceforge.net/tracker/download\.php</screen>
</para>
</listitem>
</varlistentry>
<listitem>
<para>
<screen># Let the browser revalidate without being tracked across sessions
-{+hide-if-modified-since {-60}\
-+overwrite-last-modified {randomize}\
-+crunch-if-none-match}
+{ +hide-if-modified-since{-60} \
+ +overwrite-last-modified{randomize} \
+ +crunch-if-none-match}
/</screen>
</para>
</listitem>
<listitem>
<para>
More and more websites send their content compressed by default, which
- is generally a good idea and saves bandwidth. But for the <literal><link
+ is generally a good idea and saves bandwidth. But the <literal><link
linkend="filter">filter</link></literal>, <literal><link linkend="deanimate-gifs">deanimate-gifs</link></literal>
- and <literal><link linkend="kill-popups">kill-popups</link></literal> actions to work,
- <application>Privoxy</application> needs access to the uncompressed data.
- Unfortunately, <application>Privoxy</application> can't yet(!) uncompress, filter, and
- re-compress the content on the fly. So if you want to ensure that all websites, including
- those that normally compress, can be filtered, you need to use this action.
+ and <literal><link linkend="kill-popups">kill-popups</link></literal> actions need
+ access to the uncompressed data.
+ </para>
+ <para>
+ When compiled with zlib support (available since &my-app; 3.0.7), content that should be
+ filtered is decompressed on-the-fly and you don't have to worry about this action.
+ If you are using an older &my-app; version, or one that hasn't been compiled with zlib
+ support, this action can be used to convince the server to send the content uncompressed.
+ </para>
+ <para>
+ Most text-based instances compress very well, the size is seldom decreased by less than 50%,
+ for markup-heavy instances like news feeds saving more than 90% of the original size isn't
+ unusual.
</para>
<para>
- This will slow down transfers from those websites, though. If you use any of the above-mentioned
- actions, you will typically want to use <literal>prevent-compression</literal> in conjunction
- with them.
+ Not using compression will therefore slow down the transfer, and you should only
+ enable this action if you really need it. As of &my-app; 3.0.7 it's disabled in all
+ predefined action settings.
</para>
<para>
Note that some (rare) ill-configured sites don't handle requests for uncompressed
- documents correctly (they send an empty document body). If you use <literal>prevent-compression</literal>
- per default, you'll have to add exceptions for those sites. See the example for how to do that.
+ documents correctly. Broken PHP applications tend to send an empty document body,
+ some IIS versions only send the beginning of the content. If you enable
+ <literal>prevent-compression</literal> per default, you might want to add
+ exceptions for those sites. See the example for how to do that.
</para>
</listitem>
</varlistentry>
<term>Example usage (sections):</term>
<listitem>
<para>
- <screen># Set default:
+ <screen>
+# Selectively turn off compression, and enable a filter
+#
+{ +filter{tiny-textforms} +prevent-compression }
+# Match only these sites
+ .google.
+ sourceforge.net
+ sf.net
+
+# Or instead, we could set a universal default:
#
-{+prevent-compression}
-/ # Match all sites
+{ +prevent-compression }
+ / # Match all sites
-# Make exceptions for ill sites:
+# Then maybe make exceptions for broken sites:
#
-{-prevent-compression}
-www.debianhelp.org
-www.pclinuxonline.com</screen>
+{ -prevent-compression }
+.compusa.com/</screen>
</para>
</listitem>
</varlistentry>
<listitem>
<para>
<screen># Let the browser revalidate without being tracked across sessions
-{+hide-if-modified-since {-60}\
-+overwrite-last-modified {randomize}\
-+crunch-if-none-match}
+{ +hide-if-modified-since{-60} \
+ +overwrite-last-modified{randomize} \
+ +crunch-if-none-match}
/</screen>
</para>
</listitem>
<term>Parameter:</term>
<listitem>
<para>
- Any URL.
+ An absolute URL or a single pcrs command.
</para>
</listitem>
</varlistentry>
<term>Notes:</term>
<listitem>
<para>
- This action is useful to replace whole documents with ones of your
- choosing. This can be used to enforce safe surfing, or just as a simple
- convenience.
- </para>
- <para>
- You can do the same by combining the actions
- <literal><link linkend="block">block</link></literal>,
- <literal><link linkend="handle-as-image">handle-as-image</link></literal> and
- <literal><link linkend="set-image-blocker">set-image-blocker{URL}</link></literal>.
- It doesn't sound right for non-image documents, and that's why this action
- was created.
+ Requests to which this action applies are answered with a
+ HTTP redirect to URLs of your choosing. The new URL is
+ either provided as parameter, or derived by applying a
+ single pcrs command to the original URL.
</para>
<para>
This action will be ignored if you use it together with
<literal><link linkend="block">block</link></literal>.
+ It can be combined with
+ <literal><link linkend="fast-redirects">fast-redirects{check-decoded-url}</link></literal>
+ to redirect to a decoded version of a rewritten URL.
+ </para>
+ <para>
+ Use this action carefully, make sure not to create redirection loops
+ and be aware that using your own redirects might make it
+ possible to fingerprint your requests.
</para>
</listitem>
</varlistentry>
<para>
<screen># Replace example.com's style sheet with another one
{ +redirect{http://localhost/css-replacements/example.com.css} }
- example.com/stylesheet.css
+ example.com/stylesheet\.css
# Create a short, easy to remember nickname for a favorite site
+# (relies on the browser accept and forward invalid URLs to &my-app;)
{ +redirect{http://www.privoxy.org/user-manual/actions-file.html} }
- a</screen>
+ a
+
+# Always use the expanded view for Undeadly.org articles
+# (Note the $ at the end of the URL pattern to make sure
+# the request for the rewritten URL isn't redirected as well)
+{+redirect{s@$@&mode=expanded@}}
+undeadly.org/cgi\?action=article&sid=\d*$</screen>
</para>
</listitem>
</varlistentry>
</sect3>
+<!-- ~~~~~ New section ~~~~~ -->
+<sect3 renderas="sect4" id="server-header-filter">
+<title>server-header-filter</title>
+
+<variablelist>
+ <varlistentry>
+ <term>Typical use:</term>
+ <listitem>
+ <para>
+ Rewrite or remove single server headers.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Effect:</term>
+ <listitem>
+ <para>
+ All server headers to which this action applies are filtered on-the-fly
+ through the specified regular expression based substitutions.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type:</term>
+ <!-- boolean, parameterized, Multi-value -->
+ <listitem>
+ <para>Parameterized.</para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Parameter:</term>
+ <listitem>
+ <para>
+ The name of a server-header filter, as defined in one of the
+ <link linkend="filter-file">filter files</link>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Notes:</term>
+ <listitem>
+ <para>
+ Server-header filters are applied to each header on its own, not to
+ all at once. This makes it easier to diagnose problems, but on the downside
+ you can't write filters that only change header x if header y's value is z.
+ You can do that by using tags though.
+ </para>
+ <para>
+ Server-header filters are executed after the other header actions have finished
+ and use their output as input.
+ </para>
+ <para>
+ Please refer to the <link linkend="filter-file">filter file chapter</link>
+ to learn which server-header filters are available by default, and how to
+ create your own.
+ </para>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Example usage (section):</term>
+ <listitem>
+ <para>
+ <screen>
+{+server-header-filter{html-to-xml}}
+example.org/xml-instance-that-is-delivered-as-html
+
+{+server-header-filter{xml-to-html}}
+example.org/instance-that-is-delivered-as-xml-but-is-not
+ </screen>
+ </para>
+ </listitem>
+ </varlistentry>
+
+</variablelist>
+</sect3>
+
+
+<!-- ~~~~~ New section ~~~~~ -->
+<sect3 renderas="sect4" id="server-header-tagger">
+<title>server-header-tagger</title>
+
+<variablelist>
+ <varlistentry>
+ <term>Typical use:</term>
+ <listitem>
+ <para>
+ Disable or disable filters based on the Content-Type header.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Effect:</term>
+ <listitem>
+ <para>
+ Server headers to which this action applies are filtered on-the-fly through
+ the specified regular expression based substitutions, the result is used as
+ tag.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type:</term>
+ <!-- boolean, parameterized, Multi-value -->
+ <listitem>
+ <para>Parameterized.</para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Parameter:</term>
+ <listitem>
+ <para>
+ The name of a server-header tagger, as defined in one of the
+ <link linkend="filter-file">filter files</link>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Notes:</term>
+ <listitem>
+ <para>
+ Server-header taggers are applied to each header on its own,
+ and as the header isn't modified, each tagger <quote>sees</quote>
+ the original.
+ </para>
+ <para>
+ Server-header taggers are executed before all other header actions
+ that modify server headers. Their tags can be used to control
+ all of the other server-header actions, the content filters
+ and the crunch actions (<link linkend="redirect">redirect</link>
+ and <link linkend="block">block</link>).
+ </para>
+ <para>
+ Obviously crunching based on tags created by server-header taggers
+ doesn't prevent the request from showing up in the server's log file.
+ </para>
+
+ </varlistentry>
+
+ <varlistentry>
+ <term>Example usage (section):</term>
+ <listitem>
+ <para>
+ <screen>
+# Tag every request with the declared content type
+{+client-header-filter{content-type}}
+/
+ </screen>
+ </para>
+ </listitem>
+ </varlistentry>
+
+</variablelist>
+</sect3>
+
+
<!-- ~~~~~ New section ~~~~~ -->
<sect3 renderas="sect4" id="session-cookies-only">
<title>session-cookies-only</title>
<screen>+set-image-blocker{pattern}</screen>
</para>
<para>
- Redirect to the BSD devil:
+ Redirect to the BSD daemon:
</para>
<para>
<screen>+set-image-blocker{http://www.freebsd.org/gifs/dae_up3.gif}</screen>
For <quote>Connect</quote> requests the clients tell
<application>Privoxy</application> which host they are interested
in, but not which document they plan to get later. As a result, the
- <quote>Go there anyway</quote> link becomes rather useless:
- it lets the client request the home page of the forbidden host
- through unencrypted HTTP, still using the port of the last request.
- </para>
- <para>
- If you previously configured <application>Privoxy</application> to do the
- request through a SSL tunnel, everything will work. Most likely you haven't
- and the server will respond with an error message because it is expecting
- HTTPS (SSL).
+ <quote>Go there anyway</quote> wouldn't work and is therefore suppressed.
</para>
</listitem>
</varlistentry>
them before writing. So the effects of your aliases are of course preserved,
but the aliases themselves are lost when you edit sections that use aliases
with it.
- This is likely to change in future versions of <application>Privoxy</application>.
</para>
<para>
+crunch-all-cookies = +<link linkend="CRUNCH-INCOMING-COOKIES">crunch-incoming-cookies</link> +<link linkend="CRUNCH-OUTGOING-COOKIES">crunch-outgoing-cookies</link>
-crunch-all-cookies = -<link linkend="CRUNCH-INCOMING-COOKIES">crunch-incoming-cookies</link> -<link linkend="CRUNCH-OUTGOING-COOKIES">crunch-outgoing-cookies</link>
+block-as-image = +block +handle-as-image
- mercy-for-cookies = -crunch-all-cookies -<link linkend="SESSION-COOKIES-ONLY">session-cookies-only</link> -<link linkend="FILTER-CONTENT-COOKIES">filter{content-cookies}</link>
+ allow-all-cookies = -crunch-all-cookies -<link linkend="SESSION-COOKIES-ONLY">session-cookies-only</link> -<link linkend="FILTER-CONTENT-COOKIES">filter{content-cookies}</link>
# These aliases define combinations of actions
# that are useful for certain types of sites:
#
- fragile = -<link linkend="BLOCK">block</link> -<link linkend="FILTER">filter</link> -crunch-all-cookies -<link linkend="FAST-REDIRECTS">fast-redirects</link> -<link linkend="HIDE-REFERER">hide-referrer</link> -<link linkend="KILL-POPUPS">kill-popups</link>
+ fragile = -<link linkend="BLOCK">block</link> -<link linkend="FILTER">filter</link> -crunch-all-cookies -<link linkend="FAST-REDIRECTS">fast-redirects</link> -<link linkend="HIDE-REFERER">hide-referrer</link> -<link linkend="KILL-POPUPS">kill-popups</link> -<link linkend="PREVENT-COMPRESSION">prevent-compression</link>
+
shop = -crunch-all-cookies -<link linkend="FILTER-ALL-POPUPS">filter{all-popups}</link> -<link linkend="KILL-POPUPS">kill-popups</link>
# Short names for other aliases, for really lazy people ;-)
{fragile}
.office.microsoft.com
.windowsupdate.microsoft.com
- .nytimes.com
+ # Gmail is really mail.google.com, not gmail.com
+ mail.google.com
# Shopping sites:
# Allow cookies (for setting and retrieving your customer data)
{shop}
.quietpc.com
.worldpay.com # for quietpc.com
- .scan.co.uk
+ mybank.example.com
# These shops require pop-ups:
#
- {shop -kill-popups -filter{all-popups}}
+ {-kill-popups -filter{all-popups} -filter{unsolicited-popups}}
.dabs.com
.overclockers.co.uk</screen>
</para>
<para>
- Aliases like <quote>shop</quote> and <quote>fragile</quote> are often used for
- <quote>problem</quote> sites that require some actions to be disabled
+ Aliases like <quote>shop</quote> and <quote>fragile</quote> are typically used for
+ <quote>problem</quote> sites that require more than one action to be disabled
in order to function properly.
</para>
</sect2>
##########################################################################
{ \
-<link linkend="ADD-HEADER">add-header</link> \
+ -<link linkend="CLIENT-HEADER-FILTER">client-header-filter{hide-tor-exit-notation}</link> \
-<link linkend="BLOCK">block</link> \
-<link linkend="CONTENT-TYPE-OVERWRITE">content-type-overwrite</link> \
-<link linkend="CRUNCH-CLIENT-HEADER">crunch-client-header</link> \
-<link linkend="FILTER-FUN">filter{fun}</link> \
-<link linkend="FILTER-CRUDE-PARENTAL">filter{crude-parental}</link> \
+<link linkend="FILTER-IE-EXPLOITS">filter{ie-exploits}</link> \
- -<link linkend="FILTER-CLIENT-HEADERS">filter-client-headers</link> \
- -<link linkend="FILTER-SERVER-HEADERS">filter-server-headers</link> \
- -<link linkend="FILTER-GOOGLE">filter-google</link> \
- -<link linkend="FILTER-YAHOO">filter-yahoo</link> \
- -<link linkend="FILTER-MSN">filter-msn</link> \
- -<link linkend="FILTER-BLOGSPOT">filter-blogspot</link> \
- -<link linkend="FILTER-XML-TO-HTML">filter-xml-to-html</link> \
- -<link linkend="FILTER-HTML-TO-XML">filter-html-to-xml</link> \
+ -<link linkend="FILTER-GOOGLE">filter{google}</link> \
+ -<link linkend="FILTER-YAHOO">filter{yahoo}</link> \
+ -<link linkend="FILTER-MSN">filter{msn}</link> \
+ -<link linkend="FILTER-BLOGSPOT">filter{blogspot}</link> \
+ -<link linkend="FILTER-NO-PING">filter{no-ping}</link> \
-<link linkend="FORCE-TEXT-MODE">force-text-mode</link> \
-<link linkend="HANDLE-AS-EMPTY-DOCUMENT">handle-as-empty-document</link> \
-<link linkend="HANDLE-AS-IMAGE">handle-as-image</link> \
-<link linkend="REDIRECT">redirect</link> \
-<link linkend="SEND-VANILLA-WAFER">send-vanilla-wafer</link> \
-<link linkend="SEND-WAFER">send-wafer</link> \
+ -<link linkend="SERVER-HEADER-FILTER">server-header-filter{xml-to-html}</link> \
+ -<link linkend="SERVER-HEADER-FILTER">server-header-filter{html-to-xml}</link> \
+<link linkend="SESSION-COOKIES-ONLY">session-cookies-only</link> \
+<link linkend="SET-IMAGE-BLOCKER">set-image-blocker{pattern}</link> \
-<link linkend="TREAT-FORBIDDEN-CONNECTS-LIKE-BLOCKS">treat-forbidden-connects-like-blocks</link> \
<para>
<screen>
{ +<link linkend="BLOCK">block</link> }
- www.example.com/nasty-ads/sponsor.gif
+ www.example.com/nasty-ads/sponsor\.gif
another.popular.site.net/more/junk/here/</screen>
</para>
<title>Filter Files</title>
<para>
- On-the-fly text substitutions that can be invoked through the
- <literal><link linkend="filter">filter</link></literal> action need
+ On-the-fly text substitutions need
to be defined in a <quote>filter file</quote>. Once defined, they
- can then be invoked as an <quote>action</quote>. Multiple filter files can be
- defined through the <literal> <link
+ can then be invoked as an <quote>action</quote>.
+</para>
+
+<para>
+ &my-app; supports three different filter actions:
+ <literal><link linkend="filter">filter</link></literal> to
+ rewrite the content that is send to the client,
+ <literal><link linkend="client-header-filter">client-header-filter</link></literal>
+ to rewrite headers that are send by the client, and
+ <literal><link linkend="server-header-filter">server-header-filter</link></literal>
+ to rewrite headers that are send by the server, and
+</para>
+
+<para>
+ &my-app; also supports two tagger actions:
+ <literal><link linkend="client-header-tagger">client-header-tagger</link></literal>
+ and
+ <literal><link linkend="server-header-tagger">server-header-tagger</link></literal>.
+ Taggers and filters use the same syntax in the filter files, the differnce
+ is that taggers don't modify the text they are filtering, but use a rewritten
+ version of the filtered text as tag. The tags can then be used to change the
+ applying actions through sections with <link linkend="tag-pattern">tag-patterns</link>.
+</para>
+
+
+<para>
+ Multiple filter files can be defined through the <literal> <link
linkend="filterfile">filterfile</link></literal> config directive. The filters
as supplied by the developers will be found in
<filename>default.filter</filename>. It is recommended that any locally
</para>
<para>
- Typical reasons for doing these kinds of substitutions are to eliminate
- common annoyances in HTML and JavaScript, such as pop-up windows,
+ Command tasks for content filters are to eliminate common annoyances in
+ HTML and JavaScript, such as pop-up windows,
exit consoles, crippled windows without navigation tools, the
infamous <BLINK> tag etc, to suppress images with certain
width and height attributes (standard banner sizes or web-bugs),
- or just to have fun. The possibilities are endless.
+ or just to have fun.
</para>
<para>
- Filtering works on any text-based document type, including
+ Content filtering works on any text-based document type, including
HTML, JavaScript, CSS etc. (all <literal>text/*</literal>
MIME types, <emphasis>except</emphasis> <literal>text/plain</literal>).
Substitutions are made at the source level, so if you want to <quote>roll
your own</quote> filters, you should first be familiar with HTML syntax,
- and, of course, regular expressions. By default, filters are only applied
- to the raw document content, but can be extended to the HTTP headers with
- the supplemental actions:
- <link linkend="filter-client-headers">filter-client-headers</link> and
- <link linkend="filter-server-headers">filter-server-headers</link>.
+ and, of course, regular expressions.
</para>
<para>
Just like the <link linkend="actions-file">actions files</link>, the
filter file is organized in sections, which are called <emphasis>filters</emphasis>
- here. Each filter consists of a heading line, that starts with the
- <emphasis>keyword</emphasis> <literal>FILTER:</literal>, followed by
- the filter's <emphasis>name</emphasis>, and a short (one line)
+ here. Each filter consists of a heading line, that starts with one of the
+ <emphasis>keywords</emphasis> <literal>FILTER:</literal>,
+ <literal>CLIENT-HEADER-FILTER:</literal> or <literal>SERVER-HEADER-FILTER:</literal>
+ followed by the filter's <emphasis>name</emphasis>, and a short (one line)
<emphasis>description</emphasis> of what it does. Below that line
come the <emphasis>jobs</emphasis>, i.e. lines that define the actual
text substitutions. By convention, the name of a filter
</para>
<para>
- A filter header line for a filter called <quote>foo</quote> could look
+ Filter definitions start with a header line that contains the filter
+ type, the filter name and the filter description.
+ A content filter header line for a filter called <quote>foo</quote> could look
like this:
</para>
<sect2><title>Filter File Tutorial</title>
<para>
- Now, let's complete our <quote>foo</quote> filter. We have already defined
+ Now, let's complete our <quote>foo</quote> content filter. We have already defined
the heading, but the jobs are still missing. Since all it does is to replace
<quote>foo</quote> with <quote>bar</quote>, there is only one (trivial) job
needed:
<term><emphasis>xml-to-html</emphasis></term>
<listitem>
<para>
- Header filter to change the Content-Type from xml to html.
+ Server-header filter to change the Content-Type from xml to html.
</para>
</listitem>
</varlistentry>
+
<varlistentry>
<term><emphasis>html-to-xml</emphasis></term>
<listitem>
<para>
- Header filter to change the Content-Type from html to xml.
+ Server-header filter to change the Content-Type from html to xml.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><emphasis>no-ping</emphasis></term>
+ <listitem>
+ <para>
+ Removes the non-standard <literal>ping</literal> attribute from
+ anchor and area HTML tags.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><emphasis>hide-tor-exit-notation</emphasis></term>
+ <listitem>
+ <para>
+ Client-header filter to remove the <command>Tor</command> exit node notation
+ found in Host and Referer headers.
+ </para>
+ <para>
+ If &my-app; and <command>Tor</command> are chained and &my-app;
+ is configured to use socks4a, one can use <quote>http://www.example.org.foobar.exit/</quote>
+ to access the host <quote>www.example.org</quote> through the
+ <command>Tor</command> exit node <quote>foobar</quote>.
+ </para>
+ <para>
+ As the HTTP client isn't aware of this notation, it treats the
+ whole string <quote>www.example.org.foobar.exit</quote> as host and uses it
+ for the <quote>Host</quote> and <quote>Referer</quote> headers. From the
+ server's point of view the resulting headers are invalid and can cause problems.
+ </para>
+ <para>
+ An invalid <quote>Referer</quote> header can trigger <quote>hot-linking</quote>
+ protections, an invalid <quote>Host</quote> header will make it impossible for
+ the server to find the right vhost (several domains hosted on the same IP address).
+ </para>
+ <para>
+ This client-header filter removes the <quote>foo.exit</quote> part in those headers
+ to prevent the mentioned problems. Note that it only modifies
+ the HTTP headers, it doesn't make it impossible for the server
+ to detect your <command>Tor</command> exit node based on the IP address
+ the request is coming from.
</para>
</listitem>
</varlistentry>
<sect2 id="chain">
<title>Chain of Events</title>
<para>
- Let's take a quick look at the basic sequence of events when a web page is
- requested by your browser and <application>Privoxy</application> is on duty:
+ Let's take a quick look at how some of <application>Privoxy's</application>
+ core features are triggered, and the ensuing sequence of events when a web
+ page is requested by your browser:
</para>
<para>
linkend="BLOCK"><quote>+block</quote></link> patterns. If
so, the URL is then blocked, and the remote web server will not be contacted.
<link linkend="HANDLE-AS-IMAGE"><quote>+handle-as-image</quote></link>
- is then checked and if it does not match, an
- HTML <quote>BLOCKED</quote> page is sent back. Otherwise, if it does match,
- an image is returned. The type of image depends on the setting of <link
- linkend="SET-IMAGE-BLOCKER"><quote>+set-image-blocker</quote></link>
+ and
+ <link linkend="HANDLE-AS-EMPTY-DOCUMENT"><quote>+handle-as-empty-document</quote></link>
+ are then checked, and if there is no match, an
+ HTML <quote>BLOCKED</quote> page is sent back to the browser. Otherwise, if
+ it does match, an image is returned for the former, and an empty text
+ document for the latter. The type of image would depend on the setting of
+ <link linkend="SET-IMAGE-BLOCKER"><quote>+set-image-blocker</quote></link>
(blank, checkerboard pattern, or an HTTP redirect to an image elsewhere).
</para>
</listitem>
</listitem>
<listitem>
<para>
- Now the web server starts sending its response back (i.e. typically a web page and related
- data).
+ Now the web server starts sending its response back (i.e. typically a web
+ page).
</para>
</listitem>
<listitem>
</listitem>
<listitem>
<para>
- If a <link linkend="FILTER"><quote>+filter</quote></link>
+ If any <link linkend="FILTER"><quote>+filter</quote></link> action
or <link
linkend="DEANIMATE-GIFS"><quote>+deanimate-gifs</quote></link>
action applies (and the document type fits the action), the rest of the page is
<application>Privoxy</application> back to your browser.
</para>
<para>
- If neither <link linkend="FILTER"><quote>+filter</quote></link>
+ If neither a <link linkend="FILTER"><quote>+filter</quote></link> action
or <link
linkend="DEANIMATE-GIFS"><quote>+deanimate-gifs</quote></link>
matches, then <application>Privoxy</application> passes the raw data through
As the browser receives the now (possibly filtered) page content, it
reads and then requests any URLs that may be embedded within the page
source, e.g. ad images, stylesheets, JavaScript, other HTML documents (e.g.
- frames), sounds, etc. For each of these objects, the browser issues a new
- request. And each such request is in turn processed as above. Note that a
- complex web page may have many such embedded URLs.
+ frames), sounds, etc. For each of these objects, the browser issues a
+ separate request (this is easily viewable in <application>Privoxy's</application>
+ logs). And each such request is in turn processed just as above. Note that a
+ complex web page will have many, many such embedded URLs. If these
+ secondary requests are to a different server, then quite possibly a very
+ differing set of actions is triggered.
</para>
</listitem>
</itemizedlist>
</para>
+<para>
+ NOTE: This is somewhat of a simplistic overview of what happens with each URL
+ request. For the sake of brevity and simplicity, we have focused on
+ <application>Privoxy's</application> core features only.
+</para>
</sect2>
{-add-header
-block
+ -client-header-filter{hide-tor-exit-notation}
-content-type-overwrite
-crunch-client-header
-crunch-if-none-match
-filter {yahoo}
-filter {msn}
-filter {blogspot}
- -filter {xml-to-html}
- -filter {html-to-xml}
- -filter-client-headers
- -filter-server-headers
+ -filter {no-ping}
-force-text-mode
-handle-as-empty-document
-handle-as-image
-redirect
-send-vanilla-wafer
-send-wafer
+ -server-header-filter{xml-to-html}
+ -server-header-filter{html-to-xml}
+session-cookies-only
+set-image-blocker {pattern}
-treat-forbidden-connects-like-blocks }
-add-header
-block
+ -client-header-filter{hide-tor-exit-notation}
-content-type-overwrite
-crunch-client-header
-crunch-if-none-match
-filter {yahoo}
-filter {msn}
-filter {blogspot}
- -filter {xml-to-html}
- -filter {html-to-xml}
- -filter-client-headers
- -filter-server-headers
+ -filter {no-ping}
-force-text-mode
-handle-as-empty-document
-handle-as-image
-redirect
-send-vanilla-wafer
-send-wafer
+ -server-header-filter{xml-to-html}
+ -server-header-filter{html-to-xml}
-session-cookies-only
+set-image-blocker {pattern}
-treat-forbidden-connects-like-blocks </screen>
{-add-header
-block
+ -client-header-filter{hide-tor-exit-notation}
-content-type-overwrite
-crunch-client-header
-crunch-if-none-match
-filter {yahoo}
-filter {msn}
-filter {blogspot}
- -filter {xml-to-html}
- -filter {html-to-xml}
- -filter-client-headers
- -filter-server-headers
+ -filter {no-ping}
-force-text-mode
-handle-as-empty-document
-handle-as-image
+prevent-compression
-redirect
-send-vanilla-wafer
- -send-wafer
+ -send-wafer
+ -server-header-filter{xml-to-html}
+ -server-header-filter{html-to-xml}
+session-cookies-only
+set-image-blocker{blank}
-treat-forbidden-connects-like-blocks }
put in <filename>user.action</filename>, for local site
exceptions. Note that when a simple domain pattern is used by itself (without
the subsequent path portion), all sub-pages within that domain are included
- automatcially in the scope of the action.
+ automatically in the scope of the action.
</para>
<para>
USA
$Log: user-manual.sgml,v $
+ Revision 2.31 2007/06/02 14:01:37 fabiankeil
+ Start to document forward-override{}.
+
+ Revision 2.30 2007/04/25 15:10:36 fabiankeil
+ - Describe installation for FreeBSD.
+ - Start to document taggers and tag patterns.
+ - Don't confuse devils and daemons.
+
+ Revision 2.29 2007/04/05 11:47:51 fabiankeil
+ Some updates regarding header filtering,
+ handling of compressed content and redirect's
+ support for pcrs commands.
+
+ Revision 2.28 2006/12/10 23:42:48 hal9
+ Fix various typos reported by Adam P. Thanks.
+
+ Revision 2.27 2006/11/14 01:57:47 hal9
+ Dump all docs prior to 3.0.6 release. Various minor changes to faq and user
+ manual.
+
+ Revision 2.26 2006/10/24 11:16:44 hal9
+ Add new filters.
+
+ Revision 2.25 2006/10/18 10:50:33 hal9
+ Add note that since filters are off in Cautious, compression is ON. Turn off
+ compression to make filters work on all sites.
+
+ Revision 2.24 2006/10/03 11:13:54 hal9
+ More references to the new filters. Include html this time around.
+
Revision 2.23 2006/10/02 22:43:53 hal9
Contains new filter definitions from Fabian, and few other miscellaneous
touch-ups.
projects / privoxy.git / blobdiff