-<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01
-Transitional//EN""http://www.w3.org/TR/html4/loose.dtd">
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
+"http://www.w3.org/TR/html4/loose.dtd">
<html>
- <head>
- <meta name="generator" content="HTML Tidy, see www.w3.org">
- <title>
- Filter Files
- </title>
- <meta name="GENERATOR" content=
- "Modular DocBook HTML Stylesheet Version 1.79">
- <link rel="HOME" title="Privoxy 3.0.18 User Manual" href="index.html">
- <link rel="PREVIOUS" title="Actions Files" href="actions-file.html">
- <link rel="NEXT" title="Privoxy's Template Files" href="templates.html">
- <link rel="STYLESHEET" type="text/css" href="../p_doc.css">
- <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
- <link rel="STYLESHEET" type="text/css" href="p_doc.css">
-<style type="text/css">
- body {
- background-color: #EEEEEE;
- color: #000000;
- }
- :link { color: #0000FF }
- :visited { color: #840084 }
- :active { color: #0000FF }
- hr.c1 {text-align: left}
-</style>
- </head>
- <body class="SECT1">
- <div class="NAVHEADER">
- <table summary="Header navigation table" width="100%" border="0"
- cellpadding="0" cellspacing="0">
+<head>
+ <title>Filter Files</title>
+ <meta name="GENERATOR" content="Modular DocBook HTML Stylesheet Version 1.79">
+ <link rel="HOME" title="Privoxy 3.0.29 User Manual" href="index.html">
+ <link rel="PREVIOUS" title="Actions Files" href="actions-file.html">
+ <link rel="NEXT" title="Privoxy's Template Files" href="templates.html">
+ <link rel="STYLESHEET" type="text/css" href="../p_doc.css">
+ <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
+ <link rel="STYLESHEET" type="text/css" href="p_doc.css">
+</head>
+<body class="SECT1" bgcolor="#EEEEEE" text="#000000" link="#0000FF" vlink="#840084" alink="#0000FF">
+ <div class="NAVHEADER">
+ <table summary="Header navigation table" width="100%" border="0" cellpadding="0" cellspacing="0">
+ <tr>
+ <th colspan="3" align="center">Privoxy 3.0.29 User Manual</th>
+ </tr>
+ <tr>
+ <td width="10%" align="left" valign="bottom"><a href="actions-file.html" accesskey="P">Prev</a></td>
+ <td width="80%" align="center" valign="bottom"></td>
+ <td width="10%" align="right" valign="bottom"><a href="templates.html" accesskey="N">Next</a></td>
+ </tr>
+ </table>
+ <hr align="left" width="100%">
+ </div>
+ <div class="SECT1">
+ <h1 class="SECT1"><a name="FILTER-FILE" id="FILTER-FILE">9. Filter Files</a></h1>
+ <p>On-the-fly text substitutions need to be defined in a <span class="QUOTE">"filter file"</span>. Once defined,
+ they can then be invoked as an <span class="QUOTE">"action"</span>.</p>
+ <p><span class="APPLICATION">Privoxy</span> supports three different pcrs-based filter actions: <tt class=
+ "LITERAL"><a href="actions-file.html#FILTER">filter</a></tt> to rewrite the content that is send to the client,
+ <tt class="LITERAL"><a href="actions-file.html#CLIENT-HEADER-FILTER">client-header-filter</a></tt> to rewrite
+ headers that are send by the client, and <tt class="LITERAL"><a href=
+ "actions-file.html#SERVER-HEADER-FILTER">server-header-filter</a></tt> to rewrite headers that are send by the
+ server.</p>
+ <p><span class="APPLICATION">Privoxy</span> also supports two tagger actions: <tt class="LITERAL"><a href=
+ "actions-file.html#CLIENT-HEADER-TAGGER">client-header-tagger</a></tt> and <tt class="LITERAL"><a href=
+ "actions-file.html#SERVER-HEADER-TAGGER">server-header-tagger</a></tt>. Taggers and filters use the same syntax in
+ the filter files, the difference is that taggers don't modify the text they are filtering, but use a rewritten
+ version of the filtered text as tag. The tags can then be used to change the applying actions through sections with
+ <a href="actions-file.html#TAG-PATTERN">tag-patterns</a>.</p>
+ <p>Finally <span class="APPLICATION">Privoxy</span> supports the <tt class="LITERAL"><a href=
+ "actions-file.html#EXTERNAL-FILTER">external-filter</a></tt> action to enable <tt class="LITERAL"><a href=
+ "filter-file.html#EXTERNAL-FILTER-SYNTAX">external filters</a></tt> written in proper programming languages.</p>
+ <p>Multiple filter files can be defined through the <tt class="LITERAL"><a href=
+ "config.html#FILTERFILE">filterfile</a></tt> config directive. The filters as supplied by the developers are
+ located in <tt class="FILENAME">default.filter</tt>. It is recommended that any locally defined or modified filters
+ go in a separately defined file such as <tt class="FILENAME">user.filter</tt>.</p>
+ <p>Common tasks for content filters are to eliminate common annoyances in HTML and JavaScript, such as pop-up
+ windows, exit consoles, crippled windows without navigation tools, the infamous <BLINK> tag etc, to suppress
+ images with certain width and height attributes (standard banner sizes or web-bugs), or just to have fun.</p>
+ <p>Enabled content filters are applied to any content whose <span class="QUOTE">"Content Type"</span> header is
+ recognised as a sign of text-based content, with the exception of <tt class="LITERAL">text/plain</tt>. Use the
+ <a href="actions-file.html#FORCE-TEXT-MODE">force-text-mode</a> action to also filter other content.</p>
+ <p>Substitutions are made at the source level, so if you want to <span class="QUOTE">"roll your own"</span>
+ filters, you should first be familiar with HTML syntax, and, of course, regular expressions.</p>
+ <p>Just like the <a href="actions-file.html">actions files</a>, the filter file is organized in sections, which are
+ called <span class="emphasis"><i class="EMPHASIS">filters</i></span> here. Each filter consists of a heading line,
+ that starts with one of the <span class="emphasis"><i class="EMPHASIS">keywords</i></span> <tt class=
+ "LITERAL">FILTER:</tt>, <tt class="LITERAL">CLIENT-HEADER-FILTER:</tt> or <tt class=
+ "LITERAL">SERVER-HEADER-FILTER:</tt> followed by the filter's <span class="emphasis"><i class=
+ "EMPHASIS">name</i></span>, and a short (one line) <span class="emphasis"><i class=
+ "EMPHASIS">description</i></span> of what it does. Below that line come the <span class="emphasis"><i class=
+ "EMPHASIS">jobs</i></span>, i.e. lines that define the actual text substitutions. By convention, the name of a
+ filter should describe what the filter <span class="emphasis"><i class="EMPHASIS">eliminates</i></span>. The
+ comment is used in the <a href="http://config.privoxy.org/" target="_top">web-based user interface</a>.</p>
+ <p>Once a filter called <tt class="REPLACEABLE"><i>name</i></tt> has been defined in the filter file, it can be
+ invoked by using an action of the form +<tt class="LITERAL"><a href=
+ "actions-file.html#FILTER">filter</a>{<tt class="REPLACEABLE"><i>name</i></tt>}</tt> in any <a href=
+ "actions-file.html">actions file</a>.</p>
+ <p>Filter definitions start with a header line that contains the filter type, the filter name and the filter
+ description. A content filter header line for a filter called <span class="QUOTE">"foo"</span> could look like
+ this:</p>
+ <table border="0" bgcolor="#E0E0E0" width="100%">
+ <tr>
+ <td>
+ <pre class="SCREEN">FILTER: foo Replace all "foo" with "bar"</pre>
+ </td>
+ </tr>
+ </table>
+ <p>Below that line, and up to the next header line, come the jobs that define what text replacements the filter
+ executes. They are specified in a syntax that imitates <a href="http://www.perl.org/" target="_top">Perl</a>'s
+ <tt class="LITERAL">s///</tt> operator. If you are familiar with Perl, you will find this to be quite intuitive,
+ and may want to look at the PCRS documentation for the subtle differences to Perl behaviour.</p>
+ <p>Most notably, the non-standard option letter <tt class="LITERAL">U</tt> is supported, which turns the default to
+ ungreedy matching (add <tt class="LITERAL">?</tt> to quantifiers to turn them greedy again).</p>
+ <p>The non-standard option letter <tt class="LITERAL">D</tt> (dynamic) allows to use the variables $host, $origin
+ (the IP address the request came from), $path, $url and $listen-address (the address on which Privoxy accepted the
+ client request. Example: 127.0.0.1:8118). They will be replaced with the value they refer to before the filter is
+ executed.</p>
+ <p>Note that '$' is a bad choice for a delimiter in a dynamic filter as you might end up with unintended variables
+ if you use a variable name directly after the delimiter. Variables will be resolved without escaping anything,
+ therefore you also have to be careful not to chose delimiters that appear in the replacement text. For example
+ '<' should be save, while '?' will sooner or later cause conflicts with $url.</p>
+ <p>The non-standard option letter <tt class="LITERAL">T</tt> (trivial) prevents parsing for backreferences in the
+ substitute. Use it if you want to include text like '$&' in your substitute without quoting.</p>
+ <p>If you are new to <a href="http://en.wikipedia.org/wiki/Regular_expressions" target="_top"><span class=
+ "QUOTE">"Regular Expressions"</span></a>, you might want to take a look at the <a href=
+ "appendix.html#REGEX">Appendix on regular expressions</a>, and see the <a href=
+ "http://perldoc.perl.org/perlre.html" target="_top">Perl manual</a> for <a href=
+ "http://perldoc.perl.org/perlop.html" target="_top">the <tt class="LITERAL">s///</tt> operator's syntax</a> and
+ <a href="http://perldoc.perl.org/perlre.html" target="_top">Perl-style regular expressions</a> in general. The
+ below examples might also help to get you started.</p>
+ <div class="SECT2">
+ <h2 class="SECT2"><a name="FILTER-FILE-TUT" id="FILTER-FILE-TUT">9.1. Filter File Tutorial</a></h2>
+ <p>Now, let's complete our <span class="QUOTE">"foo"</span> content filter. We have already defined the heading,
+ but the jobs are still missing. Since all it does is to replace <span class="QUOTE">"foo"</span> with
+ <span class="QUOTE">"bar"</span>, there is only one (trivial) job needed:</p>
+ <table border="0" bgcolor="#E0E0E0" width="100%">
<tr>
- <th colspan="3" align="center">
- Privoxy 3.0.18 User Manual
- </th>
+ <td>
+ <pre class="SCREEN">s/foo/bar/</pre>
+ </td>
</tr>
+ </table>
+ <p>But wait! Didn't the comment say that <span class="emphasis"><i class="EMPHASIS">all</i></span> occurrences of
+ <span class="QUOTE">"foo"</span> should be replaced? Our current job will only take care of the first
+ <span class="QUOTE">"foo"</span> on each page. For global substitution, we'll need to add the <tt class=
+ "LITERAL">g</tt> option:</p>
+ <table border="0" bgcolor="#E0E0E0" width="100%">
<tr>
- <td width="10%" align="left" valign="bottom">
- <a href="actions-file.html" accesskey="P">Prev</a>
- </td>
- <td width="80%" align="center" valign="bottom">
- </td>
- <td width="10%" align="right" valign="bottom">
- <a href="templates.html" accesskey="N">Next</a>
+ <td>
+ <pre class="SCREEN">s/foo/bar/g</pre>
</td>
</tr>
</table>
- <hr width="100%" class="c1">
- </div>
- <div class="SECT1">
- <h1 class="SECT1">
- <a name="FILTER-FILE">9. Filter Files</a>
- </h1>
- <p>
- On-the-fly text substitutions need to be defined in a <span class=
- "QUOTE">"filter file"</span>. Once defined, they can then be invoked
- as an <span class="QUOTE">"action"</span>.
- </p>
- <p>
- <span class="APPLICATION">Privoxy</span> supports three different
- filter actions: <tt class="LITERAL"><a href=
- "actions-file.html#FILTER">filter</a></tt> to rewrite the content
- that is send to the client, <tt class="LITERAL"><a href=
- "actions-file.html#CLIENT-HEADER-FILTER">client-header-filter</a></tt>
- to rewrite headers that are send by the client, and <tt class=
- "LITERAL"><a href=
- "actions-file.html#SERVER-HEADER-FILTER">server-header-filter</a></tt>
- to rewrite headers that are send by the server.
- </p>
- <p>
- <span class="APPLICATION">Privoxy</span> also supports two tagger
- actions: <tt class="LITERAL"><a href=
- "actions-file.html#CLIENT-HEADER-TAGGER">client-header-tagger</a></tt>
- and <tt class="LITERAL"><a href=
- "actions-file.html#SERVER-HEADER-TAGGER">server-header-tagger</a></tt>.
- Taggers and filters use the same syntax in the filter files, the
- difference is that taggers don't modify the text they are filtering,
- but use a rewritten version of the filtered text as tag. The tags can
- then be used to change the applying actions through sections with <a
- href="actions-file.html#TAG-PATTERN">tag-patterns</a>.
- </p>
- <p>
- Multiple filter files can be defined through the <tt class=
- "LITERAL"><a href="config.html#FILTERFILE">filterfile</a></tt> config
- directive. The filters as supplied by the developers are located in
- <tt class="FILENAME">default.filter</tt>. It is recommended that any
- locally defined or modified filters go in a separately defined file
- such as <tt class="FILENAME">user.filter</tt>.
- </p>
- <p>
- Common tasks for content filters are to eliminate common annoyances
- in HTML and JavaScript, such as pop-up windows, exit consoles,
- crippled windows without navigation tools, the infamous <BLINK>
- tag etc, to suppress images with certain width and height attributes
- (standard banner sizes or web-bugs), or just to have fun.
- </p>
- <p>
- Enabled content filters are applied to any content whose <span class=
- "QUOTE">"Content Type"</span> header is recognised as a sign of
- text-based content, with the exception of <tt class=
- "LITERAL">text/plain</tt>. Use the <a href=
- "actions-file.html#FORCE-TEXT-MODE">force-text-mode</a> action to
- also filter other content.
- </p>
- <p>
- Substitutions are made at the source level, so if you want to <span
- class="QUOTE">"roll your own"</span> filters, you should first be
- familiar with HTML syntax, and, of course, regular expressions.
- </p>
- <p>
- Just like the <a href="actions-file.html">actions files</a>, the
- filter file is organized in sections, which are called <span class=
- "emphasis"><i class="EMPHASIS">filters</i></span> here. Each filter
- consists of a heading line, that starts with one of the <span class=
- "emphasis"><i class="EMPHASIS">keywords</i></span> <tt class=
- "LITERAL">FILTER:</tt>, <tt class=
- "LITERAL">CLIENT-HEADER-FILTER:</tt> or <tt class=
- "LITERAL">SERVER-HEADER-FILTER:</tt> followed by the filter's <span
- class="emphasis"><i class="EMPHASIS">name</i></span>, and a short
- (one line) <span class="emphasis"><i class=
- "EMPHASIS">description</i></span> of what it does. Below that line
- come the <span class="emphasis"><i class="EMPHASIS">jobs</i></span>,
- i.e. lines that define the actual text substitutions. By convention,
- the name of a filter should describe what the filter <span class=
- "emphasis"><i class="EMPHASIS">eliminates</i></span>. The comment is
- used in the <a href="http://config.privoxy.org/" target=
- "_top">web-based user interface</a>.
- </p>
- <p>
- Once a filter called <tt class="REPLACEABLE"><i>name</i></tt> has
- been defined in the filter file, it can be invoked by using an action
- of the form +<tt class="LITERAL"><a href=
- "actions-file.html#FILTER">filter</a>{<tt class=
- "REPLACEABLE"><i>name</i></tt>}</tt> in any <a href=
- "actions-file.html">actions file</a>.
- </p>
- <p>
- Filter definitions start with a header line that contains the filter
- type, the filter name and the filter description. A content filter
- header line for a filter called <span class="QUOTE">"foo"</span>
- could look like this:
- </p>
- <p>
- </p>
+ <p>Our complete filter now looks like this:</p>
<table border="0" bgcolor="#E0E0E0" width="100%">
<tr>
<td>
-<pre class="SCREEN">
-FILTER: foo Replace all "foo" with "bar"
-</pre>
+ <pre class="SCREEN">FILTER: foo Replace all "foo" with "bar"
+s/foo/bar/g</pre>
</td>
</tr>
</table>
-
- <p>
- Below that line, and up to the next header line, come the jobs that
- define what text replacements the filter executes. They are specified
- in a syntax that imitates <a href="http://www.perl.org/" target=
- "_top">Perl</a>'s <tt class="LITERAL">s///</tt> operator. If you are
- familiar with Perl, you will find this to be quite intuitive, and may
- want to look at the PCRS documentation for the subtle differences to
- Perl behaviour. Most notably, the non-standard option letter <tt
- class="LITERAL">U</tt> is supported, which turns the default to
- ungreedy matching.
- </p>
- <p>
- If you are new to <a href=
- "http://en.wikipedia.org/wiki/Regular_expressions" target=
- "_top"><span class="QUOTE">"Regular Expressions"</span></a>, you
- might want to take a look at the <a href=
- "appendix.html#REGEX">Appendix on regular expressions</a>, and see
- the <a href="http://perldoc.perl.org/perlre.html" target="_top">Perl
- manual</a> for <a href="http://perldoc.perl.org/perlop.html" target=
- "_top">the <tt class="LITERAL">s///</tt> operator's syntax</a> and <a
- href="http://perldoc.perl.org/perlre.html" target="_top">Perl-style
- regular expressions</a> in general. The below examples might also
- help to get you started.
- </p>
- <div class="SECT2">
- <h2 class="SECT2">
- <a name="AEN4903">9.1. Filter File Tutorial</a>
- </h2>
- <p>
- Now, let's complete our <span class="QUOTE">"foo"</span> content
- filter. We have already defined the heading, but the jobs are still
- missing. Since all it does is to replace <span class=
- "QUOTE">"foo"</span> with <span class="QUOTE">"bar"</span>, there
- is only one (trivial) job needed:
- </p>
- <p>
- </p>
- <table border="0" bgcolor="#E0E0E0" width="100%">
- <tr>
- <td>
-<pre class="SCREEN">
-s/foo/bar/
-</pre>
- </td>
- </tr>
- </table>
-
- <p>
- But wait! Didn't the comment say that <span class="emphasis"><i
- class="EMPHASIS">all</i></span> occurrences of <span class=
- "QUOTE">"foo"</span> should be replaced? Our current job will only
- take care of the first <span class="QUOTE">"foo"</span> on each
- page. For global substitution, we'll need to add the <tt class=
- "LITERAL">g</tt> option:
- </p>
- <p>
- </p>
- <table border="0" bgcolor="#E0E0E0" width="100%">
- <tr>
- <td>
-<pre class="SCREEN">
-s/foo/bar/g
-</pre>
- </td>
- </tr>
- </table>
-
- <p>
- Our complete filter now looks like this:
- </p>
- <p>
- </p>
- <table border="0" bgcolor="#E0E0E0" width="100%">
- <tr>
- <td>
-<pre class="SCREEN">
-FILTER: foo Replace all "foo" with "bar"
-s/foo/bar/g
-</pre>
- </td>
- </tr>
- </table>
-
- <p>
- Let's look at some real filters for more interesting examples. Here
- you see a filter that protects against some common annoyances that
- arise from JavaScript abuse. Let's look at its jobs one after the
- other:
- </p>
- <p>
- </p>
- <table border="0" bgcolor="#E0E0E0" width="100%">
- <tr>
- <td>
-<pre class="SCREEN">
-FILTER: js-annoyances Get rid of particularly annoying JavaScript abuse
+ <p>Let's look at some real filters for more interesting examples. Here you see a filter that protects against
+ some common annoyances that arise from JavaScript abuse. Let's look at its jobs one after the other:</p>
+ <table border="0" bgcolor="#E0E0E0" width="100%">
+ <tr>
+ <td>
+ <pre class="SCREEN">FILTER: js-annoyances Get rid of particularly annoying JavaScript abuse
# Get rid of JavaScript referrer tracking. Test page: http://www.randomoddness.com/untitled.htm
#
-s|(<script.*)document\.referrer(.*</script>)|$1"Not Your Business!"$2|Usg
-</pre>
- </td>
- </tr>
- </table>
-
- <p>
- Following the header line and a comment, you see the job. Note that
- it uses <tt class="LITERAL">|</tt> as the delimiter instead of <tt
- class="LITERAL">/</tt>, because the pattern contains a forward
- slash, which would otherwise have to be escaped by a backslash (<tt
- class="LITERAL">\</tt>).
- </p>
- <p>
- Now, let's examine the pattern: it starts with the text <tt class=
- "LITERAL"><script.*</tt> enclosed in parentheses. Since the dot
- matches any character, and <tt class="LITERAL">*</tt> means: <span
- class="QUOTE">"Match an arbitrary number of the element left of
- myself"</span>, this matches <span class=
- "QUOTE">"<script"</span>, followed by <span class="emphasis"><i
- class="EMPHASIS">any</i></span> text, i.e. it matches the whole
- page, from the start of the first <script> tag.
- </p>
- <p>
- That's more than we want, but the pattern continues: <tt class=
- "LITERAL">document\.referrer</tt> matches only the exact string
- <span class="QUOTE">"document.referrer"</span>. The dot needed to
- be <span class="emphasis"><i class="EMPHASIS">escaped</i></span>,
- i.e. preceded by a backslash, to take away its special meaning as a
- joker, and make it just a regular dot. So far, the meaning is:
- Match from the start of the first <script> tag in a the page,
- up to, and including, the text <span class=
- "QUOTE">"document.referrer"</span>, if <span class="emphasis"><i
- class="EMPHASIS">both</i></span> are present in the page (and
- appear in that order).
- </p>
- <p>
- But there's still more pattern to go. The next element, again
- enclosed in parentheses, is <tt class=
- "LITERAL">.*</script></tt>. You already know what <tt class=
- "LITERAL">.*</tt> means, so the whole pattern translates to: Match
- from the start of the first <script> tag in a page to the end
- of the last <script> tag, provided that the text <span class=
- "QUOTE">"document.referrer"</span> appears somewhere in between.
- </p>
- <p>
- This is still not the whole story, since we have ignored the
- options and the parentheses: The portions of the page matched by
- sub-patterns that are enclosed in parentheses, will be remembered
- and be available through the variables <tt class="LITERAL">$1, $2,
- ...</tt> in the substitute. The <tt class="LITERAL">U</tt> option
- switches to ungreedy matching, which means that the first <tt
- class="LITERAL">.*</tt> in the pattern will only <span class=
- "QUOTE">"eat up"</span> all text in between <span class=
- "QUOTE">"<script"</span> and the <span class="emphasis"><i
- class="EMPHASIS">first</i></span> occurrence of <span class=
- "QUOTE">"document.referrer"</span>, and that the second <tt class=
- "LITERAL">.*</tt> will only span the text up to the <span class=
- "emphasis"><i class="EMPHASIS">first</i></span> <span class=
- "QUOTE">"</script>"</span> tag. Furthermore, the <tt class=
- "LITERAL">s</tt> option says that the match may span multiple lines
- in the page, and the <tt class="LITERAL">g</tt> option again means
- that the substitution is global.
- </p>
- <p>
- So, to summarize, the pattern means: Match all scripts that contain
- the text <span class="QUOTE">"document.referrer"</span>. Remember
- the parts of the script from (and including) the start tag up to
- (and excluding) the string <span class=
- "QUOTE">"document.referrer"</span> as <tt class="LITERAL">$1</tt>,
- and the part following that string, up to and including the closing
- tag, as <tt class="LITERAL">$2</tt>.
- </p>
- <p>
- Now the pattern is deciphered, but wasn't this about substituting
- things? So lets look at the substitute: <tt class="LITERAL">$1"Not
- Your Business!"$2</tt> is easy to read: The text remembered as <tt
- class="LITERAL">$1</tt>, followed by <tt class="LITERAL">"Not Your
- Business!"</tt> (<span class="emphasis"><i class=
- "EMPHASIS">including</i></span> the quotation marks!), followed by
- the text remembered as <tt class="LITERAL">$2</tt>. This produces
- an exact copy of the original string, with the middle part (the
- <span class="QUOTE">"document.referrer"</span>) replaced by <tt
- class="LITERAL">"Not Your Business!"</tt>.
- </p>
- <p>
- The whole job now reads: Replace <span class=
- "QUOTE">"document.referrer"</span> by <tt class="LITERAL">"Not Your
- Business!"</tt> wherever it appears inside a <script> tag.
- Note that this job won't break JavaScript syntax, since both the
- original and the replacement are syntactically valid string
- objects. The script just won't have access to the referrer
- information anymore.
- </p>
- <p>
- We'll show you two other jobs from the JavaScript taming
- department, but this time only point out the constructs of special
- interest:
- </p>
- <p>
- </p>
- <table border="0" bgcolor="#E0E0E0" width="100%">
- <tr>
- <td>
-<pre class="SCREEN">
-# The status bar is for displaying link targets, not pointless blahblah
+s|(<script.*)document\.referrer(.*</script>)|$1"Not Your Business!"$2|Usg</pre>
+ </td>
+ </tr>
+ </table>
+ <p>Following the header line and a comment, you see the job. Note that it uses <tt class="LITERAL">|</tt> as the
+ delimiter instead of <tt class="LITERAL">/</tt>, because the pattern contains a forward slash, which would
+ otherwise have to be escaped by a backslash (<tt class="LITERAL">\</tt>).</p>
+ <p>Now, let's examine the pattern: it starts with the text <tt class="LITERAL"><script.*</tt> enclosed in
+ parentheses. Since the dot matches any character, and <tt class="LITERAL">*</tt> means: <span class=
+ "QUOTE">"Match an arbitrary number of the element left of myself"</span>, this matches <span class=
+ "QUOTE">"<script"</span>, followed by <span class="emphasis"><i class="EMPHASIS">any</i></span> text, i.e. it
+ matches the whole page, from the start of the first <script> tag.</p>
+ <p>That's more than we want, but the pattern continues: <tt class="LITERAL">document\.referrer</tt> matches only
+ the exact string <span class="QUOTE">"document.referrer"</span>. The dot needed to be <span class=
+ "emphasis"><i class="EMPHASIS">escaped</i></span>, i.e. preceded by a backslash, to take away its special meaning
+ as a joker, and make it just a regular dot. So far, the meaning is: Match from the start of the first
+ <script> tag in a the page, up to, and including, the text <span class="QUOTE">"document.referrer"</span>,
+ if <span class="emphasis"><i class="EMPHASIS">both</i></span> are present in the page (and appear in that
+ order).</p>
+ <p>But there's still more pattern to go. The next element, again enclosed in parentheses, is <tt class=
+ "LITERAL">.*</script></tt>. You already know what <tt class="LITERAL">.*</tt> means, so the whole pattern
+ translates to: Match from the start of the first <script> tag in a page to the end of the last
+ <script> tag, provided that the text <span class="QUOTE">"document.referrer"</span> appears somewhere in
+ between.</p>
+ <p>This is still not the whole story, since we have ignored the options and the parentheses: The portions of the
+ page matched by sub-patterns that are enclosed in parentheses, will be remembered and be available through the
+ variables <tt class="LITERAL">$1, $2, ...</tt> in the substitute. The <tt class="LITERAL">U</tt> option switches
+ to ungreedy matching, which means that the first <tt class="LITERAL">.*</tt> in the pattern will only
+ <span class="QUOTE">"eat up"</span> all text in between <span class="QUOTE">"<script"</span> and the
+ <span class="emphasis"><i class="EMPHASIS">first</i></span> occurrence of <span class=
+ "QUOTE">"document.referrer"</span>, and that the second <tt class="LITERAL">.*</tt> will only span the text up to
+ the <span class="emphasis"><i class="EMPHASIS">first</i></span> <span class="QUOTE">"</script>"</span> tag.
+ Furthermore, the <tt class="LITERAL">s</tt> option says that the match may span multiple lines in the page, and
+ the <tt class="LITERAL">g</tt> option again means that the substitution is global.</p>
+ <p>So, to summarize, the pattern means: Match all scripts that contain the text <span class=
+ "QUOTE">"document.referrer"</span>. Remember the parts of the script from (and including) the start tag up to
+ (and excluding) the string <span class="QUOTE">"document.referrer"</span> as <tt class="LITERAL">$1</tt>, and the
+ part following that string, up to and including the closing tag, as <tt class="LITERAL">$2</tt>.</p>
+ <p>Now the pattern is deciphered, but wasn't this about substituting things? So lets look at the substitute:
+ <tt class="LITERAL">$1"Not Your Business!"$2</tt> is easy to read: The text remembered as <tt class=
+ "LITERAL">$1</tt>, followed by <tt class="LITERAL">"Not Your Business!"</tt> (<span class="emphasis"><i class=
+ "EMPHASIS">including</i></span> the quotation marks!), followed by the text remembered as <tt class=
+ "LITERAL">$2</tt>. This produces an exact copy of the original string, with the middle part (the <span class=
+ "QUOTE">"document.referrer"</span>) replaced by <tt class="LITERAL">"Not Your Business!"</tt>.</p>
+ <p>The whole job now reads: Replace <span class="QUOTE">"document.referrer"</span> by <tt class="LITERAL">"Not
+ Your Business!"</tt> wherever it appears inside a <script> tag. Note that this job won't break JavaScript
+ syntax, since both the original and the replacement are syntactically valid string objects. The script just won't
+ have access to the referrer information anymore.</p>
+ <p>We'll show you two other jobs from the JavaScript taming department, but this time only point out the
+ constructs of special interest:</p>
+ <table border="0" bgcolor="#E0E0E0" width="100%">
+ <tr>
+ <td>
+ <pre class="SCREEN"># The status bar is for displaying link targets, not pointless blahblah
#
-s/window\.status\s*=\s*(['"]).*?\1/dUmMy=1/ig
-</pre>
- </td>
- </tr>
- </table>
-
- <p>
- <tt class="LITERAL">\s</tt> stands for whitespace characters
- (space, tab, newline, carriage return, form feed), so that <tt
- class="LITERAL">\s*</tt> means: <span class="QUOTE">"zero or more
- whitespace"</span>. The <tt class="LITERAL">?</tt> in <tt class=
- "LITERAL">.*?</tt> makes this matching of arbitrary text ungreedy.
- (Note that the <tt class="LITERAL">U</tt> option is not set). The
- <tt class="LITERAL">['"]</tt> construct means: <span class=
- "QUOTE">"a single <span class="emphasis"><i class=
- "EMPHASIS">or</i></span> a double quote"</span>. Finally, <tt
- class="LITERAL">\1</tt> is a back-reference to the first
- parenthesis just like <tt class="LITERAL">$1</tt> above, with the
- difference that in the <span class="emphasis"><i class=
- "EMPHASIS">pattern</i></span>, a backslash indicates a
- back-reference, whereas in the <span class="emphasis"><i class=
- "EMPHASIS">substitute</i></span>, it's the dollar.
- </p>
- <p>
- So what does this job do? It replaces assignments of single- or
- double-quoted strings to the <span class=
- "QUOTE">"window.status"</span> object with a dummy assignment
- (using a variable name that is hopefully odd enough not to conflict
- with real variables in scripts). Thus, it catches many cases where
- e.g. pointless descriptions are displayed in the status bar instead
- of the link target when you move your mouse over links.
- </p>
- <p>
- </p>
- <table border="0" bgcolor="#E0E0E0" width="100%">
- <tr>
- <td>
-<pre class="SCREEN">
-# Kill OnUnload popups. Yummy. Test: http://www.zdnet.com/zdsubs/yahoo/tree/yfs.html
+s/window\.status\s*=\s*(['"]).*?\1/dUmMy=1/ig</pre>
+ </td>
+ </tr>
+ </table>
+ <p><tt class="LITERAL">\s</tt> stands for whitespace characters (space, tab, newline, carriage return, form
+ feed), so that <tt class="LITERAL">\s*</tt> means: <span class="QUOTE">"zero or more whitespace"</span>. The
+ <tt class="LITERAL">?</tt> in <tt class="LITERAL">.*?</tt> makes this matching of arbitrary text ungreedy. (Note
+ that the <tt class="LITERAL">U</tt> option is not set). The <tt class="LITERAL">['"]</tt> construct means:
+ <span class="QUOTE">"a single <span class="emphasis"><i class="EMPHASIS">or</i></span> a double quote"</span>.
+ Finally, <tt class="LITERAL">\1</tt> is a back-reference to the first parenthesis just like <tt class=
+ "LITERAL">$1</tt> above, with the difference that in the <span class="emphasis"><i class=
+ "EMPHASIS">pattern</i></span>, a backslash indicates a back-reference, whereas in the <span class=
+ "emphasis"><i class="EMPHASIS">substitute</i></span>, it's the dollar.</p>
+ <p>So what does this job do? It replaces assignments of single- or double-quoted strings to the <span class=
+ "QUOTE">"window.status"</span> object with a dummy assignment (using a variable name that is hopefully odd enough
+ not to conflict with real variables in scripts). Thus, it catches many cases where e.g. pointless descriptions
+ are displayed in the status bar instead of the link target when you move your mouse over links.</p>
+ <table border="0" bgcolor="#E0E0E0" width="100%">
+ <tr>
+ <td>
+ <pre class="SCREEN"># Kill OnUnload popups. Yummy. Test: http://www.zdnet.com/zdsubs/yahoo/tree/yfs.html
#
-s/(<body [^>]*)onunload(.*>)/$1never$2/iU
-</pre>
- </td>
- </tr>
- </table>
-
- <p>
- Including the <a href=
- "http://www.w3.org/TR/2000/REC-DOM-Level-2-Events-20001113/events.html#Events-eventgroupings-htmlevents"
- target="_top">OnUnload event binding</a> in the HTML DOM was a
- <span class="emphasis"><i class="EMPHASIS">CRIME</i></span>. When I
- close a browser window, I want it to close and die. Basta. This job
- replaces the <span class="QUOTE">"onunload"</span> attribute in
- <span class="QUOTE">"<body>"</span> tags with the dummy word
- <tt class="LITERAL">never</tt>. Note that the <tt class=
- "LITERAL">i</tt> option makes the pattern matching
- case-insensitive. Also note that ungreedy matching alone doesn't
- always guarantee a minimal match: In the first parenthesis, we had
- to use <tt class="LITERAL">[^>]*</tt> instead of <tt class=
- "LITERAL">.*</tt> to prevent the match from exceeding the
- <body> tag if it doesn't contain <span class=
- "QUOTE">"OnUnload"</span>, but the page's content does.
- </p>
- <p>
- The last example is from the fun department:
- </p>
- <p>
- </p>
- <table border="0" bgcolor="#E0E0E0" width="100%">
- <tr>
- <td>
-<pre class="SCREEN">
-FILTER: fun Fun text replacements
+s/(<body [^>]*)onunload(.*>)/$1never$2/iU</pre>
+ </td>
+ </tr>
+ </table>
+ <p>Including the <a href=
+ "http://www.w3.org/TR/2000/REC-DOM-Level-2-Events-20001113/events.html#Events-eventgroupings-htmlevents" target=
+ "_top">OnUnload event binding</a> in the HTML DOM was a <span class="emphasis"><i class=
+ "EMPHASIS">CRIME</i></span>. When I close a browser window, I want it to close and die. Basta. This job replaces
+ the <span class="QUOTE">"onunload"</span> attribute in <span class="QUOTE">"<body>"</span> tags with the
+ dummy word <tt class="LITERAL">never</tt>. Note that the <tt class="LITERAL">i</tt> option makes the pattern
+ matching case-insensitive. Also note that ungreedy matching alone doesn't always guarantee a minimal match: In
+ the first parenthesis, we had to use <tt class="LITERAL">[^>]*</tt> instead of <tt class="LITERAL">.*</tt> to
+ prevent the match from exceeding the <body> tag if it doesn't contain <span class=
+ "QUOTE">"OnUnload"</span>, but the page's content does.</p>
+ <p>The last example is from the fun department:</p>
+ <table border="0" bgcolor="#E0E0E0" width="100%">
+ <tr>
+ <td>
+ <pre class="SCREEN">FILTER: fun Fun text replacements
# Spice the daily news:
#
-s/microsoft(?!\.com)/MicroSuck/ig
-</pre>
- </td>
- </tr>
- </table>
-
- <p>
- Note the <tt class="LITERAL">(?!\.com)</tt> part (a so-called
- negative lookahead) in the job's pattern, which means: Don't match,
- if the string <span class="QUOTE">".com"</span> appears directly
- following <span class="QUOTE">"microsoft"</span> in the page. This
- prevents links to microsoft.com from being trashed, while still
- replacing the word everywhere else.
- </p>
- <p>
- </p>
- <table border="0" bgcolor="#E0E0E0" width="100%">
- <tr>
- <td>
-<pre class="SCREEN">
-# Buzzword Bingo (example for extended regex syntax)
+s/microsoft(?!\.com)/MicroSuck/ig</pre>
+ </td>
+ </tr>
+ </table>
+ <p>Note the <tt class="LITERAL">(?!\.com)</tt> part (a so-called negative lookahead) in the job's pattern, which
+ means: Don't match, if the string <span class="QUOTE">".com"</span> appears directly following <span class=
+ "QUOTE">"microsoft"</span> in the page. This prevents links to microsoft.com from being trashed, while still
+ replacing the word everywhere else.</p>
+ <table border="0" bgcolor="#E0E0E0" width="100%">
+ <tr>
+ <td>
+ <pre class="SCREEN"># Buzzword Bingo (example for extended regex syntax)
#
s* industry[ -]leading \
| cutting[ -]edge \
| unparalleled \
| unrivalled \
*<font color="red"><b>BINGO!</b></font> \
-*igx
-</pre>
- </td>
- </tr>
- </table>
-
- <p>
- The <tt class="LITERAL">x</tt> option in this job turns on extended
- syntax, and allows for e.g. the liberal use of (non-interpreted!)
- whitespace for nicer formatting.
- </p>
- <p>
- You get the idea?
- </p>
- </div>
- <div class="SECT2">
- <h2 class="SECT2">
- <a name="PREDEFINED-FILTERS">9.2. The Pre-defined Filters</a>
- </h2>
- <p>
- The distribution <tt class="FILENAME">default.filter</tt> file
- contains a selection of pre-defined filters for your convenience:
- </p>
- <div class="VARIABLELIST">
- <dl>
- <dt>
- <span class="emphasis"><i class=
- "EMPHASIS">js-annoyances</i></span>
- </dt>
- <dd>
- <p>
- The purpose of this filter is to get rid of particularly
- annoying JavaScript abuse. To that end, it
- </p>
- <ul>
- <li>
- <p>
- replaces JavaScript references to the browser's referrer
- information with the string "Not Your Business!". This
- compliments the <tt class="LITERAL"><a href=
- "actions-file.html#HIDE-REFERRER">hide-referrer</a></tt>
- action on the content level.
- </p>
- </li>
- <li>
- <p>
- removes the bindings to the DOM's <a href=
- "http://www.w3.org/TR/2000/REC-DOM-Level-2-Events-20001113/events.html#Events-eventgroupings-htmlevents"
- target="_top">unload event</a> which we feel has no
- right to exist and is responsible for most <span class=
- "QUOTE">"exit consoles"</span>, i.e. nasty windows that
- pop up when you close another one.
- </p>
- </li>
- <li>
- <p>
- removes code that causes new windows to be opened with
- undesired properties, such as being full-screen,
- non-resizeable, without location, status or menu bar etc.
- </p>
- </li>
- </ul>
-
- <p>
- Use with caution. This is an aggressive filter, and can break
- sites that rely heavily on JavaScript.
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class="EMPHASIS">js-events</i></span>
- </dt>
- <dd>
- <p>
- This is a very radical measure. It removes virtually all
- JavaScript event bindings, which means that scripts can not
- react to user actions such as mouse movements or clicks,
- window resizing etc, anymore. Use with caution!
- </p>
- <p>
- We <span class="emphasis"><i class="EMPHASIS">strongly
- discourage</i></span> using this filter as a default since it
- breaks many legitimate scripts. It is meant for use only on
- extra-nasty sites (should you really need to go there).
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class=
- "EMPHASIS">html-annoyances</i></span>
- </dt>
- <dd>
- <p>
- This filter will undo many common instances of HTML based
- abuse.
- </p>
- <p>
- The <tt class="LITERAL">BLINK</tt> and <tt class=
- "LITERAL">MARQUEE</tt> tags are neutralized (yeah baby!), and
- browser windows will be created as resizeable (as of course
- they should be!), and will have location, scroll and menu
- bars -- even if specified otherwise.
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class=
- "EMPHASIS">content-cookies</i></span>
- </dt>
- <dd>
- <p>
- Most cookies are set in the HTTP dialog, where they can be
- intercepted by the <tt class="LITERAL"><a href=
- "actions-file.html#CRUNCH-INCOMING-COOKIES">crunch-incoming-cookies</a></tt>
- and <tt class="LITERAL"><a href=
- "actions-file.html#CRUNCH-OUTGOING-COOKIES">crunch-outgoing-cookies</a></tt>
- actions. But web sites increasingly make use of HTML meta
- tags and JavaScript to sneak cookies to the browser on the
- content level.
- </p>
- <p>
- This filter disables most HTML and JavaScript code that reads
- or sets cookies. It cannot detect all clever uses of these
- types of code, so it should not be relied on as an absolute
- fix. Use it wherever you would also use the cookie crunch
- actions.
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class="EMPHASIS">refresh
- tags</i></span>
- </dt>
- <dd>
- <p>
- Disable any refresh tags if the interval is greater than nine
- seconds (so that redirections done via refresh tags are not
- destroyed). This is useful for dial-on-demand setups, or for
- those who find this HTML feature annoying.
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class=
- "EMPHASIS">unsolicited-popups</i></span>
- </dt>
- <dd>
- <p>
- This filter attempts to prevent only <span class=
- "QUOTE">"unsolicited"</span> pop-up windows from opening, yet
- still allow pop-up windows that the user has explicitly
- chosen to open. It was added in version 3.0.1, as an
- improvement over earlier such filters.
- </p>
- <p>
- Technical note: The filter works by redefining the
- window.open JavaScript function to a dummy function, <tt
- class="LITERAL">PrivoxyWindowOpen()</tt>, during the loading
- and rendering phase of each HTML page access, and restoring
- the function afterward.
- </p>
- <p>
- This is recommended only for browsers that cannot perform
- this function reliably themselves. And be aware that some
- sites require such windows in order to function normally. Use
- with caution.
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class=
- "EMPHASIS">all-popups</i></span>
- </dt>
- <dd>
- <p>
- Attempt to prevent <span class="emphasis"><i class=
- "EMPHASIS">all</i></span> pop-up windows from opening. Note
- this should be used with even more discretion than the above,
- since it is more likely to break some sites that require
- pop-ups for normal usage. Use with caution.
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class=
- "EMPHASIS">img-reorder</i></span>
- </dt>
- <dd>
- <p>
- This is a helper filter that has no value if used alone. It
- makes the <tt class="LITERAL">banners-by-size</tt> and <tt
- class="LITERAL">banners-by-link</tt> (see below) filters more
- effective and should be enabled together with them.
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class=
- "EMPHASIS">banners-by-size</i></span>
- </dt>
- <dd>
- <p>
- This filter removes image tags purely based on what size they
- are. Fortunately for us, many ads and banner images tend to
- conform to certain standardized sizes, which makes this
- filter quite effective for ad stripping purposes.
- </p>
- <p>
- Occasionally this filter will cause false positives on images
- that are not ads, but just happen to be of one of the
- standard banner sizes.
- </p>
- <p>
- Recommended only for those who require extreme ad blocking.
- The default block rules should catch 95+% of all ads <span
- class="emphasis"><i class="EMPHASIS">without</i></span> this
- filter enabled.
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class=
- "EMPHASIS">banners-by-link</i></span>
- </dt>
- <dd>
- <p>
- This is an experimental filter that attempts to kill any
- banners if their URLs seem to point to known or suspected
- click trackers. It is currently not of much value and is not
- recommended for use by default.
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class="EMPHASIS">webbugs</i></span>
- </dt>
- <dd>
- <p>
- Webbugs are small, invisible images (technically 1X1 GIF
- images), that are used to track users across websites, and
- collect information on them. As an HTML page is loaded by the
- browser, an embedded image tag causes the browser to contact
- a third-party site, disclosing the tracking information
- through the requested URL and/or cookies for that third-party
- domain, without the user ever becoming aware of the
- interaction with the third-party site. HTML-ized spam also
- uses a similar technique to verify email addresses.
- </p>
- <p>
- This filter removes the HTML code that loads such <span
- class="QUOTE">"webbugs"</span>.
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class=
- "EMPHASIS">tiny-textforms</i></span>
- </dt>
- <dd>
- <p>
- A rather special-purpose filter that can be used to enlarge
- textareas (those multi-line text boxes in web forms) and turn
- off hard word wrap in them. It was written for the
- sourceforge.net tracker system where such boxes are a
- nuisance, but it can be handy on other sites, too.
- </p>
- <p>
- It is not recommended to use this filter as a default.
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class=
- "EMPHASIS">jumping-windows</i></span>
- </dt>
- <dd>
- <p>
- Many consider windows that move, or resize themselves to be
- abusive. This filter neutralizes the related JavaScript code.
- Note that some sites might not display or behave as intended
- when using this filter. Use with caution.
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class=
- "EMPHASIS">frameset-borders</i></span>
- </dt>
- <dd>
- <p>
- Some web designers seem to assume that everyone in the world
- will view their web sites using the same browser brand and
- version, screen resolution etc, because only that assumption
- could explain why they'd use static frame sizes, yet prevent
- their frames from being resized by the user, should they be
- too small to show their whole content.
- </p>
- <p>
- This filter removes the related HTML code. It should only be
- applied to sites which need it.
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class=
- "EMPHASIS">demoronizer</i></span>
- </dt>
- <dd>
- <p>
- Many Microsoft products that generate HTML use non-standard
- extensions (read: violations) of the ISO 8859-1 aka Latin-1
- character set. This can cause those HTML documents to display
- with errors on standard-compliant platforms.
- </p>
- <p>
- This filter translates the MS-only characters into Latin-1
- equivalents. It is not necessary when using MS products, and
- will cause corruption of all documents that use 8-bit
- character sets other than Latin-1. It's mostly worthwhile for
- Europeans on non-MS platforms, if weird garbage characters
- sometimes appear on some pages, or user agents that don't
- correct for this on the fly.
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class=
- "EMPHASIS">shockwave-flash</i></span>
- </dt>
- <dd>
- <p>
- A filter for shockwave haters. As the name suggests, this
- filter strips code out of web pages that is used to embed
- shockwave flash objects.
- </p>
- <p>
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class=
- "EMPHASIS">quicktime-kioskmode</i></span>
- </dt>
- <dd>
- <p>
- Change HTML code that embeds Quicktime objects so that
- kioskmode, which prevents saving, is disabled.
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class="EMPHASIS">fun</i></span>
- </dt>
- <dd>
- <p>
- Text replacements for subversive browsing fun. Make fun of
- your favorite Monopolist or play buzzword bingo.
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class=
- "EMPHASIS">crude-parental</i></span>
- </dt>
- <dd>
- <p>
- A demonstration-only filter that shows how <span class=
- "APPLICATION">Privoxy</span> can be used to delete web
- content on a keyword basis.
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class=
- "EMPHASIS">ie-exploits</i></span>
- </dt>
- <dd>
- <p>
- An experimental collection of text replacements to disable
- malicious HTML and JavaScript code that exploits known
- security holes in Internet Explorer.
- </p>
- <p>
- Presently, it only protects against Nimda and a cross-site
- scripting bug, and would need active maintenance to provide
- more substantial protection.
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class=
- "EMPHASIS">site-specifics</i></span>
- </dt>
- <dd>
- <p>
- Some web sites have very specific problems, the cure for
- which doesn't apply anywhere else, or could even cause damage
- on other sites.
- </p>
- <p>
- This is a collection of such site-specific cures which should
- only be applied to the sites they were intended for, which is
- what the supplied <tt class="FILENAME">default.action</tt>
- file does. Users shouldn't need to change anything regarding
- this filter.
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class="EMPHASIS">google</i></span>
- </dt>
- <dd>
- <p>
- A CSS based block for Google text ads. Also removes a width
- limitation and the toolbar advertisement.
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class="EMPHASIS">yahoo</i></span>
- </dt>
- <dd>
- <p>
- Another CSS based block, this time for Yahoo text ads. And
- removes a width limitation as well.
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class="EMPHASIS">msn</i></span>
- </dt>
- <dd>
- <p>
- Another CSS based block, this time for MSN text ads. And
- removes tracking URLs, as well as a width limitation.
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class="EMPHASIS">blogspot</i></span>
- </dt>
- <dd>
- <p>
- Cleans up some Blogspot blogs. Read the fine print before
- using this one!
- </p>
- <p>
- This filter also intentionally removes some navigation stuff
- and sets the page width to 100%. As a result, some rounded
- <span class="QUOTE">"corners"</span> would appear to early or
- not at all and as fixing this would require a browser that
- understands background-size (CSS3), they are removed instead.
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class=
- "EMPHASIS">xml-to-html</i></span>
- </dt>
- <dd>
- <p>
- Server-header filter to change the Content-Type from xml to
- html.
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class=
- "EMPHASIS">html-to-xml</i></span>
- </dt>
- <dd>
- <p>
- Server-header filter to change the Content-Type from html to
- xml.
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class="EMPHASIS">no-ping</i></span>
- </dt>
- <dd>
- <p>
- Removes the non-standard <tt class="LITERAL">ping</tt>
- attribute from anchor and area HTML tags.
- </p>
- </dd>
- <dt>
- <span class="emphasis"><i class=
- "EMPHASIS">hide-tor-exit-notation</i></span>
- </dt>
- <dd>
- <p>
- Client-header filter to remove the <b class="COMMAND">Tor</b>
- exit node notation found in Host and Referer headers.
- </p>
- <p>
- If <span class="APPLICATION">Privoxy</span> and <b class=
- "COMMAND">Tor</b> are chained and <span class=
- "APPLICATION">Privoxy</span> is configured to use socks4a,
- one can use <span class=
- "QUOTE">"http://www.example.org.foobar.exit/"</span> to
- access the host <span class="QUOTE">"www.example.org"</span>
- through the <b class="COMMAND">Tor</b> exit node <span class=
- "QUOTE">"foobar"</span>.
- </p>
- <p>
- As the HTTP client isn't aware of this notation, it treats
- the whole string <span class=
- "QUOTE">"www.example.org.foobar.exit"</span> as host and uses
- it for the <span class="QUOTE">"Host"</span> and <span class=
- "QUOTE">"Referer"</span> headers. From the server's point of
- view the resulting headers are invalid and can cause
- problems.
- </p>
- <p>
- An invalid <span class="QUOTE">"Referer"</span> header can
- trigger <span class="QUOTE">"hot-linking"</span> protections,
- an invalid <span class="QUOTE">"Host"</span> header will make
- it impossible for the server to find the right vhost (several
- domains hosted on the same IP address).
- </p>
- <p>
- This client-header filter removes the <span class=
- "QUOTE">"foo.exit"</span> part in those headers to prevent
- the mentioned problems. Note that it only modifies the HTTP
- headers, it doesn't make it impossible for the server to
- detect your <b class="COMMAND">Tor</b> exit node based on the
- IP address the request is coming from.
- </p>
- </dd>
- </dl>
- </div>
- </div>
- </div>
- <div class="NAVFOOTER">
- <hr width="100%" class="c1">
- <table summary="Footer navigation table" width="100%" border="0"
- cellpadding="0" cellspacing="0">
- <tr>
- <td width="33%" align="left" valign="top">
- <a href="actions-file.html" accesskey="P">Prev</a>
- </td>
- <td width="34%" align="center" valign="top">
- <a href="index.html" accesskey="H">Home</a>
- </td>
- <td width="33%" align="right" valign="top">
- <a href="templates.html" accesskey="N">Next</a>
+*igx</pre>
</td>
</tr>
+ </table>
+ <p>The <tt class="LITERAL">x</tt> option in this job turns on extended syntax, and allows for e.g. the liberal
+ use of (non-interpreted!) whitespace for nicer formatting.</p>
+ <p>You get the idea?</p>
+ </div>
+ <div class="SECT2">
+ <h2 class="SECT2"><a name="PREDEFINED-FILTERS" id="PREDEFINED-FILTERS">9.2. The Pre-defined Filters</a></h2>
+ <p>The distribution <tt class="FILENAME">default.filter</tt> file contains a selection of pre-defined filters for
+ your convenience:</p>
+ <div class="VARIABLELIST">
+ <dl>
+ <dt><span class="emphasis"><i class="EMPHASIS">js-annoyances</i></span></dt>
+ <dd>
+ <p>The purpose of this filter is to get rid of particularly annoying JavaScript abuse. To that end, it</p>
+ <ul>
+ <li>
+ <p>replaces JavaScript references to the browser's referrer information with the string "Not Your
+ Business!". This compliments the <tt class="LITERAL"><a href=
+ "actions-file.html#HIDE-REFERRER">hide-referrer</a></tt> action on the content level.</p>
+ </li>
+ <li>
+ <p>removes the bindings to the DOM's <a href=
+ "http://www.w3.org/TR/2000/REC-DOM-Level-2-Events-20001113/events.html#Events-eventgroupings-htmlevents"
+ target="_top">unload event</a> which we feel has no right to exist and is responsible for most
+ <span class="QUOTE">"exit consoles"</span>, i.e. nasty windows that pop up when you close another
+ one.</p>
+ </li>
+ <li>
+ <p>removes code that causes new windows to be opened with undesired properties, such as being
+ full-screen, non-resizeable, without location, status or menu bar etc.</p>
+ </li>
+ </ul>
+ <p>Use with caution. This is an aggressive filter, and can break sites that rely heavily on JavaScript.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">js-events</i></span></dt>
+ <dd>
+ <p>This is a very radical measure. It removes virtually all JavaScript event bindings, which means that
+ scripts can not react to user actions such as mouse movements or clicks, window resizing etc, anymore. Use
+ with caution!</p>
+ <p>We <span class="emphasis"><i class="EMPHASIS">strongly discourage</i></span> using this filter as a
+ default since it breaks many legitimate scripts. It is meant for use only on extra-nasty sites (should you
+ really need to go there).</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">html-annoyances</i></span></dt>
+ <dd>
+ <p>This filter will undo many common instances of HTML based abuse.</p>
+ <p>The <tt class="LITERAL">BLINK</tt> and <tt class="LITERAL">MARQUEE</tt> tags are neutralized (yeah
+ baby!), and browser windows will be created as resizeable (as of course they should be!), and will have
+ location, scroll and menu bars -- even if specified otherwise.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">content-cookies</i></span></dt>
+ <dd>
+ <p>Most cookies are set in the HTTP dialog, where they can be intercepted by the <tt class=
+ "LITERAL"><a href="actions-file.html#CRUNCH-INCOMING-COOKIES">crunch-incoming-cookies</a></tt> and
+ <tt class="LITERAL"><a href="actions-file.html#CRUNCH-OUTGOING-COOKIES">crunch-outgoing-cookies</a></tt>
+ actions. But web sites increasingly make use of HTML meta tags and JavaScript to sneak cookies to the
+ browser on the content level.</p>
+ <p>This filter disables most HTML and JavaScript code that reads or sets cookies. It cannot detect all
+ clever uses of these types of code, so it should not be relied on as an absolute fix. Use it wherever you
+ would also use the cookie crunch actions.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">refresh-tags</i></span></dt>
+ <dd>
+ <p>Disable any refresh tags if the interval is greater than nine seconds (so that redirections done via
+ refresh tags are not destroyed). This is useful for dial-on-demand setups, or for those who find this HTML
+ feature annoying.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">unsolicited-popups</i></span></dt>
+ <dd>
+ <p>This filter attempts to prevent only <span class="QUOTE">"unsolicited"</span> pop-up windows from
+ opening, yet still allow pop-up windows that the user has explicitly chosen to open. It was added in
+ version 3.0.1, as an improvement over earlier such filters.</p>
+ <p>Technical note: The filter works by redefining the window.open JavaScript function to a dummy function,
+ <tt class="LITERAL">PrivoxyWindowOpen()</tt>, during the loading and rendering phase of each HTML page
+ access, and restoring the function afterward.</p>
+ <p>This is recommended only for browsers that cannot perform this function reliably themselves. And be
+ aware that some sites require such windows in order to function normally. Use with caution.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">all-popups</i></span></dt>
+ <dd>
+ <p>Attempt to prevent <span class="emphasis"><i class="EMPHASIS">all</i></span> pop-up windows from
+ opening. Note this should be used with even more discretion than the above, since it is more likely to
+ break some sites that require pop-ups for normal usage. Use with caution.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">img-reorder</i></span></dt>
+ <dd>
+ <p>This is a helper filter that has no value if used alone. It makes the <tt class=
+ "LITERAL">banners-by-size</tt> and <tt class="LITERAL">banners-by-link</tt> (see below) filters more
+ effective and should be enabled together with them.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">banners-by-size</i></span></dt>
+ <dd>
+ <p>This filter removes image tags purely based on what size they are. Fortunately for us, many ads and
+ banner images tend to conform to certain standardized sizes, which makes this filter quite effective for ad
+ stripping purposes.</p>
+ <p>Occasionally this filter will cause false positives on images that are not ads, but just happen to be of
+ one of the standard banner sizes.</p>
+ <p>Recommended only for those who require extreme ad blocking. The default block rules should catch 95+% of
+ all ads <span class="emphasis"><i class="EMPHASIS">without</i></span> this filter enabled.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">banners-by-link</i></span></dt>
+ <dd>
+ <p>This is an experimental filter that attempts to kill any banners if their URLs seem to point to known or
+ suspected click trackers. It is currently not of much value and is not recommended for use by default.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">webbugs</i></span></dt>
+ <dd>
+ <p>Webbugs are small, invisible images (technically 1X1 GIF images), that are used to track users across
+ websites, and collect information on them. As an HTML page is loaded by the browser, an embedded image tag
+ causes the browser to contact a third-party site, disclosing the tracking information through the requested
+ URL and/or cookies for that third-party domain, without the user ever becoming aware of the interaction
+ with the third-party site. HTML-ized spam also uses a similar technique to verify email addresses.</p>
+ <p>This filter removes the HTML code that loads such <span class="QUOTE">"webbugs"</span>.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">tiny-textforms</i></span></dt>
+ <dd>
+ <p>A rather special-purpose filter that can be used to enlarge textareas (those multi-line text boxes in
+ web forms) and turn off hard word wrap in them. It was written for the sourceforge.net tracker system where
+ such boxes are a nuisance, but it can be handy on other sites, too.</p>
+ <p>It is not recommended to use this filter as a default.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">jumping-windows</i></span></dt>
+ <dd>
+ <p>Many consider windows that move, or resize themselves to be abusive. This filter neutralizes the related
+ JavaScript code. Note that some sites might not display or behave as intended when using this filter. Use
+ with caution.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">frameset-borders</i></span></dt>
+ <dd>
+ <p>Some web designers seem to assume that everyone in the world will view their web sites using the same
+ browser brand and version, screen resolution etc, because only that assumption could explain why they'd use
+ static frame sizes, yet prevent their frames from being resized by the user, should they be too small to
+ show their whole content.</p>
+ <p>This filter removes the related HTML code. It should only be applied to sites which need it.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">demoronizer</i></span></dt>
+ <dd>
+ <p>Many Microsoft products that generate HTML use non-standard extensions (read: violations) of the ISO
+ 8859-1 aka Latin-1 character set. This can cause those HTML documents to display with errors on
+ standard-compliant platforms.</p>
+ <p>This filter translates the MS-only characters into Latin-1 equivalents. It is not necessary when using
+ MS products, and will cause corruption of all documents that use 8-bit character sets other than Latin-1.
+ It's mostly worthwhile for Europeans on non-MS platforms, if weird garbage characters sometimes appear on
+ some pages, or user agents that don't correct for this on the fly.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">shockwave-flash</i></span></dt>
+ <dd>
+ <p>A filter for shockwave haters. As the name suggests, this filter strips code out of web pages that is
+ used to embed shockwave flash objects.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">quicktime-kioskmode</i></span></dt>
+ <dd>
+ <p>Change HTML code that embeds Quicktime objects so that kioskmode, which prevents saving, is
+ disabled.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">fun</i></span></dt>
+ <dd>
+ <p>Text replacements for subversive browsing fun. Make fun of your favorite Monopolist or play buzzword
+ bingo.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">crude-parental</i></span></dt>
+ <dd>
+ <p>A demonstration-only filter that shows how <span class="APPLICATION">Privoxy</span> can be used to
+ delete web content on a keyword basis.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">ie-exploits</i></span></dt>
+ <dd>
+ <p>An experimental collection of text replacements to disable malicious HTML and JavaScript code that
+ exploits known security holes in Internet Explorer.</p>
+ <p>Presently, it only protects against Nimda and a cross-site scripting bug, and would need active
+ maintenance to provide more substantial protection.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">site-specifics</i></span></dt>
+ <dd>
+ <p>Some web sites have very specific problems, the cure for which doesn't apply anywhere else, or could
+ even cause damage on other sites.</p>
+ <p>This is a collection of such site-specific cures which should only be applied to the sites they were
+ intended for, which is what the supplied <tt class="FILENAME">default.action</tt> file does. Users
+ shouldn't need to change anything regarding this filter.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">google</i></span></dt>
+ <dd>
+ <p>A CSS based block for Google text ads. Also removes a width limitation and the toolbar
+ advertisement.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">yahoo</i></span></dt>
+ <dd>
+ <p>Another CSS based block, this time for Yahoo text ads. And removes a width limitation as well.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">msn</i></span></dt>
+ <dd>
+ <p>Another CSS based block, this time for MSN text ads. And removes tracking URLs, as well as a width
+ limitation.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">blogspot</i></span></dt>
+ <dd>
+ <p>Cleans up some Blogspot blogs. Read the fine print before using this one!</p>
+ <p>This filter also intentionally removes some navigation stuff and sets the page width to 100%. As a
+ result, some rounded <span class="QUOTE">"corners"</span> would appear to early or not at all and as fixing
+ this would require a browser that understands background-size (CSS3), they are removed instead.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">xml-to-html</i></span></dt>
+ <dd>
+ <p>Server-header filter to change the Content-Type from xml to html.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">html-to-xml</i></span></dt>
+ <dd>
+ <p>Server-header filter to change the Content-Type from html to xml.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">no-ping</i></span></dt>
+ <dd>
+ <p>Removes the non-standard <tt class="LITERAL">ping</tt> attribute from anchor and area HTML tags.</p>
+ </dd>
+ <dt><span class="emphasis"><i class="EMPHASIS">hide-tor-exit-notation</i></span></dt>
+ <dd>
+ <p>Client-header filter to remove the <b class="COMMAND">Tor</b> exit node notation found in Host and
+ Referer headers.</p>
+ <p>If <span class="APPLICATION">Privoxy</span> and <b class="COMMAND">Tor</b> are chained and <span class=
+ "APPLICATION">Privoxy</span> is configured to use socks4a, one can use <span class=
+ "QUOTE">"http://www.example.org.foobar.exit/"</span> to access the host <span class=
+ "QUOTE">"www.example.org"</span> through the <b class="COMMAND">Tor</b> exit node <span class=
+ "QUOTE">"foobar"</span>.</p>
+ <p>As the HTTP client isn't aware of this notation, it treats the whole string <span class=
+ "QUOTE">"www.example.org.foobar.exit"</span> as host and uses it for the <span class="QUOTE">"Host"</span>
+ and <span class="QUOTE">"Referer"</span> headers. From the server's point of view the resulting headers are
+ invalid and can cause problems.</p>
+ <p>An invalid <span class="QUOTE">"Referer"</span> header can trigger <span class=
+ "QUOTE">"hot-linking"</span> protections, an invalid <span class="QUOTE">"Host"</span> header will make it
+ impossible for the server to find the right vhost (several domains hosted on the same IP address).</p>
+ <p>This client-header filter removes the <span class="QUOTE">"foo.exit"</span> part in those headers to
+ prevent the mentioned problems. Note that it only modifies the HTTP headers, it doesn't make it impossible
+ for the server to detect your <b class="COMMAND">Tor</b> exit node based on the IP address the request is
+ coming from.</p>
+ </dd>
+ </dl>
+ </div>
+ </div>
+ <div class="SECT2">
+ <h2 class="SECT2"><a name="EXTERNAL-FILTER-SYNTAX" id="EXTERNAL-FILTER-SYNTAX">9.3. External filter
+ syntax</a></h2>
+ <p>External filters are scripts or programs that can modify the content in case common <tt class=
+ "LITERAL"><a href="actions-file.html#FILTER">filters</a></tt> aren't powerful enough.</p>
+ <p>External filters can be written in any language the platform <span class="APPLICATION">Privoxy</span> runs on
+ supports.</p>
+ <p>They are controlled with the <tt class="LITERAL"><a href=
+ "actions-file.html#EXTERNAL-FILTER">external-filter</a></tt> action and have to be defined in the <tt class=
+ "LITERAL"><a href="config.html#FILTERFILE">filterfile</a></tt> first.</p>
+ <p>The header looks like any other filter, but instead of pcrs jobs, external filters contain a single job which
+ can be a program or a shell script (which may call other scripts or programs).</p>
+ <p>External filters read the content from STDIN and write the rewritten content to STDOUT. The environment
+ variables PRIVOXY_URL, PRIVOXY_PATH, PRIVOXY_HOST, PRIVOXY_ORIGIN, PRIVOXY_LISTEN_ADDRESS can be used to get some
+ details about the client request.</p>
+ <p><span class="APPLICATION">Privoxy</span> will temporary store the content to filter in the <tt class=
+ "LITERAL"><a href="config.html#TEMPORARY-DIRECTORY">temporary-directory</a></tt>.</p>
+ <table border="0" bgcolor="#E0E0E0" width="100%">
<tr>
- <td width="33%" align="left" valign="top">
- Actions Files
- </td>
- <td width="34%" align="center" valign="top">
-
- </td>
- <td width="33%" align="right" valign="top">
- Privoxy's Template Files
+ <td>
+ <pre class="SCREEN">EXTERNAL-FILTER: cat Pointless example filter that doesn't actually modify the content
+/bin/cat
+
+# Incorrect reimplementation of the filter above in POSIX shell.
+#
+# Note that it's a single job that spans multiple lines, the line
+# breaks are not passed to the shell, thus the semicolons are required.
+#
+# If the script isn't trivial, it is recommended to put it into an external file.
+#
+# In general, writing external filters entirely in POSIX shell is not
+# considered a good idea.
+EXTERNAL-FILTER: cat2 Pointless example filter that despite its name may actually modify the content
+while read line; \
+do \
+ echo "$line"; \
+done
+
+EXTERNAL-FILTER: rotate-image Rotate an image by 180 degree. Test filter with limited value.
+/usr/local/bin/convert - -rotate 180 -
+
+EXTERNAL-FILTER: citation-needed Adds a "[citation needed]" tag to an image. The coordinates may need adjustment.
+/usr/local/bin/convert - -pointsize 16 -fill white -annotate +17+418 "[citation needed]" -</pre>
</td>
</tr>
</table>
+ <div class="WARNING">
+ <table class="WARNING" border="1" width="100%">
+ <tr>
+ <td align="center"><b>Warning</b></td>
+ </tr>
+ <tr>
+ <td align="left">
+ <p>Currently external filters are executed with <span class="APPLICATION">Privoxy</span>'s privileges!
+ Only use external filters you understand and trust.</p>
+ </td>
+ </tr>
+ </table>
+ </div>
+ <p>External filters are experimental and the syntax may change in the future.</p>
</div>
- </body>
+ </div>
+ <div class="NAVFOOTER">
+ <hr align="left" width="100%">
+ <table summary="Footer navigation table" width="100%" border="0" cellpadding="0" cellspacing="0">
+ <tr>
+ <td width="33%" align="left" valign="top"><a href="actions-file.html" accesskey="P">Prev</a></td>
+ <td width="34%" align="center" valign="top"><a href="index.html" accesskey="H">Home</a></td>
+ <td width="33%" align="right" valign="top"><a href="templates.html" accesskey="N">Next</a></td>
+ </tr>
+ <tr>
+ <td width="33%" align="left" valign="top">Actions Files</td>
+ <td width="34%" align="center" valign="top"> </td>
+ <td width="33%" align="right" valign="top">Privoxy's Template Files</td>
+ </tr>
+ </table>
+ </div>
+</body>
</html>
-