1 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
2 "http://www.w3.org/TR/html4/loose.dtd">
6 <title>Appendix</title>
7 <meta name="GENERATOR" content=
8 "Modular DocBook HTML Stylesheet Version 1.79">
9 <link rel="HOME" title="Privoxy 3.0.20 User Manual" href="index.html">
10 <link rel="PREVIOUS" title="See Also" href="seealso.html">
11 <link rel="STYLESHEET" type="text/css" href="../p_doc.css">
12 <meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
13 <link rel="STYLESHEET" type="text/css" href="p_doc.css">
14 <style type="text/css">
16 background-color: #EEEEEE;
19 :link { color: #0000FF }
20 :visited { color: #840084 }
21 :active { color: #0000FF }
22 table.c3 {background-color: #E0E0E0}
23 span.c2 {font-style: italic}
24 hr.c1 {text-align: left}
29 <div class="NAVHEADER">
30 <table summary="Header navigation table" width="100%" border="0"
31 cellpadding="0" cellspacing="0">
33 <th colspan="3" align="center">Privoxy 3.0.20 User Manual</th>
37 <td width="10%" align="left" valign="bottom"><a href="seealso.html"
38 accesskey="P">Prev</a></td>
40 <td width="80%" align="center" valign="bottom"></td>
42 <td width="10%" align="right" valign="bottom"> </td>
45 <hr class="c1" width="100%">
49 <h1 class="SECT1"><a name="APPENDIX" id="APPENDIX">14. Appendix</a></h1>
52 <h2 class="SECT2"><a name="REGEX" id="REGEX">14.1. Regular
55 <p><span class="APPLICATION">Privoxy</span> uses Perl-style
56 <span class="QUOTE">"regular expressions"</span> in its <a href=
57 "actions-file.html">actions files</a> and <a href=
58 "filter-file.html">filter file</a>, through the <a href=
59 "http://www.pcre.org/" target="_top">PCRE</a> and <span class=
60 "APPLICATION">PCRS</span> libraries.</p>
62 <p>If you are reading this, you probably don't understand what
63 <span class="QUOTE">"regular expressions"</span> are, or what they can
64 do. So this will be a very brief introduction only. A full explanation
65 would require a <a href="http://www.oreilly.com/catalog/regex/" target=
66 "_top">book</a> ;-)</p>
68 <p>Regular expressions provide a language to describe patterns that can
69 be run against strings of characters (letter, numbers, etc), to see if
70 they match the string or not. The patterns are themselves (sometimes
71 complex) strings of literal characters, combined with wild-cards, and
72 other special characters, called meta-characters. The <span class=
73 "QUOTE">"meta-characters"</span> have special meanings and are used to
74 build complex patterns to be matched against. Perl Compatible Regular
75 Expressions are an especially convenient <span class=
76 "QUOTE">"dialect"</span> of the regular expression language.</p>
78 <p>To make a simple analogy, we do something similar when we use
79 wild-card characters when listing files with the <b class=
80 "COMMAND">dir</b> command in DOS. <tt class="LITERAL">*.*</tt> matches
81 all filenames. The <span class="QUOTE">"special"</span> character here
82 is the asterisk which matches any and all characters. We can be more
83 specific and use <tt class="LITERAL">?</tt> to match just individual
84 characters. So <span class="QUOTE">"dir file?.text"</span> would match
85 <span class="QUOTE">"file1.txt"</span>, <span class=
86 "QUOTE">"file2.txt"</span>, etc. We are pattern matching, using a
87 similar technique to <span class="QUOTE">"regular
88 expressions"</span>!</p>
90 <p>Regular expressions do essentially the same thing, but are much,
91 much more powerful. There are many more <span class="QUOTE">"special
92 characters"</span> and ways of building complex patterns however. Let's
93 look at a few of the common ones, and then some examples:</p>
98 <td><span class="emphasis EMPHASIS c2">.</span> - Matches any
99 single character, e.g. <span class="QUOTE">"a"</span>,
100 <span class="QUOTE">"A"</span>, <span class="QUOTE">"4"</span>,
101 <span class="QUOTE">":"</span>, or <span class=
102 "QUOTE">"@"</span>.</td>
110 <td><span class="emphasis EMPHASIS c2">?</span> - The preceding
111 character or expression is matched ZERO or ONE times.
120 <td><span class="emphasis EMPHASIS c2">+</span> - The preceding
121 character or expression is matched ONE or MORE times.</td>
129 <td><span class="emphasis EMPHASIS c2">*</span> - The preceding
130 character or expression is matched ZERO or MORE times.</td>
138 <td><span class="emphasis EMPHASIS c2">\</span> - The
139 <span class="QUOTE">"escape"</span> character denotes that the
140 following character should be taken literally. This is used where
141 one of the special characters (e.g. <span class=
142 "QUOTE">"."</span>) needs to be taken literally and not as a
143 special meta-character. Example: <span class=
144 "QUOTE">"example\.com"</span>, makes sure the period is
145 recognized only as a period (and not expanded to its
146 meta-character meaning of any single character).</td>
154 <td><span class="emphasis EMPHASIS c2">[ ]</span> - Characters
155 enclosed in brackets will be matched if any of the enclosed
156 characters are encountered. For instance, <span class=
157 "QUOTE">"[0-9]"</span> matches any numeric digit (zero through
158 nine). As an example, we can combine this with <span class=
159 "QUOTE">"+"</span> to match any digit one of more times:
160 <span class="QUOTE">"[0-9]+"</span>.</td>
168 <td><span class="emphasis EMPHASIS c2">( )</span> - parentheses
169 are used to group a sub-expression, or multiple
170 sub-expressions.</td>
178 <td><span class="emphasis EMPHASIS c2">|</span> - The
179 <span class="QUOTE">"bar"</span> character works like an
180 <span class="QUOTE">"or"</span> conditional statement. A match is
181 successful if the sub-expression on either side of <span class=
182 "QUOTE">"|"</span> matches. As an example: <span class=
183 "QUOTE">"/(this|that) example/"</span> uses grouping and the bar
184 character and would match either <span class="QUOTE">"this
185 example"</span> or <span class="QUOTE">"that example"</span>, and
191 <p>These are just some of the ones you are likely to use when matching
192 URLs with <span class="APPLICATION">Privoxy</span>, and is a long way
193 from a definitive list. This is enough to get us started with a few
194 simple examples which may be more illuminating:</p>
196 <p><span class="emphasis EMPHASIS c2"><tt class=
197 "LITERAL">/.*/banners/.*</tt></span> - A simple example that uses the
198 common combination of <span class="QUOTE">"."</span> and <span class=
199 "QUOTE">"*"</span> to denote any character, zero or more times. In
200 other words, any string at all. So we start with a literal forward
201 slash, then our regular expression pattern (<span class=
202 "QUOTE">".*"</span>) another literal forward slash, the string
203 <span class="QUOTE">"banners"</span>, another forward slash, and lastly
204 another <span class="QUOTE">".*"</span>. We are building a directory
205 path here. This will match any file with the path that has a directory
206 named <span class="QUOTE">"banners"</span> in it. The <span class=
207 "QUOTE">".*"</span> matches any characters, and this could conceivably
208 be more forward slashes, so it might expand into a much longer looking
209 path. For example, this could match: <span class=
210 "QUOTE">"/eye/hate/spammers/banners/annoy_me_please.gif"</span>, or
211 just <span class="QUOTE">"/banners/annoying.html"</span>, or almost an
212 infinite number of other possible combinations, just so it has
213 <span class="QUOTE">"banners"</span> in the path somewhere.</p>
215 <p>And now something a little more complex:</p>
217 <p><span class="emphasis EMPHASIS c2"><tt class=
218 "LITERAL">/.*/adv((er)?ts?|ertis(ing|ements?))?/</tt></span> - We have
219 several literal forward slashes again (<span class="QUOTE">"/"</span>),
220 so we are building another expression that is a file path statement. We
221 have another <span class="QUOTE">".*"</span>, so we are matching
222 against any conceivable sub-path, just so it matches our expression.
223 The only true literal that <span class="emphasis EMPHASIS c2">must
224 match</span> our pattern is <span class="APPLICATION">adv</span>,
225 together with the forward slashes. What comes after the <span class=
226 "QUOTE">"adv"</span> string is the interesting part.</p>
228 <p>Remember the <span class="QUOTE">"?"</span> means the preceding
229 expression (either a literal character or anything grouped with
230 <span class="QUOTE">"(...)"</span> in this case) can exist or not,
231 since this means either zero or one match. So <span class=
232 "QUOTE">"((er)?ts?|ertis(ing|ements?))"</span> is optional, as are the
233 individual sub-expressions: <span class="QUOTE">"(er)"</span>,
234 <span class="QUOTE">"(ing|ements?)"</span>, and the <span class=
235 "QUOTE">"s"</span>. The <span class="QUOTE">"|"</span> means
236 <span class="QUOTE">"or"</span>. We have two of those. For instance,
237 <span class="QUOTE">"(ing|ements?)"</span>, can expand to match either
238 <span class="QUOTE">"ing"</span> <span class=
239 "emphasis EMPHASIS c2">OR</span> <span class="QUOTE">"ements?"</span>.
240 What is being done here, is an attempt at matching as many variations
241 of <span class="QUOTE">"advertisement"</span>, and similar, as
242 possible. So this would expand to match just <span class=
243 "QUOTE">"adv"</span>, or <span class="QUOTE">"advert"</span>, or
244 <span class="QUOTE">"adverts"</span>, or <span class=
245 "QUOTE">"advertising"</span>, or <span class=
246 "QUOTE">"advertisement"</span>, or <span class=
247 "QUOTE">"advertisements"</span>. You get the idea. But it would not
248 match <span class="QUOTE">"advertizements"</span> (with a <span class=
249 "QUOTE">"z"</span>). We could fix that by changing our regular
250 expression to: <span class=
251 "QUOTE">"/.*/adv((er)?ts?|erti(s|z)(ing|ements?))?/"</span>, which
252 would then match either spelling.</p>
254 <p><span class="emphasis EMPHASIS c2"><tt class=
255 "LITERAL">/.*/advert[0-9]+\.(gif|jpe?g)</tt></span> - Again another
256 path statement with forward slashes. Anything in the square brackets
257 <span class="QUOTE">"[ ]"</span> can be matched. This is using
258 <span class="QUOTE">"0-9"</span> as a shorthand expression to mean any
259 digit one through nine. It is the same as saying <span class=
260 "QUOTE">"0123456789"</span>. So any digit matches. The <span class=
261 "QUOTE">"+"</span> means one or more of the preceding expression must
262 be included. The preceding expression here is what is in the square
263 brackets -- in this case, any digit one through nine. Then, at the end,
264 we have a grouping: <span class="QUOTE">"(gif|jpe?g)"</span>. This
265 includes a <span class="QUOTE">"|"</span>, so this needs to match the
266 expression on either side of that bar character also. A simple
267 <span class="QUOTE">"gif"</span> on one side, and the other side will
268 in turn match either <span class="QUOTE">"jpeg"</span> or <span class=
269 "QUOTE">"jpg"</span>, since the <span class="QUOTE">"?"</span> means
270 the letter <span class="QUOTE">"e"</span> is optional and can be
271 matched once or not at all. So we are building an expression here to
272 match image GIF or JPEG type image file. It must include the literal
273 string <span class="QUOTE">"advert"</span>, then one or more digits,
274 and a <span class="QUOTE">"."</span> (which is now a literal, and not a
275 special character, since it is escaped with <span class=
276 "QUOTE">"\"</span>), and lastly either <span class=
277 "QUOTE">"gif"</span>, or <span class="QUOTE">"jpeg"</span>, or
278 <span class="QUOTE">"jpg"</span>. Some possible matches would include:
279 <span class="QUOTE">"//advert1.jpg"</span>, <span class=
280 "QUOTE">"/nasty/ads/advert1234.gif"</span>, <span class=
281 "QUOTE">"/banners/from/hell/advert99.jpg"</span>. It would not match
282 <span class="QUOTE">"advert1.gif"</span> (no leading slash), or
283 <span class="QUOTE">"/adverts232.jpg"</span> (the expression does not
284 include an <span class="QUOTE">"s"</span>), or <span class=
285 "QUOTE">"/advert1.jsp"</span> (<span class="QUOTE">"jsp"</span> is not
286 in the expression anywhere).</p>
288 <p>We are barely scratching the surface of regular expressions here so
289 that you can understand the default <span class=
290 "APPLICATION">Privoxy</span> configuration files, and maybe use this
291 knowledge to customize your own installation. There is much, much more
292 that can be done with regular expressions. Now that you know enough to
293 get started, you can learn more on your own :/</p>
295 <p>More reading on Perl Compatible Regular expressions: <a href=
296 "http://perldoc.perl.org/perlre.html" target=
297 "_top">http://perldoc.perl.org/perlre.html</a></p>
299 <p>For information on regular expression based substitutions and their
300 applications in filters, please see the <a href=
301 "filter-file.html">filter file tutorial</a> in this manual.</p>
305 <h2 class="SECT2"><a name="AEN5814" id="AEN5814">14.2. Privoxy's
306 Internal Pages</a></h2>
308 <p>Since <span class="APPLICATION">Privoxy</span> proxies each
309 requested web page, it is easy for <span class=
310 "APPLICATION">Privoxy</span> to trap certain special URLs. In this way,
311 we can talk directly to <span class="APPLICATION">Privoxy</span>, and
312 see how it is configured, see how our rules are being applied, change
313 these rules and other configuration options, and even turn <span class=
314 "APPLICATION">Privoxy's</span> filtering off, all with a web
317 <p>The URLs listed below are the special ones that allow direct access
318 to <span class="APPLICATION">Privoxy</span>. Of course, <span class=
319 "APPLICATION">Privoxy</span> must be running to access these. If not,
320 you will get a friendly error message. Internet access is not necessary
325 <p>Privoxy main page:</p><a name="AEN5828" id="AEN5828"></a>
327 <blockquote class="BLOCKQUOTE">
328 <p><a href="http://config.privoxy.org/" target=
329 "_top">http://config.privoxy.org/</a></p>
332 <p>There is a shortcut: <a href="http://p.p/" target=
333 "_top">http://p.p/</a> (But it doesn't provide a fall-back to a
334 real page, in case the request is not sent through <span class=
335 "APPLICATION">Privoxy</span>)</p>
339 <p>Show information about the current configuration, including
340 viewing and editing of actions files:</p><a name="AEN5836" id=
343 <blockquote class="BLOCKQUOTE">
344 <p><a href="http://config.privoxy.org/show-status" target=
345 "_top">http://config.privoxy.org/show-status</a></p>
350 <p>Show the source code version numbers:</p><a name="AEN5841" id=
353 <blockquote class="BLOCKQUOTE">
354 <p><a href="http://config.privoxy.org/show-version" target=
355 "_top">http://config.privoxy.org/show-version</a></p>
360 <p>Show the browser's request headers:</p><a name="AEN5846" id=
363 <blockquote class="BLOCKQUOTE">
364 <p><a href="http://config.privoxy.org/show-request" target=
365 "_top">http://config.privoxy.org/show-request</a></p>
370 <p>Show which actions apply to a URL and why:</p><a name="AEN5851"
373 <blockquote class="BLOCKQUOTE">
374 <p><a href="http://config.privoxy.org/show-url-info" target=
375 "_top">http://config.privoxy.org/show-url-info</a></p>
380 <p>Toggle Privoxy on or off. This feature can be turned off/on in
381 the main <tt class="FILENAME">config</tt> file. When toggled
382 <span class="QUOTE">"off"</span>, <span class=
383 "QUOTE">"Privoxy"</span> continues to run, but only as a
384 pass-through proxy, with no actions taking place:</p><a name=
385 "AEN5859" id="AEN5859"></a>
387 <blockquote class="BLOCKQUOTE">
388 <p><a href="http://config.privoxy.org/toggle" target=
389 "_top">http://config.privoxy.org/toggle</a></p>
392 <p>Short cuts. Turn off, then on:</p><a name="AEN5863" id=
395 <blockquote class="BLOCKQUOTE">
396 <p><a href="http://config.privoxy.org/toggle?set=disable" target=
397 "_top">http://config.privoxy.org/toggle?set=disable</a></p>
398 </blockquote><a name="AEN5866" id="AEN5866"></a>
400 <blockquote class="BLOCKQUOTE">
401 <p><a href="http://config.privoxy.org/toggle?set=enable" target=
402 "_top">http://config.privoxy.org/toggle?set=enable</a></p>
407 <p>These may be bookmarked for quick reference. See next.</p>
410 <h3 class="SECT3"><a name="BOOKMARKLETS" id="BOOKMARKLETS">14.2.1.
411 Bookmarklets</a></h3>
413 <p>Below are some <span class="QUOTE">"bookmarklets"</span> to allow
414 you to easily access a <span class="QUOTE">"mini"</span> version of
415 some of <span class="APPLICATION">Privoxy's</span> special pages.
416 They are designed for MS Internet Explorer, but should work equally
417 well in Netscape, Mozilla, and other browsers which support
418 JavaScript. They are designed to run directly from your bookmarks -
419 not by clicking the links below (although that should work for
422 <p>To save them, right-click the link and choose <span class=
423 "QUOTE">"Add to Favorites"</span> (IE) or <span class="QUOTE">"Add
424 Bookmark"</span> (Netscape). You will get a warning that the bookmark
425 <span class="QUOTE">"may not be safe"</span> - just click OK. Then
426 you can run the Bookmarklet directly from your favorites/bookmarks.
427 For even faster access, you can put them on the <span class=
428 "QUOTE">"Links"</span> bar (IE) or the <span class="QUOTE">"Personal
429 Toolbar"</span> (Netscape), and run them with a single click.</p>
434 "javascript:void(window.open('http://config.privoxy.org/toggle?mini=y&set=enabled','ijbstatus','width=250,height=100,resizable=yes,scrollbars=no,toolbar=no,location=no,directories=no,status=no,menubar=no,copyhistory=no').focus());"
435 target="_top">Privoxy - Enable</a></p>
440 "javascript:void(window.open('http://config.privoxy.org/toggle?mini=y&set=disabled','ijbstatus','width=250,height=100,resizable=yes,scrollbars=no,toolbar=no,location=no,directories=no,status=no,menubar=no,copyhistory=no').focus());"
441 target="_top">Privoxy - Disable</a></p>
446 "javascript:void(window.open('http://config.privoxy.org/toggle?mini=y&set=toggle','ijbstatus','width=250,height=100,resizable=yes,scrollbars=no,toolbar=no,location=no,directories=no,status=no,menubar=no,copyhistory=no').focus());"
447 target="_top">Privoxy - Toggle Privoxy</a> (Toggles between
448 enabled and disabled)</p>
453 "javascript:void(window.open('http://config.privoxy.org/toggle?mini=y','ijbstatus','width=250,height=2,resizable=yes,scrollbars=no,toolbar=no,location=no,directories=no,status=no,menubar=no,copyhistory=no').focus());"
454 target="_top">Privoxy- View Status</a></p>
459 "javascript:void(window.open('http://config.privoxy.org/show-url-info?url='+escape(location.href),'Why').focus());"
460 target="_top">Privoxy - Why?</a></p>
464 <p>Credit: The site which gave us the general idea for these
465 bookmarklets is <a href="http://www.bookmarklets.com/" target=
466 "_top">www.bookmarklets.com</a>. They have more information about
472 <h2 class="SECT2"><a name="CHAIN" id="CHAIN">14.3. Chain of
475 <p>Let's take a quick look at how some of <span class=
476 "APPLICATION">Privoxy's</span> core features are triggered, and the
477 ensuing sequence of events when a web page is requested by your
482 <p>First, your web browser requests a web page. The browser knows
483 to send the request to <span class="APPLICATION">Privoxy</span>,
484 which will in turn, relay the request to the remote web server
485 after passing the following tests:</p>
489 <p><span class="APPLICATION">Privoxy</span> traps any request for
490 its own internal CGI pages (e.g <a href="http://p.p/" target=
491 "_top">http://p.p/</a>) and sends the CGI page back to the
496 <p>Next, <span class="APPLICATION">Privoxy</span> checks to see if
497 the URL matches any <a href="actions-file.html#BLOCK"><span class=
498 "QUOTE">"+block"</span></a> patterns. If so, the URL is then
499 blocked, and the remote web server will not be contacted. <a href=
500 "actions-file.html#HANDLE-AS-IMAGE"><span class=
501 "QUOTE">"+handle-as-image"</span></a> and <a href=
502 "actions-file.html#HANDLE-AS-EMPTY-DOCUMENT"><span class=
503 "QUOTE">"+handle-as-empty-document"</span></a> are then checked,
504 and if there is no match, an HTML <span class=
505 "QUOTE">"BLOCKED"</span> page is sent back to the browser.
506 Otherwise, if it does match, an image is returned for the former,
507 and an empty text document for the latter. The type of image would
508 depend on the setting of <a href=
509 "actions-file.html#SET-IMAGE-BLOCKER"><span class=
510 "QUOTE">"+set-image-blocker"</span></a> (blank, checkerboard
511 pattern, or an HTTP redirect to an image elsewhere).</p>
515 <p>Untrusted URLs are blocked. If URLs are being added to the
516 <tt class="FILENAME">trust</tt> file, then that is done.</p>
520 <p>If the URL pattern matches the <a href=
521 "actions-file.html#FAST-REDIRECTS"><span class=
522 "QUOTE">"+fast-redirects"</span></a> action, it is then processed.
523 Unwanted parts of the requested URL are stripped.</p>
527 <p>Now the rest of the client browser's request headers are
528 processed. If any of these match any of the relevant actions (e.g.
529 <a href="actions-file.html#HIDE-USER-AGENT"><span class=
530 "QUOTE">"+hide-user-agent"</span></a>, etc.), headers are
531 suppressed or forged as determined by these actions and their
536 <p>Now the web server starts sending its response back (i.e.
537 typically a web page).</p>
541 <p>First, the server headers are read and processed to determine,
542 among other things, the MIME type (document type) and encoding. The
543 headers are then filtered as determined by the <a href=
544 "actions-file.html#CRUNCH-INCOMING-COOKIES"><span class=
545 "QUOTE">"+crunch-incoming-cookies"</span></a>, <a href=
546 "actions-file.html#SESSION-COOKIES-ONLY"><span class=
547 "QUOTE">"+session-cookies-only"</span></a>, and <a href=
548 "actions-file.html#DOWNGRADE-HTTP-VERSION"><span class=
549 "QUOTE">"+downgrade-http-version"</span></a> actions.</p>
553 <p>If any <a href="actions-file.html#FILTER"><span class=
554 "QUOTE">"+filter"</span></a> action or <a href=
555 "actions-file.html#DEANIMATE-GIFS"><span class=
556 "QUOTE">"+deanimate-gifs"</span></a> action applies (and the
557 document type fits the action), the rest of the page is read into
558 memory (up to a configurable limit). Then the filter rules (from
559 <tt class="FILENAME">default.filter</tt> and any other filter
560 files) are processed against the buffered content. Filters are
561 applied in the order they are specified in one of the filter files.
562 Animated GIFs, if present, are reduced to either the first or last
563 frame, depending on the action setting.The entire page, which is
564 now filtered, is then sent by <span class=
565 "APPLICATION">Privoxy</span> back to your browser.</p>
567 <p>If neither a <a href="actions-file.html#FILTER"><span class=
568 "QUOTE">"+filter"</span></a> action or <a href=
569 "actions-file.html#DEANIMATE-GIFS"><span class=
570 "QUOTE">"+deanimate-gifs"</span></a> matches, then <span class=
571 "APPLICATION">Privoxy</span> passes the raw data through to the
572 client browser as it becomes available.</p>
576 <p>As the browser receives the now (possibly filtered) page
577 content, it reads and then requests any URLs that may be embedded
578 within the page source, e.g. ad images, stylesheets, JavaScript,
579 other HTML documents (e.g. frames), sounds, etc. For each of these
580 objects, the browser issues a separate request (this is easily
581 viewable in <span class="APPLICATION">Privoxy's</span> logs). And
582 each such request is in turn processed just as above. Note that a
583 complex web page will have many, many such embedded URLs. If these
584 secondary requests are to a different server, then quite possibly a
585 very differing set of actions is triggered.</p>
589 <p>NOTE: This is somewhat of a simplistic overview of what happens with
590 each URL request. For the sake of brevity and simplicity, we have
591 focused on <span class="APPLICATION">Privoxy's</span> core features
596 <h2 class="SECT2"><a name="ACTIONSANAT" id="ACTIONSANAT">14.4.
597 Troubleshooting: Anatomy of an Action</a></h2>
599 <p>The way <span class="APPLICATION">Privoxy</span> applies <a href=
600 "actions-file.html#ACTIONS">actions</a> and <a href=
601 "actions-file.html#FILTER">filters</a> to any given URL can be complex,
602 and not always so easy to understand what is happening. And sometimes
603 we need to be able to <span class="emphasis EMPHASIS c2">see</span>
604 just what <span class="APPLICATION">Privoxy</span> is doing.
605 Especially, if something <span class="APPLICATION">Privoxy</span> is
606 doing is causing us a problem inadvertently. It can be a little
607 daunting to look at the actions and filters files themselves, since
608 they tend to be filled with <a href="appendix.html#REGEX">regular
609 expressions</a> whose consequences are not always so obvious.</p>
611 <p>One quick test to see if <span class="APPLICATION">Privoxy</span> is
612 causing a problem or not, is to disable it temporarily. This should be
613 the first troubleshooting step. See <a href=
614 "appendix.html#BOOKMARKLETS">the Bookmarklets</a> section on a quick
615 and easy way to do this (be sure to flush caches afterward!). Looking
616 at the logs is a good idea too. (Note that both the toggle feature and
617 logging are enabled via <tt class="FILENAME">config</tt> file settings,
618 and may need to be turned <span class="QUOTE">"on"</span>.)</p>
620 <p>Another easy troubleshooting step to try is if you have done any
621 customization of your installation, revert back to the installed
622 defaults and see if that helps. There are times the developers get
623 complaints about one thing or another, and the problem is more related
624 to a customized configuration issue.</p>
626 <p><span class="APPLICATION">Privoxy</span> also provides the <a href=
627 "http://config.privoxy.org/show-url-info" target=
628 "_top">http://config.privoxy.org/show-url-info</a> page that can show
629 us very specifically how <span class="APPLICATION">actions</span> are
630 being applied to any given URL. This is a big help for
633 <p>First, enter one URL (or partial URL) at the prompt, and then
634 <span class="APPLICATION">Privoxy</span> will tell us how the current
635 configuration will handle it. This will not help with filtering effects
636 (i.e. the <a href="actions-file.html#FILTER"><span class=
637 "QUOTE">"+filter"</span></a> action) from one of the filter files since
638 this is handled very differently and not so easy to trap! It also will
639 not tell you about any other URLs that may be embedded within the URL
640 you are testing. For instance, images such as ads are expressed as URLs
641 within the raw page source of HTML pages. So you will only get info for
642 the actual URL that is pasted into the prompt area -- not any sub-URLs.
643 If you want to know about embedded URLs like ads, you will have to dig
644 those out of the HTML source. Use your browser's <span class=
645 "QUOTE">"View Page Source"</span> option for this. Or right click on
646 the ad, and grab the URL.</p>
648 <p>Let's try an example, <a href="http://google.com" target=
649 "_top">google.com</a>, and look at it one section at a time in a sample
650 configuration (your real configuration may vary):</p>
652 <table class="c3" border="0" width="100%">
656 Matches for http://www.google.com:
658 In file: default.action <span class="GUIBUTTON">[ View ]</span> <span class=
659 "GUIBUTTON">[ Edit ]</span>
661 {+change-x-forwarded-for{block}
662 +deanimate-gifs {last}
663 +fast-redirects {check-decoded-url}
664 +filter {refresh-tags}
665 +filter {img-reorder}
666 +filter {banners-by-size}
668 +filter {jumping-windows}
669 +filter {ie-exploits}
670 +hide-from-header {block}
671 +hide-referrer {forge}
672 +session-cookies-only
673 +set-image-blocker {pattern}
676 { -session-cookies-only }
682 In file: user.action <span class="GUIBUTTON">[ View ]</span> <span class=
683 "GUIBUTTON">[ Edit ]</span>
684 (no matches in this file)
690 <p>This is telling us how we have defined our <a href=
691 "actions-file.html#ACTIONS"><span class="QUOTE">"actions"</span></a>,
692 and which ones match for our test case, <span class=
693 "QUOTE">"google.com"</span>. Displayed is all the actions that are
694 available to us. Remember, the <tt class="LITERAL">+</tt> sign denotes
695 <span class="QUOTE">"on"</span>. <tt class="LITERAL">-</tt> denotes
696 <span class="QUOTE">"off"</span>. So some are <span class=
697 "QUOTE">"on"</span> here, but many are <span class=
698 "QUOTE">"off"</span>. Each example we try may provide a slightly
699 different end result, depending on our configuration directives.</p>
701 <p>The first listing is for our <tt class=
702 "FILENAME">default.action</tt> file. The large, multi-line listing, is
703 how the actions are set to match for all URLs, i.e. our default
704 settings. If you look at your <span class="QUOTE">"actions"</span>
705 file, this would be the section just below the <span class=
706 "QUOTE">"aliases"</span> section near the top. This will apply to all
707 URLs as signified by the single forward slash at the end of the listing
708 -- <span class="QUOTE">" / "</span>.</p>
710 <p>But we have defined additional actions that would be exceptions to
711 these general rules, and then we list specific URLs (or patterns) that
712 these exceptions would apply to. Last match wins. Just below this then
713 are two explicit matches for <span class="QUOTE">".google.com"</span>.
714 The first is negating our previous cookie setting, which was for
715 <a href="actions-file.html#SESSION-COOKIES-ONLY"><span class=
716 "QUOTE">"+session-cookies-only"</span></a> (i.e. not persistent). So we
717 will allow persistent cookies for google, at least that is how it is in
718 this example. The second turns <span class=
719 "emphasis EMPHASIS c2">off</span> any <a href=
720 "actions-file.html#FAST-REDIRECTS"><span class=
721 "QUOTE">"+fast-redirects"</span></a> action, allowing this to take
722 place unmolested. Note that there is a leading dot here -- <span class=
723 "QUOTE">".google.com"</span>. This will match any hosts and
724 sub-domains, in the google.com domain also, such as <span class=
725 "QUOTE">"www.google.com"</span> or <span class=
726 "QUOTE">"mail.google.com"</span>. But it would not match <span class=
727 "QUOTE">"www.google.de"</span>! So, apparently, we have these two
728 actions defined as exceptions to the general rules at the top somewhere
729 in the lower part of our <tt class="FILENAME">default.action</tt> file,
730 and <span class="QUOTE">"google.com"</span> is referenced somewhere in
731 these latter sections.</p>
733 <p>Then, for our <tt class="FILENAME">user.action</tt> file, we again
734 have no hits. So there is nothing google-specific that we might have
735 added to our own, local configuration. If there was, those actions
736 would over-rule any actions from previously processed files, such as
737 <tt class="FILENAME">default.action</tt>. <tt class=
738 "FILENAME">user.action</tt> typically has the last word. This is the
739 best place to put hard and fast exceptions,</p>
741 <p>And finally we pull it all together in the bottom section and
742 summarize how <span class="APPLICATION">Privoxy</span> is applying all
743 its <span class="QUOTE">"actions"</span> to <span class=
744 "QUOTE">"google.com"</span>:</p>
746 <table class="c3" border="0" width="100%">
754 +change-x-forwarded-for{block}
755 -client-header-filter{hide-tor-exit-notation}
756 -content-type-overwrite
757 -crunch-client-header
758 -crunch-if-none-match
759 -crunch-incoming-cookies
760 -crunch-outgoing-cookies
761 -crunch-server-header
762 +deanimate-gifs {last}
763 -downgrade-http-version
766 -filter {content-cookies}
768 -filter {banners-by-link}
769 -filter {tiny-textforms}
770 -filter {frameset-borders}
771 -filter {demoronizer}
772 -filter {shockwave-flash}
773 -filter {quicktime-kioskmode}
775 -filter {crude-parental}
776 -filter {site-specifics}
777 -filter {js-annoyances}
778 -filter {html-annoyances}
779 +filter {refresh-tags}
780 -filter {unsolicited-popups}
781 +filter {img-reorder}
782 +filter {banners-by-size}
784 +filter {jumping-windows}
785 +filter {ie-exploits}
792 -handle-as-empty-document
794 -hide-accept-language
795 -hide-content-disposition
796 +hide-from-header {block}
797 -hide-if-modified-since
798 +hide-referrer {forge}
801 -overwrite-last-modified
804 -server-header-filter{xml-to-html}
805 -server-header-filter{html-to-xml}
806 -session-cookies-only
807 +set-image-blocker {pattern}
813 <p>Notice the only difference here to the previous listing, is to
814 <span class="QUOTE">"fast-redirects"</span> and <span class=
815 "QUOTE">"session-cookies-only"</span>, which are activated specifically
816 for this site in our configuration, and thus show in the <span class=
817 "QUOTE">"Final Results"</span>.</p>
819 <p>Now another example, <span class=
820 "QUOTE">"ad.doubleclick.net"</span>:</p>
822 <table class="c3" border="0" width="100%">
826 { +block{Domains starts with "ad"} }
829 { +block{Domain contains "ad"} }
832 { +block{Doubleclick banner server} +handle-as-image }
833 .[a-vx-z]*.doubleclick.net
839 <p>We'll just show the interesting part here - the explicit matches. It
840 is matched three different times. Two <span class=
841 "QUOTE">"+block{}"</span> sections, and a <span class="QUOTE">"+block{}
842 +handle-as-image"</span>, which is the expanded form of one of our
843 aliases that had been defined as: <span class=
844 "QUOTE">"+block-as-image"</span>. (<a href=
845 "actions-file.html#ALIASES"><span class="QUOTE">"Aliases"</span></a>
846 are defined in the first section of the actions file and typically used
847 to combine more than one action.)</p>
849 <p>Any one of these would have done the trick and blocked this as an
850 unwanted image. This is unnecessarily redundant since the last case
851 effectively would also cover the first. No point in taking chances with
852 these guys though ;-) Note that if you want an ad or obnoxious URL to
853 be invisible, it should be defined as <span class=
854 "QUOTE">"ad.doubleclick.net"</span> is done here -- as both a <a href=
855 "actions-file.html#BLOCK"><span class="QUOTE">"+block{}"</span></a>
856 <span class="emphasis EMPHASIS c2">and</span> an <a href=
857 "actions-file.html#HANDLE-AS-IMAGE"><span class=
858 "QUOTE">"+handle-as-image"</span></a>. The custom alias <span class=
859 "QUOTE">"<tt class="LITERAL">+block-as-image</tt>"</span> just
860 simplifies the process and make it more readable.</p>
862 <p>One last example. Let's try <span class=
863 "QUOTE">"http://www.example.net/adsl/HOWTO/"</span>. This one is giving
864 us problems. We are getting a blank page. Hmmm ...</p>
866 <table class="c3" border="0" width="100%">
870 Matches for http://www.example.net/adsl/HOWTO/:
872 In file: default.action <span class="GUIBUTTON">[ View ]</span> <span class=
873 "GUIBUTTON">[ Edit ]</span>
877 +change-x-forwarded-for{block}
878 -client-header-filter{hide-tor-exit-notation}
879 -content-type-overwrite
880 -crunch-client-header
881 -crunch-if-none-match
882 -crunch-incoming-cookies
883 -crunch-outgoing-cookies
884 -crunch-server-header
886 -downgrade-http-version
887 +fast-redirects {check-decoded-url}
889 -filter {content-cookies}
891 -filter {banners-by-link}
892 -filter {tiny-textforms}
893 -filter {frameset-borders}
894 -filter {demoronizer}
895 -filter {shockwave-flash}
896 -filter {quicktime-kioskmode}
898 -filter {crude-parental}
899 -filter {site-specifics}
900 -filter {js-annoyances}
901 -filter {html-annoyances}
902 +filter {refresh-tags}
903 -filter {unsolicited-popups}
904 +filter {img-reorder}
905 +filter {banners-by-size}
907 +filter {jumping-windows}
908 +filter {ie-exploits}
915 -handle-as-empty-document
917 -hide-accept-language
918 -hide-content-disposition
919 +hide-from-header{block}
922 -overwrite-last-modified
925 -server-header-filter{xml-to-html}
926 -server-header-filter{html-to-xml}
927 +session-cookies-only
928 +set-image-blocker{blank} }
931 { +block{Path contains "ads".} +handle-as-image }
938 <p>Ooops, the <span class="QUOTE">"/adsl/"</span> is matching
939 <span class="QUOTE">"/ads"</span> in our configuration! But we did not
940 want this at all! Now we see why we get the blank page. It is actually
941 triggering two different actions here, and the effects are aggregated
942 so that the URL is blocked, and <span class=
943 "APPLICATION">Privoxy</span> is told to treat the block as if it were
944 an image. But this is, of course, all wrong. We could now add a new
945 action below this (or better in our own <tt class=
946 "FILENAME">user.action</tt> file) that explicitly <span class=
947 "emphasis EMPHASIS c2">un</span> blocks ( <a href=
948 "actions-file.html#BLOCK"><span class="QUOTE">"{-block}"</span></a>)
949 paths with <span class="QUOTE">"adsl"</span> in them (remember, last
950 match in the configuration wins). There are various ways to handle such
951 exceptions. Example:</p>
953 <table class="c3" border="0" width="100%">
964 <p>Now the page displays ;-) Remember to flush your browser's caches
965 when making these kinds of changes to your configuration to insure that
966 you get a freshly delivered page! Or, try using <tt class=
967 "LITERAL">Shift+Reload</tt>.</p>
969 <p>But now what about a situation where we get no explicit matches like
972 <table class="c3" border="0" width="100%">
976 { +block{Path starts with "ads".} +handle-as-image }
983 <p>That actually was very helpful and pointed us quickly to where the
984 problem was. If you don't get this kind of match, then it means one of
985 the default rules in the first section of <tt class=
986 "FILENAME">default.action</tt> is causing the problem. This would
987 require some guesswork, and maybe a little trial and error to isolate
988 the offending rule. One likely cause would be one of the <a href=
989 "actions-file.html#FILTER"><span class="QUOTE">"+filter"</span></a>
990 actions. These tend to be harder to troubleshoot. Try adding the URL
991 for the site to one of aliases that turn off <a href=
992 "actions-file.html#FILTER"><span class=
993 "QUOTE">"+filter"</span></a>:</p>
995 <table class="c3" border="0" width="100%">
1001 .worldpay.com # for quietpc.com
1010 <p><span class="QUOTE">"<tt class="LITERAL">{ shop }</tt>"</span> is an
1011 <span class="QUOTE">"alias"</span> that expands to <span class=
1012 "QUOTE">"<tt class="LITERAL">{ -filter -session-cookies-only
1013 }</tt>"</span>. Or you could do your own exception to negate
1016 <table class="c3" border="0" width="100%">
1019 <pre class="SCREEN">
1021 # Disable ALL filter actions for sites in this section
1030 <p>This would turn off all filtering for these sites. This is best put
1031 in <tt class="FILENAME">user.action</tt>, for local site exceptions.
1032 Note that when a simple domain pattern is used by itself (without the
1033 subsequent path portion), all sub-pages within that domain are included
1034 automatically in the scope of the action.</p>
1036 <p>Images that are inexplicably being blocked, may well be hitting the
1037 <a href="actions-file.html#FILTER-BANNERS-BY-SIZE"><span class=
1038 "QUOTE">"+filter{banners-by-size}"</span></a> rule, which assumes that
1039 images of certain sizes are ad banners (works well <span class=
1040 "emphasis EMPHASIS c2">most of the time</span> since these tend to be
1043 <p><span class="QUOTE">"<tt class="LITERAL">{ fragile }</tt>"</span> is
1044 an alias that disables most actions that are the most likely to cause
1045 trouble. This can be used as a last resort for problem sites.</p>
1047 <table class="c3" border="0" width="100%">
1050 <pre class="SCREEN">
1052 # Handle with care: easy to break
1060 <p><span class="emphasis EMPHASIS c2">Remember to flush caches!</span>
1061 Note that the <tt class="LITERAL">mail.google</tt> reference lacks the
1062 TLD portion (e.g. <span class="QUOTE">".com"</span>). This will
1063 effectively match any TLD with <tt class="LITERAL">google</tt> in it,
1064 such as <tt class="LITERAL">mail.google.de.</tt>, just as an
1067 <p>If this still does not work, you will have to go through the
1068 remaining actions one by one to find which one(s) is causing the
1073 <div class="NAVFOOTER">
1074 <hr class="c1" width="100%">
1076 <table summary="Footer navigation table" width="100%" border="0"
1077 cellpadding="0" cellspacing="0">
1079 <td width="33%" align="left" valign="top"><a href="seealso.html"
1080 accesskey="P">Prev</a></td>
1082 <td width="34%" align="center" valign="top"><a href="index.html"
1083 accesskey="H">Home</a></td>
1085 <td width="33%" align="right" valign="top"> </td>
1089 <td width="33%" align="left" valign="top">See Also</td>
1091 <td width="34%" align="center" valign="top"> </td>
1093 <td width="33%" align="right" valign="top"> </td>