X-Git-Url: http://www.privoxy.org/gitweb/?p=privoxy.git;a=blobdiff_plain;f=doc%2Fsource%2Fuser-manual.sgml;h=290806b6025e02db244514baa6981b74065670b6;hp=c4014234ca8ca94df571d8e4c5bb989dc1ba48c3;hb=6e60985f26cd3570e76ab8db392a846af410ea24;hpb=9b59f5405815ce0a1eefe80069f8f95f48881178 diff --git a/doc/source/user-manual.sgml b/doc/source/user-manual.sgml index c4014234..290806b6 100644 --- a/doc/source/user-manual.sgml +++ b/doc/source/user-manual.sgml @@ -11,8 +11,8 @@ - - + + @@ -24,6 +24,7 @@ + Privoxy"> ]> - Copyright &my-copy; 2001 - 2008 by + Copyright &my-copy; 2001-2009 by Privoxy Developers -$Id: user-manual.sgml,v 2.74 2008/05/26 15:55:46 fabiankeil Exp $ +$Id: user-manual.sgml,v 2.113 2009/10/10 05:48:55 fabiankeil Exp $ @@ -358,7 +359,7 @@ How to install the binary packages depends on your operating system: Before installing Privoxy under Gentoo just do - first emerge rsync to get the latest changes from the + first emerge --sync to get the latest changes from the Portage tree. With emerge privoxy you install the latest version. @@ -436,181 +437,304 @@ How to install the binary packages depends on your operating system: What's New in this Release - There are many improvements and new features since Privoxy 3.0.8, the last stable release: + Privoxy 3.0.15 beta is a bug-fix release + for the previous beta. The changes since 3.0.14 are: - Added SOCKS5 support (with address resolution done by - the SOCKS5 server). Patch provided by Eric M. Hopper. + In case of missing server data, no error message is send to the + client if the request arrived on a reused connection. The client + is then supposed to silently retry the request without bothering + the user. This should significantly reduce the frequency of the + "No server or forwarder data received" error message many users + reported. - The "blocked" CGI pages include a block reason that was - provided as argument to the last-applying block action. + More reliable detection of prematurely closed client sockets + with keep-alive enabled. - If enable-edit-actions is disabled (the default since 3.0.7 beta) - the show-status page hides the edit buttons and explains why. - Previously the user would get the "this feature has been disabled" - message after using the edit button. + FEATURE_CONNECTION_KEEP_ALIVE is decoupled from + FEATURE_CONNECTION_SHARING and now available on + all platforms. - Forbidden CONNECT requests are treated like blocks by default. - The now-pointless treat-forbidden-connects-like-blocks action - has been removed. + Improved handling of POST requests on reused connections. + Should fix problems with stalled connections after submitting + form data with some browser configurations. - Not enabling limit-connect now allows CONNECT requests to all ports. - In previous versions it would only allow CONNECT requests to port 443. - Use +limit-connect{443} if you think you need the old default behaviour. + Fixed various latency calculation issues. - The CGI editor gets turned off after three edit requests with invalid - file modification timestamps. This makes life harder for attackers - who can leverage browser bugs to send fake Referers and intend to - brute-force edit URLs. + Allows the client to pass NTLM authentication requests to a + forwarding proxy. This was already assumed and hinted to work + in 3.0.13 beta but actually didn't. Now it's confirmed to work + with IE, Firefox and Chrome. + Thanks to Francois Botha and Wan-Teh Chang - Action settings for multiple patterns in the same section are - shared in memory. As a result these sections take up less space - (and are loaded slightly faster). Problem reported by Franz Schwartau. + Fixed a calculation problem if receiving the server headers + takes more than two reads, that could cause Privoxy to terminate + the connection prematurely. Reported by Oliver. - Linear white space in HTTP headers will be normalized to single - spaces before parsing the header's content, headers split across - multiple lines get merged first. + Compiles again on platforms such as OpenBSD and systems + using earlier glibc version that don't support AI_ADDRCONFIG. + Anonymously submitted in #2872591. - Host information is gathered outside the main thread so it's less - likely to delay other incoming connections if the host is misconfigured. + A bunch of MS VC project files and Suse and Redhat RPM spec + files have been removed as they were no longer maintained for + quite some time. - New config option "hostname" to use a hostname other than - the one returned by the operating system. Useful to speed-up responses - for CGI requests on misconfigured systems. Requested by Max Khon. + Overly long action lines are properly rejected with a proper + error message. Previously they would be either rejected as + invalid or cause a core dump through abort(). - The CGI editor supports the "disable all filters of this type" - directives "-client-header-filter", "-server-header-filter", - "-client-header-tagger" and "-server-header-tagger". + Already timed-out connections are no longer temporarily remembered. + They weren't reused anyway, but wasted a socket slot. - Fixed false-positives with the link-by-url filter and URLs that - contain the pattern "/jump/". + len refers to the number of bytes actually read which might + differ from the ones received. Adjust log messages accordingly. - The less-download-windows filter no longer messes - "Content-Type: application/x-shockwave-flash" headers up. + The optional JavaScript on the CGI page uses encodeURIComponent() + instead of escape() which doesn't encode all characters that matter. + Anonymously reported in #2832722. - In the show-url-info page's "Final results" section active and - inactive actions are listed separately. Patch provided by Lee. + Fix gcc45 warnings in decompress_iob(). - The GNUmakefile supports the DESTDIR variable. Patch for - the install target submitted by Radoslaw Zielinski. + Various log message improvements. - Embedding the content of configuration files in the show-status - page is significantly faster now. For a largish action file (1 MB) - a speedup of about 2450 times has been measured. This is mostly - interesting if you are using large action files or regularly use - Privoxy-Regression-Test while running Privoxy through Valgrind, - for stock configuration files it doesn't really matter. + Privoxy-Regression-Test supports redirect tests. - If zlib support is unavailable and there are content - filters active but the prevent-compression action is disabled, - the show-url-info page includes a warning that compression - might prevent filtering. + Privoxy-Log-Parser can gather some connection statistics. + + + + + + + If you missed the previous two beta versions, you may also be + interested in the additional changes since since 3.0.12, the + last stable release: + + + + + + + Added IPv6 support. Thanks to Petr Pisar who not only provided + the initial patch but also helped a lot with the integration. + + + + + Added client-side keep-alive support. + + + + + The connection sharing code is only used if the connection-sharing + option is enabled. + + + + + The latency is taken into account when evaluating whether or not to + reuse a connection. This should significantly reduce the number of + connections problems several users reported. + + + + + The max-client-connections option has been added to restrict + the number of client connections below a value enforced by + the operating system. + + + + + If the server doesn't specify how long the connection stays alive, + Privoxy errs on the safe side of caution and assumes it's only a second. - The show-url-info page provides an OpenSearch Description that - allows to access the page through browser search plugins. + Setting keep-alive-timeout to 0 disables keep-alive support. Previously + Privoxy would claim to allow persistence but not reuse the connection. - The obsolete kill-popups action has been removed as the - PCRS-based popup filters can do the same and are slightly - less unreliable. + Pipelined requests are less likely to be mistaken for the request + body of the previous request. Note that Privoxy still has no real + pipeline support and will either serialize pipelined requests or + drop them in which case the client has to resent them. - The inspect-jpegs action has been removed. + Fixed a crash on some Windows versions when header randomization + is enabled and the date couldn't be parsed. - The send-wafer and send-vanilla-wafer actions have been removed. - They weren't particular useful and their behaviour could be emulated - with add-header anyway. + Privoxy's keep-alive timeout for the current connection is reduced + to the one specified in the client's Keep-Alive header. - Privoxy-Regression-Test has been significantly improved. + For HTTP/1.1 requests, Privoxy implies keep-alive support by not + setting any Connection header instead of using 'Connection: keep-alive'. - Most sections in the default.action file contain tests for - Privoxy-Regression-Test to verify that they are working as intended. + If the socket isn't reusable, Privoxy doesn't temporarily waste + a socket slot to remember the connection. - Parts of Privoxy have been refactored to increase maintainability. + If keep-alive support is disabled but compiled in, the client's + Keep-Alive header is removed. - Building with zlib (if available) is done by default. + Fixed a bug on mingw32 where downloading large files failed if + keep-alive support was enabled. + + + + + Fixed a bug that (at least theoretically) could cause log + timestamps to be occasionally off by about a second. + + + + + The configure script respects the $PATH variable when searching + for groups and id. + + + + + Compressed content with extra fields couldn't be decompressed + and would get passed to the client unfiltered. This problem + has only be detected through statical analysis with clang as + nobody seems to be using extra fields anyway. + + + + + If the server resets the Connection after sending only the headers + Privoxy forwards what it got to the client. Previously Privoxy + would deliver an error message instead. + + + + + Error messages in case of connection timeouts use the right + HTTP status code. + + + + + If spawning a child to handle a request fails, the client + gets an error message and Privoxy continues to listen for + new requests right away. + + + + + The error messages in case of server-connection timeouts or + prematurely closed server connections are now template-based. + + + + + If zlib support isn't compiled in, Privoxy no longer tries to + filter compressed content unless explicitly asked to do so. + + + + + In case of connections that are denied based on ACL directives, + the memory used for the client IP is no longer leaked. + + + + + Fixed another small memory leak if the client request times out + while waiting for client headers other than the request line. + + + + + The client socket is kept open until the server socket has + been marked as unused. This should increase the chances that + the still-open connection will be reused for the client's next + request to the same destination. Note that this only matters + if connection-sharing is enabled. + + + + + A TODO list has been added to the source tarballs to give potential + volunteers a better idea of what the current goals are. Donations + are still welcome too: http://www.privoxy.org/faq/general.html#DONATE - - For a more detailed list of changes please have a look at the ChangeLog. - @@ -655,8 +779,8 @@ How to install the binary packages depends on your operating system: - standard.action now only includes the enabled actions. - Not all actions as before. + standard.action has been merged into + the default.action file. @@ -680,18 +804,6 @@ How to install the binary packages depends on your operating system: - - - The filter-client-headers and - filter-server-headers actions that were introduced with - Privoxy 3.0.5 to apply content filters to - the headers have been removed and replaced with new actions. - See the What's New section above. - - - - The actions files are used to define what actions Privoxy takes for which URLs, and thus determines @@ -1911,77 +2014,71 @@ for details. There are three action files included with Privoxy with differing purposes: - - - - - - - default.action - is the primary action file - that sets the initial values for all actions. It is intended to - provide a base level of functionality for - Privoxy's array of features. So it is - a set of broad rules that should work reasonably well as-is for most users. - This is the file that the developers are keeping updated, and making available to users. - The user's preferences as set in standard.action, - e.g. either Cautious (the default), - Medium, or Advanced (see - below). - - - - - user.action - is intended to be for local site - preferences and exceptions. As an example, if your ISP or your bank - has specific requirements, and need special handling, this kind of - thing should go here. This file will not be upgraded. - + + + + + + match-all.action - is used to define which + actions relating to banner-blocking, images, pop-ups, + content modification, cookie handling etc should be applied by default. + It should be the first actions file loaded + - - - standard.action - is used only by the web based editor - at - http://config.privoxy.org/edit-actions-list?f=default, - to set various pre-defined sets of rules for the default actions section - in default.action. - - - Edit Set to Cautious Set to Medium Set to Advanced - - - These have increasing levels of aggressiveness and have no - influence on your browsing unless you select them explicitly in the - editor. A default installation should be pre-set to - Cautious (versions prior to 3.0.5 were set to - Medium). New users should try this for a while before - adjusting the settings to more aggressive levels. The more aggressive - the settings, then the more likelihood there is of problems such as sites - not working as they should. - - - The Edit button allows you to turn each - action on/off individually for fine-tuning. The Cautious - button changes the actions list to low/safe settings which will activate - ad blocking and a minimal set of &my-app;'s features, and subsequently - there will be less of a chance for accidental problems. The - Medium button sets the list to a medium level of - other features and a low level set of privacy features. The - Advanced button sets the list to a high level of - ad blocking and medium level of privacy. See the chart below. The latter - three buttons over-ride any changes via with the - Edit button. More fine-tuning can be done in the - lower sections of this internal page. - - - It is not recommend to edit the standard.action file - itself. - - - The default profiles, and their associated actions, as pre-defined in - standard.action are - - + + + default.action - defines many exceptions (both + positive and negative) from the default set of actions that's configured + in match-all.action. It is a set of rules that should + work reasonably well as-is for most users. This file is only supposed to + be edited by the developers. It should be the second actions file loaded. + + + + + user.action - is intended to be for local site + preferences and exceptions. As an example, if your ISP or your bank + has specific requirements, and need special handling, this kind of + thing should go here. This file will not be upgraded. + + + + + Edit Set to Cautious Set to Medium Set to Advanced + + + These have increasing levels of aggressiveness and have no + influence on your browsing unless you select them explicitly in the + editor. A default installation should be pre-set to + Cautious. New users should try this for a while before + adjusting the settings to more aggressive levels. The more aggressive + the settings, then the more likelihood there is of problems such as sites + not working as they should. + + + The Edit button allows you to turn each + action on/off individually for fine-tuning. The Cautious + button changes the actions list to low/safe settings which will activate + ad blocking and a minimal set of &my-app;'s features, and subsequently + there will be less of a chance for accidental problems. The + Medium button sets the list to a medium level of + other features and a low level set of privacy features. The + Advanced button sets the list to a high level of + ad blocking and medium level of privacy. See the chart below. The latter + three buttons over-ride any changes via with the + Edit button. More fine-tuning can be done in the + lower sections of this internal page. + + + While the actions file editor allows to enable these settings in all + actions files, they are only supposed to be enabled in the first one + to make sure you don't unintentionally overrule earlier rules. + + + The default profiles, and their associated actions, as pre-defined in + default.action are: + + Default Configurations @@ -2054,7 +2151,6 @@ for details. yes - GIF de-animation no @@ -2062,7 +2158,6 @@ for details. yes - Fast redirects no @@ -2103,9 +2198,9 @@ for details.
- - - + + + The list of actions files to be used are defined in the main configuration @@ -2263,12 +2358,12 @@ for details. Generally, an URL pattern has the form - <domain>/<path>, where both the - <domain> and <path> are - optional. (This is why the special / pattern matches all - URLs). Note that the protocol portion of the URL pattern (e.g. - http://) should not be included in - the pattern. This is assumed already! + <domain><port>/<path>, where the + <domain>, the <port> + and the <path> are optional. (This is why the special + / pattern matches all URLs). Note that the protocol + portion of the URL pattern (e.g. http://) should + not be included in the pattern. This is assumed already! The pattern matching syntax is different for the domain and path parts of @@ -2277,6 +2372,12 @@ for details. Regular Expressions (POSIX 1003.2). + + The port part of a pattern is a decimal port number preceded by a colon + (:). If the domain part contains a numerical IPv6 address, + it has to be put into angle brackets + (<, >). + @@ -2300,7 +2401,7 @@ for details. - www.example.com/index.html$ + www.example.com/index.html matches all the documents on www.example.com @@ -2326,6 +2427,23 @@ for details. + + :8000/ + + + Matches any URL pointing to TCP port 8000. + + + + + <2001:db8::1>/ + + + Matches any URL with the host address 2001:db8::1. + (Note that the real URL uses plain brackets, not angle brackets.) + + + index.html @@ -2910,6 +3028,76 @@ for details. + + +change-x-forwarded-for + + + + Typical use: + + Improve privacy by not forwarding the source of the request in the HTTP headers. + + + + + Effect: + + + Deletes the X-Forwarded-For: HTTP header from the client request, + or adds a new one. + + + + + + Type: + + + Parameterized. + + + + + Parameter: + + + + block to delete the header. + + + + add to create the header (or append + the client's IP address to an already existing one). + + + + + + + + Notes: + + + It is safe and recommended to use block. + + + Forwarding the source address of the request may make + sense in some multi-user setups but is also a privacy risk. + + + + + Example usage: + + + +change-x-forwarded-for{block} + + + + + + client-header-filter @@ -4049,7 +4237,7 @@ problem-host.example.com - +filter{crude-parental} # Crude parental filtering. Note that this filter doesn't work reliable. + +filter{crude-parental} # Crude parental filtering. Note that this filter doesn't work reliably. @@ -4707,64 +4895,6 @@ new action - - -hide-forwarded-for-headers - - - Typical use: - - Improve privacy by not forwarding the source of the request in the HTTP headers. - - - - - Effect: - - - Deletes any existing X-Forwarded-for: HTTP header from client requests. - - - - - - Type: - - - Boolean. - - - - - Parameter: - - - N/A - - - - - - Notes: - - - It is safe and recommended to leave this on. - - - - - - Example usage: - - - +hide-forwarded-for-headers - - - - - - - hide-from-header @@ -5275,7 +5405,7 @@ new action reset-to-request-time overwrites the value of the Last-Modified: header with the current time. You could use this option together with - hided-if-modified-since + hide-if-modified-since to further customize your random range. @@ -5933,24 +6063,71 @@ hal stop here linkend="actions">specified and applied to URLs, how patterns work, and how to define and use aliases. Now, let's look at an - example default.action and user.action - file and see how all these pieces come together: + example match-all.action, default.action + and user.action file and see how all these pieces come together: -default.action + +match-all.action + + Remember all actions are disabled when matching starts, + so we have to explicitly enable the ones we want. + -Every config file should start with a short comment stating its purpose: + While the match-all.action file only contains a + single section, it is probably the most important one. It has only one + pattern, /, but this pattern + matches all URLs. Therefore, the set of + actions used in this default section will + be applied to all requests as a start. It can be partly or + wholly overridden by other actions files like default.action + and user.action, but it will still be largely responsible + for your overall browsing experience. - # Sample default.action file <ijbswa-developers@lists.sourceforge.net> + Again, at the start of matching, all actions are disabled, so there is + no need to disable any actions here. (Remember: a + + preceding the action name enables the action, a - disables!). + Also note how this long line has been made more readable by splitting it into + multiple lines with line continuation. + + + + +{ \ + +change-x-forwarded-for{block} \ + +hide-from-header{block} \ + +set-image-blocker{pattern} \ +} +/ # Match all URLs + + + + + The default behavior is now set. + + + + +default.action + + + If you aren't a developer, there's no need for you to edit the + default.action file. It is maintained by + the &my-app; developers and if you disagree with some of the + sections, you should overrule them in your user.action. -Then, since this is the default.action file, the -first section is a special section for internal use that you needn't -change or worry about: + Understanding the default.action file can + help you with your user.action, though. + + + + The first section in this file is a special section for internal use + that prevents older &my-app; versions from reading the file: @@ -5958,15 +6135,14 @@ change or worry about: ########################################################################## # Settings -- Don't change! For internal Privoxy use ONLY. ########################################################################## - {{settings}} -for-privoxy-version=3.0 +for-privoxy-version=3.0.11 -After that comes the (optional) alias section. We'll use the example -section from the above chapter on aliases, -that also explains why and how aliases are used: + After that comes the (optional) alias section. We'll use the example + section from the above chapter on aliases, + that also explains why and how aliases are used: @@ -5991,68 +6167,6 @@ that also explains why and how aliases are used: shop = -crunch-all-cookies -filter{all-popups} - - Now come the regular sections, i.e. sets of actions, accompanied - by URL patterns to which they apply. Remember all actions - are disabled when matching starts, so we have to explicitly - enable the ones we want. - - - - The first regular section is probably the most important. It has only - one pattern, /, but this pattern - matches all URLs. Therefore, the - set of actions used in this default section will - be applied to all requests as a start. It can be partly or - wholly overridden by later matches further down this file, or in user.action, - but it will still be largely responsible for your overall browsing - experience. - - - - Again, at the start of matching, all actions are disabled, so there is - no need to disable any actions here. (Remember: a + - preceding the action name enables the action, a - disables!). - Also note how this long line has been made more readable by splitting it into - multiple lines with line continuation. - - - - -########################################################################## -# "Defaults" section: -########################################################################## - { \ - +deanimate-gifs \ - +filter{html-annoyances} \ - +filter{refresh-tags} \ - +filter{webbugs} \ - +filter{ie-exploits} \ - +hide-forwarded-for-headers \ - +hide-from-header{block} \ - +hide-referrer{forge} \ - +prevent-compression \ - +session-cookies-only \ - +set-image-blocker{pattern} \ - } - / # forward slash will match *all* potential URL patterns. - - - - The default behavior is now set. - - - The first of our specialized sections is concerned with fragile sites, i.e. sites that require minimum interference, because they are either @@ -6093,36 +6207,10 @@ mail.google.com .scan.co.uk - - The fast-redirects - action, which we enabled per default above, breaks some sites. So disable - it for popular sites where we know it misbehaves: + action, which may have been enabled in match-all.action, + breaks some sites. So disable it for popular sites where we know it misbehaves: @@ -6142,8 +6230,8 @@ edit.*.yahoo.com be blocked, a substitute image can be sent, rather than an HTML page. Contacting the remote site to find out is not an option, since it would destroy the loading time advantage of banner blocking, and it - would feed the advertisers (in terms of money and - information). We can mark any URL as an image with the handle-as-image action, and marking all URLs that end in a known image file extension is a good start: @@ -8177,7 +8265,8 @@ Requests In file: default.action [ View ] [ Edit ] - {+deanimate-gifs {last} + {+change-x-forwarded-for{block} + +deanimate-gifs {last} +fast-redirects {check-decoded-url} +filter {refresh-tags} +filter {img-reorder} @@ -8185,7 +8274,6 @@ Requests +filter {webbugs} +filter {jumping-windows} +filter {ie-exploits} - +hide-forwarded-for-headers +hide-from-header {block} +hide-referrer {forge} +session-cookies-only @@ -8267,6 +8355,7 @@ In file: user.action [ View ] [ Edit ][ View ] [ Edit ][ View ] [ Edit ][ View ] [ Edit ][ View ] [ Edit ]