X-Git-Url: http://www.privoxy.org/gitweb/?p=privoxy.git;a=blobdiff_plain;f=doc%2Fsource%2Fuser-manual.sgml;h=3928054806d7632931e33783789c7d3c035fad46;hp=6e2e6eb541d74e376fd1a2a29ffaf356f34dc25a;hb=1e978c46c5b5b21a8a283a9d62069cfc300ea0d1;hpb=ec4fc8c32f1367996ede49c3fc9f867fa5b0e79d diff --git a/doc/source/user-manual.sgml b/doc/source/user-manual.sgml index 6e2e6eb5..39280548 100644 --- a/doc/source/user-manual.sgml +++ b/doc/source/user-manual.sgml @@ -9,9 +9,11 @@ + - + + @@ -34,9 +36,9 @@ This file belongs into ijbswa.sourceforge.net:/home/groups/i/ij/ijbswa/htdocs/ - $Id: user-manual.sgml,v 2.157 2013/01/05 23:50:35 ler762 Exp $ + $Id: user-manual.sgml,v 2.212 2016/05/25 10:50:55 fabiankeil Exp $ - Copyright (C) 2001-2013 Privoxy Developers http://www.privoxy.org/ + Copyright (C) 2001-2016 Privoxy Developers https://www.privoxy.org/ See LICENSE. ======================================================================== @@ -55,12 +57,12 @@ - Copyright &my-copy; 2001-2011 by - Privoxy Developers + Copyright &my-copy; 2001-2016 by + Privoxy Developers -$Id: user-manual.sgml,v 2.157 2013/01/05 23:50:35 ler762 Exp $ +$Id: user-manual.sgml,v 2.212 2016/05/25 10:50:55 fabiankeil Exp $ @@ -99,14 +101,11 @@ Hal. You can find the latest version of the Privoxy User Manual at http://www.privoxy.org/user-manual/. + url="https://www.privoxy.org/user-manual/">https://www.privoxy.org/user-manual/. Please see the Contact section on how to contact the developers. - - - @@ -115,7 +114,7 @@ Hal. Introduction This documentation is included with the current &p-status; version of - Privoxy, v.&p-version;Privoxy, &p-version;Privoxy is available both in convenient pre-compiled packages for a wide range of operating systems, and as raw source code. For most users, we recommend using the packages, which can be downloaded from our - Privoxy Project + Privoxy Project Page. @@ -180,36 +179,6 @@ How to install the binary packages depends on your operating system: - -Red Hat and Fedora RPMs - - - RPMs can be installed with rpm -Uvh privoxy-&p-version;-1.rpm, - and will use /etc/privoxy for the location - of configuration files. - - - - Note that on Red Hat, Privoxy will - not be automatically started on system boot. You will - need to enable that using chkconfig, - ntsysv, or similar methods. - - - - If you have problems with failed dependencies, try rebuilding the SRC RPM: - rpm --rebuild privoxy-&p-version;-1.src.rpm. This - will use your locally installed libraries and RPM version. - - - - Also note that if you have a Junkbuster RPM installed - on your system, you need to remove it first, because the packages conflict. - Otherwise, RPM will try to remove Junkbuster - automatically if found, before installing Privoxy. - - - Debian and Ubuntu @@ -229,1297 +198,209 @@ How to install the binary packages depends on your operating system: Version 3.0.5 beta introduced full Windows service - functionality. On Windows only, the Privoxy - program has two new command line arguments to install and uninstall - Privoxy as a service. - - - - Arguments: - - - --install[:service_name] - - - --uninstall[:service_name] - - - - - - After invoking Privoxy with - --install, you will need to bring up the - Windows service console to assign the user you - want Privoxy to run under, and whether or not you - want it to run whenever the system starts. You can start the - Windows services console with the following - command: services.msc. If you do not take the manual step - of modifying Privoxy's service settings, it will - not start. Note too that you will need to give Privoxy a user account that - actually exists, or it will not be permitted to - write to its log and configuration files. - - - - - -Solaris <!--, NetBSD, HP-UX--> - - - Create a new directory, cd to it, then unzip and - untar the archive. For the most part, you'll have to figure out where - things go. - - - - -OS/2 - - - First, make sure that no previous installations of - Junkbuster and / or - Privoxy are left on your - system. Check that no Junkbuster - or Privoxy objects are in - your startup folder. - - - - - Then, just double-click the WarpIN self-installing archive, which will - guide you through the installation process. A shadow of the - Privoxy executable will be placed in your - startup folder so it will start automatically whenever OS/2 starts. - - - - The directory you choose to install Privoxy - into will contain all of the configuration files. - - - - -Mac OS X - - Installation instructions for the OS X platform depend upon whether - you downloaded a ready-built installation package (.pkg or .mpkg) or have - downloaded the source code. - - - -Installation from ready-built package - - The downloaded file will either be a .pkg (for OS X 10.5 upwards) or a bzipped - .mpkg file (for OS X 10.4). The former can be double-clicked as is and the - installation will start; double-clicking the latter will unzip the .mpkg file - which can then be double-clicked to commence the installation. - - - The privoxy service will automatically start after a successful installation - (and thereafter every time your computer starts up) however you will need to - configure your web browser(s) to use it. To do so, configure them to use a - proxy for HTTP and HTTPS at the address 127.0.0.1:8118. - - - To prevent the privoxy service from automatically starting when your computer - starts up, remove or rename the file /Library/LaunchDaemons/org.ijbswa.privoxy.plist - (on OS X 10.5 and higher) or the folder named - /Library/StartupItems/Privoxy (on OS X 10.4 'Tiger'). - - - To manually start or stop the privoxy service, use the scripts startPrivoxy.sh - and stopPrivoxy.sh supplied in /Applications/Privoxy. They must be run from an - administrator account, using sudo. - - - To uninstall, run /Applications/Privoxy/uninstall.command as sudo from an - administrator account. - - - -Installation from source - - To build and install the Privoxy source code on OS X you will need to obtain - the macsetup module from the Privoxy Sourceforge CVS repository (refer to - Sourceforge help for details of how to set up a CVS client to have read-only - access to the repository). This module contains scripts that leverage the usual - open-source tools (available as part of Apple's free of charge Xcode - distribution or via the usual open-source software package managers for OS X - (MacPorts, Homebrew, Fink etc.) to build and then install the privoxy binary - and associated files. The macsetup module's README file contains complete - instructions for its use. - - - The privoxy service will automatically start after a successful installation - (and thereafter every time your computer starts up) however you will need to - configure your web browser(s) to use it. To do so, configure them to use a - proxy for HTTP and HTTPS at the address 127.0.0.1:8118. - - - To prevent the privoxy service from automatically starting when your computer - starts up, remove or rename the file /Library/LaunchDaemons/org.ijbswa.privoxy.plist - (on OS X 10.5 and higher) or the folder named - /Library/StartupItems/Privoxy (on OS X 10.4 'Tiger'). - - - To manually start or stop the privoxy service, use the Privoxy Utility - for Mac OS X (also part of the macsetup module). This application can start - and stop the privoxy service and display its log and configuration files. - - - To uninstall, run the macsetup module's uninstall.sh as sudo from an - administrator account. - - - - -AmigaOS - - Copy and then unpack the lha archive to a suitable location. - All necessary files will be installed into Privoxy - directory, including all configuration and log files. To uninstall, just - remove this directory. - - - - -FreeBSD - - - Privoxy is part of FreeBSD's Ports Collection, you can build and install - it with cd /usr/ports/www/privoxy; make install clean. - - - If you don't use the ports, you can fetch and install - the package with pkg_add -r privoxy. - - - The port skeleton and the package can also be downloaded from the - File Release - Page, but there's no reason to use them unless you're interested in the - beta releases which are only available there. - - - - -Gentoo - - Gentoo source packages (Ebuilds) for Privoxy are - contained in the Gentoo Portage Tree (they are not on the download page, - but there is a Gentoo section, where you can see when a new - Privoxy Version is added to the Portage Tree). - - - Before installing Privoxy under Gentoo just do - first emerge --sync to get the latest changes from the - Portage tree. With emerge privoxy you install the latest - version. - - - Configuration files are in /etc/privoxy, the - documentation is in /usr/share/doc/privoxy-&p-version; - and the Log directory is in /var/log/privoxy. - - - - - - -Building from Source - - - The most convenient way to obtain the Privoxy sources - is to download the source tarball from our - project download - page. - - - - If you like to live on the bleeding edge and are not afraid of using - possibly unstable development versions, you can check out the up-to-the-minute - version directly from the - CVS repository. - - - - -&buildsource; - - - - -Keeping your Installation Up-to-Date - - As user feedback comes in and development continues, we will make updated versions - of both the main actions file (as a separate - package) and the software itself (including the actions file) available for - download. - - - - If you wish to receive an email notification whenever we release updates of - Privoxy or the actions file, subscribe - to our announce mailing list, ijbswa-announce@lists.sourceforge.net. - - - - In order not to lose your personal changes and adjustments when updating - to the latest default.action file we strongly - recommend that you use user.action and - user.filter for your local - customizations of Privoxy. See the Chapter on actions files for details. - - - - - - - - - - - -What's New in this Release - - Privoxy 3.0.19 is a stable release. - The changes since 3.0.18 stable are: - - - - - - - Bug fixes: - - - - Prevent a segmentation fault when de-chunking buffered content. - It could be triggered by malicious web servers if Privoxy was - configured to filter the content and running on a platform - where SIZE_T_MAX isn't larger than UINT_MAX, which probably - includes most 32-bit systems. On those platforms, all Privoxy - versions before 3.0.19 appear to be affected. - To be on the safe side, this bug should be presumed to allow - code execution as proving that it doesn't seems unrealistic. - - - - - Do not expect a response from the SOCKS4/4A server until it - got something to respond to. This regression was introduced - in 3.0.18 and prevented the SOCKS4/4A negotiation from working. - Reported by qqqqqw in #3459781. - - - - - - - - General improvements: - - - - Fix an off-by-one in an error message about connect failures. - - - - - Use a GNUMakefile variable for the webserver root directory and - update the path. Sourceforge changed it which broke various - web-related targets. - - - - - Update the CODE_STATUS description. - - - - - - - - - - The following changes were made between 3.0.17 and 3.0.18: - - - - - - - Bug fixes: - - - - If a generated redirect URL contains characters RFC 3986 doesn't - permit, they are (re)encoded. Not doing this makes Privoxy versions - from 3.0.5 to 3.0.17 susceptible to HTTP response splitting (CWE-113) - attacks if the +fast-redirects{check-decoded-url} action is used. - - - - - Fix a logic bug that could cause Privoxy to reuse a server - socket after it got tainted by a server-header-tagger-induced - block that was triggered before the whole server response had - been read. If keep-alive was enabled and the request following - the blocked one was to the same host and using the same forwarding - settings, Privoxy would send it on the tainted server socket. - While the server would simply treat it as a pipelined request, - Privoxy would later on fail to properly parse the server's - response as it would try to parse the unread data from the - first response as server headers for the second one. - Regression introduced in 3.0.17. - - - - - When implying keep-alive in client_connection(), remember that - the client didn't. Fixes a regression introduced in 3.0.13 that - would cause Privoxy to wait for additional client requests after - receiving a HTTP/1.1 request with "Connection: close" set - and connection sharing enabled. - With clients which terminates the client connection after detecting - that the whole body has been received it doesn't really matter, - but with clients that don't the connection would be kept open until - it timed out. - - - - - Fix a subtle race condition between prepare_csp_for_next_request() - and sweep(). A thread preparing itself for the next client request - could briefly appear to be inactive. - If all other threads were already using more recent files, - the thread could get its files swept away under its feet. - So far this has only been reproduced while stress testing in - valgrind while touching action files in a loop. It's unlikely - to have caused any actual problems in the real world. - - - - - Disable filters if SDCH compression is used unless filtering is forced. - If SDCH was combined with a supported compression algorithm, Privoxy - previously could try to decompress it and ditch the Content-Encoding - header even though the SDCH compression wasn't dealt with. - Reported by zebul666 in #3225863. - - - - - Make a copy of the --user value and only mess with that when splitting - user and group. On some operating systems modifying the value directly - is reflected in the output of ps and friends and can be misleading. - Reported by zepard in #3292710. - - - - - If forwarded-connect-retries is set, only retry if Privoxy is actually - forwarding the request. Previously direct connections would be retried - as well. - - - - - Fixed a small memory leak when retrying connections with IPv6 - support enabled. - - - - - Remove an incorrect assertion in compile_dynamic_pcrs_job_list() - It could be triggered by a pcrs job with an invalid pcre - pattern (for example one that contains a lone quantifier). - - - - - If the --user argument user[.group] contains a dot, always bail out - if no group has been specified. Previously the intended, but undocumented - (and apparently untested), behaviour was to try interpreting the whole - argument as user name, but the detection was flawed and checked for '0' - instead of '\0', thus merely preventing group names beginning with a zero. - - - - - In html_code_map[], use a numeric character reference instead of ' - which wasn't standardized before XHTML 1.0. - - - - - Fix an invalid free when compiled with FEATURE_GRACEFUL_TERMINATION - and shut down through http://config.privoxy.org/die - - - - - In get_actions(), fix the "temporary" backwards compatibility hack - to accept block actions without reason. - It also covered other actions that should be rejected as invalid. - Reported by Billy Crook. - - - - - - - - General improvements: - - - - Privoxy can (re)compress buffered content before delivering - it to the client. Disabled by default as most users wouldn't - benefit from it. - - - - - The +fast-redirects{check-decoded-url} action checks URL - segments separately. If there are other parameters behind - the redirect URL, this makes it unnecessary to cut them off - by additionally using a +redirect{} pcrs command. - Initial patch submitted by Jamie Zawinski in #3429848. - - - - - When loading action sections, verify that the referenced filters - exist. Currently missing filters only result in an error message, - but eventually the severity will be upgraded to fatal. - - - - - Allow to bind to multiple separate addresses. - Patch set submitted by Petr Pisar in #3354485. - - - - - Set socket_error to errno if connecting fails in rfc2553_connect_to(). - Previously rejected direct connections could be incorrectly reported - as DNS issues if Privoxy was compiled with IPv6 support. - - - - - Adjust url_code_map[] so spaces are replaced with %20 instead of '+' - While '+' can be used by client's submitting form data, this is not - actually what Privoxy is using the lookups for. This is more of a - cosmetic issue and doesn't fix any known problems. - - - - - When compiled without FEATURE_FAST_REDIRECTS, do not silently - ignore +fast-redirect{} directives - - - - - Added a workaround for GNU libc's strptime() reporting negative - year values when the parsed year is only specified with two digits. - On affected systems cookies with such a date would not be turned - into session cookies by the +session-cookies-only action. - Reported by Vaeinoe in #3403560 - - - - - Fixed bind failures with certain GNU libc versions if no non-loopback - IP address has been configured on the system. This is mainly an issue - if the system is using DHCP and Privoxy is started before the network - is completely configured. - Reported by Raphael Marichez in #3349356. - Additional insight from Petr Pisar. - - - - - Privoxy log messages now use the ISO 8601 date format %Y-%m-%d. - It's only slightly longer than the old format, but contains - the full date including the year and allows sorting by date - (when grepping in multiple log files) without hassle. - - - - - In get_last_url(), do not bother trying to decode URLs that do - not contain at least one '%' sign. It reduces the log noise and - a number of unnecessary memory allocations. - - - - - In case of SOCKS5 failures, dump the socks response in the log message. - - - - - Simplify the signal setup in main(). - - - - - Streamline socks5_connect() slightly. - - - - - In socks5_connect(), require a complete socks response from the server. - Previously Privoxy didn't care how much data the server response - contained as long as the first two bytes contained the expected - values. While at it, shrink the buffer size so Privoxy can't read - more than a whole socks response. - - - - - In chat(), do not bother to generate a client request in case of - direct CONNECT requests. It will not be used anyway. - - - - - Reduce server_last_modified()'s stack size. - - - - - Shorten get_http_time() by using strftime(). - - - - - Constify the known_http_methods pointers in unknown_method(). - - - - - Constify the time_formats pointers in parse_header_time(). - - - - - Constify the formerly_valid_actions pointers in action_used_to_be_valid(). - - - - - Introduce a GNUMakefile MAN_PAGE variable that defaults to privoxy.1. - The Debian package uses section 8 for the man page and this - should simplify the patch. - - - - - Deduplicate the INADDR_NONE definition for Solaris by moving it to jbsockets.h - - - - - In block_url(), ditch the obsolete workaround for ancient Netscape versions - that supposedly couldn't properly deal with status code 403. - - - - - Remove a useless NULL pointer check in load_trustfile(). - - - - - Remove two useless NULL pointer checks in load_one_re_filterfile(). - - - - - Change url_code_map[] from an array of pointers to an array of arrays - It removes an unnecessary layer of indirection and on 64bit system reduces - the size of the binary a bit. - - - - - Fix various typos. Fixes taken from Debian's 29_typos.dpatch by Roland Rosenfeld. - - - - - Add a dok-tidy GNUMakefile target to clean up the messy HTML - generated by the other dok targets. - - - - - GNUisms in the GNUMakefile have been removed. - - - - - Change the HTTP version in static responses to 1.1 - - - - - Synced config.sub and config.guess with upstream - 2011-11-11/386c7218162c145f5f9e1ff7f558a3fbb66c37c5. - - - - - Add a dedicated function to parse the values of toggles. Reduces duplicated - code in load_config() and provides better error handling. Invalid or missing - toggle values are now a fatal error instead of being silently ignored. - - - - - Terminate HTML lines in static error messages with \n instead of \r\n. - - - - - Simplify cgi_error_unknown() a bit. - - - - - In LogPutString(), don't bother looking at pszText when not - actually logging anything. - - - - - Change ssplit()'s fourth parameter from int to size_t. - Fixes a clang complaint. - - - - - Add a warning that the statistics currently can't be trusted. - Mention Privoxy-Log-Parser's --statistics option as - an alternative for the time being. - - - - - In rfc2553_connect_to(), start setting cgi->error_message on error. - - - - - Change the expected status code returned for http://p.p/die depending - on whether or not FEATURE_GRACEFUL_TERMINATION is available. - - - - - In cgi_die(), mark the client connection for closing. - If the client will fetch the style sheet through another connection - it gets the main thread out of the accept() state and should thus - trigger the actual shutdown. - - - - - Add a proper CGI message for cgi_die(). - - - - - Don't enforce a logical line length limit in read_config_line(). - - - - - Slightly refactor server_last_modified() to remove useless gmtime*() calls. - - - - - In get_content_type(), also recognize '.jpeg' as JPEG extension. - - - - - Add '.png' to the list of recognized file extensions in get_content_type(). - - - - - In block_url(), consistently use the block reason "Request blocked by Privoxy" - In two places the reason was "Request for blocked URL" which hides the - fact that the request got blocked by Privoxy and isn't necessarily - correct as the block may be due to tags. - - - - - In listen_loop(), reload the configuration files after accepting - a new connection instead of before. - Previously the first connection that arrived after a configuration - change would still be handled with the old configuration. - - - - - In chat()'s receive-data loop, skip a client socket check if - the socket will be written to right away anyway. This can - increase the transfer speed for unfiltered content on fast - network connections. - - - - - The socket timeout is used for SOCKS negotiations as well which - previously couldn't timeout. - - - - - Don't keep the client connection alive if any configuration file - changed since the time the connection came in. This is closer to - Privoxy's behaviour before keep-alive support for client connection - has been added and also less confusing in general. - - - - - Treat all Content-Type header values containing the pattern - 'script' as a sign of text. Reported by pribog in #3134970. - - - - - - - - Action file improvements: - - - - Moved the site-specific block pattern section below the one for the - generic patterns so for requests that are matched in both, the block - reason for the domain is shown which is usually more useful than showing - the one for the generic pattern. - - - - - Remove -prevent-compression from the fragile alias. It's no longer - used anywhere by default and isn't known to break stuff anyway. - - - - - Add a (disabled) section to block various Facebook tracking URLs. - Reported by Dan Stahlke in #3421764. - - - - - Add a (disabled) section to rewrite and redirect click-tracking - URLs used on news.google.com. - Reported by Dan Stahlke in #3421755. - - - - - Unblock linuxcounter.net/. - Reported by Dan Stahlke in #3422612. - - - - - Block 'www91.intel.com/' which is used by Omniture. - Reported by Adam Piggott in #3167370. - - - - - Disable the handle-as-empty-doc-returns-ok option and mark it as deprecated. - Reminded by tceverling in #2790091. - - - - - Add ".ivwbox.de/" to the "Cross-site user tracking" section. - Reported by Nettozahler in #3172525. - - - - - Unblock and fast-redirect ".awin1.com/.*=http://". - Reported by Adam Piggott in #3170921. - - - - - Block "b.collective-media.net/". - - - - - Widen the Debian popcon exception to "qa.debian.org/popcon". - Seen in Debian's 05_default_action.dpatch by Roland Rosenfeld. - - - - - Block ".gemius.pl/" which only seems to be used for user tracking. - Reported by johnd16 in #3002731. Additional input from Lee and movax. - - - - - Disable banners-by-size filters for '.thinkgeek.com/'. - The filter only seems to catch pictures of the inventory. - - - - - Block requests for 'go.idmnet.bbelements.com/please/showit/'. - Reported by kacperdominik in #3372959. - - - - - Unblock adainitiative.org/. - - - - - Add a fast-redirects exception for '.googleusercontent.com/.*=cache'. - - - - - Add a fast-redirects exception for webcache.googleusercontent.com/. - - - - - Unblock http://adassier.wordpress.com/ and http://adassier.files.wordpress.com/. - - - - - - - - Filter file improvements: - - - - Let the yahoo filter hide '.ads'. - - - - - Let the msn filter hide overlay ads for Facebook 'likes' in search - results and elements with the id 's_notf_div'. They only seem to be - used to advertise site 'enhancements'. - - - - - Let the js-events filter additionally disarm setInterval(). - Suggested by dg1727 in #3423775. - - - - - - - - Documentation improvements: - - - - Clarify the effect of compiling Privoxy with zlib support. - Suggested by dg1727 in #3423782. - - - - - Point out that the SourceForge messaging system works like a black - hole and should thus not be used to contact individual developers. - - - - - Mention some of the problems one can experience when not explicitly - configuring an IP addresses as listen address. - - - - - Explicitly mention that hostnames can be used instead of IP addresses - for the listen-address, that only the first address returned will be - used and what happens if the address is invalid. - Requested by Calestyo in #3302213. - - - - - - - - Log message improvements: - - - - If only the server connection is kept alive, do not pretend to - wait for a new client request. - - - - - Remove a superfluous log message in forget_connection(). - - - - - In chat(), properly report missing server responses as such - instead of calling them empty. - - - - - In forwarded_connect(), fix a log message nobody should ever see. - - - - - Fix a log message in socks5_connect(), a failed write operation - was logged as failed read operation. - - - - - Let load_one_actions_file() properly complain about a missing - '{' at the beginning of the file. - Simply stating that a line is invalid isn't particularly helpful. - - - - - Do not claim to listen on a socket until Privoxy actually does. - Patch submitted by Petr Pisar #3354485 - - - - - Prevent a duplicated LOG_LEVEL_CLF message when sending out - the "no-server-data" response. - - - - - Also log the client socket when dropping a connection. - - - - - Include the destination host in the 'Request ... marked for - blocking. limit-connect{...} doesn't allow CONNECT ...' message - Patch submitted by Saperski in #3296250. - - - - - Prevent a duplicated log message if none of the resolved IP - addresses were reachable. - - - - - In connect_to(), do not pretend to retry if forwarded-connect-retries - is zero or unset. - - - - - When a specified user or group can't be found, put the name in - single-quotes when logging it. - - - - - In rfc2553_connect_to(), explain getnameinfo() errors better. - - - - - Remove a useless log message in chat(). - - - - - When retrying to connect, also log the maximum number of connection - attempts. - - - - - Rephrase a log message in compile_dynamic_pcrs_job_list(). - Divide the error code and its meaning with a colon. Call the pcrs - job dynamic and not the filter. Filters may contain dynamic and - non-dynamic pcrs jobs at the same time. Only mention the name of - the filter or tagger, but don't claim it's a filter when it could - be a tagger. - - - - - In a fatal error message in load_one_actions_file(), cover both - URL and TAG patterns. - - - - - In pcrs_strerror(), properly report unknown positive error code - values as such. Previously they were handled like 0 (no error). - - - - - In compile_dynamic_pcrs_job_list(), also log the actual error code as - pcrs_strerror() doesn't handle all errors reported by pcre. - - - - - Don't bother trying to continue chatting if the client didn't ask for it. - Reduces log noise a bit. - - - - - Make two fatal error message in load_one_actions_file() more descriptive. - - - - - In cgi_send_user_manual(), log when rejecting a file name due to '/' or '..'. - - - - - In load_file(), log a message if opening a file failed. - The CGI error message alone isn't too helpful. - - - - - In connection_destination_matches(), improve two log messages - to help understand why the destinations don't match. - - - - - Rephrase a log message in serve(). Client request arrival - should be differentiated from closed client connections now. - - - - - In serve(), log if a client connection isn't reused due to a - configuration file change. - - - - - Let mark_server_socket_tainted() always mark the server socket tainted, - just don't talk about it in cases where it has no effect. It doesn't change - Privoxy's behaviour, but makes understanding the log file easier. - - - - - - - - configure: - - - - Added a --disable-ipv6-support switch for platforms where support - is detected but doesn't actually work. - - - - - Do not check for the existence of strerror() and memmove() twice - - - - - Remove a useless test for setpgrp(2). Privoxy doesn't need it and - it can cause problems when cross-compiling. - - - - - Rename the --disable-acl-files switch to --disable-acl-support. - Since about 2001, ACL directives are specified in the standard - config file. - - - - - Update the URL of the 'Removing outdated PCRE version after the - next stable release' posting. The old URL stopped working after - one of SF's recent site "optimizations". Reported by Han Liu. - - - - - - - - Privoxy-Regression-Test: - - - - Added --shuffle-tests option to increase the chances of detection race conditions. - - - - - Added a --local-test-file option that allows to use Privoxy-Regression-Test without Privoxy. - - - - - Added tests for missing socks4 and socks4a forwarders. - - - - - The --privoxy-address option now works with IPv6 addresses containing brackets, too. - - - - - Perform limited sanity checks for parameters that are supposed to have numerical values. - - - - - Added a --sleep-time option to specify a number of seconds to - sleep between tests, defaults to 0. - - - - - Disable the range-requests tagger for tests that break if it's enabled. - - - - - Log messages use the ISO 8601 date format %Y-%m-%d. - - - - - Fix spelling in two error messages. - - - - - In the --help output, include a list of supported tests and their default levels. - - - - - Adjust the tests to properly deal with FEATURE_TOGGLE being disabled. - - - - - - - - Privoxy-Log-Parser: - - - - Perform limited sanity checks for command line parameters that - are supposed to have numerical values. - - - - - Implement a --unbreak-lines-only option to try to revert MUA breakage. - - - - - Accept and highlight: Added header: Content-Encoding: deflate - - - - - Accept and highlight: Compressed content from 29258 to 8630 bytes. - - - - - Accept and highlight: Client request arrived in time on socket 21. - - - - - Highlight: Didn't receive data in time: a.fsdn.com:443 - - - - - Accept log messages with ISO 8601 time stamps, too. - - - - - - - - uagen: - - - - Bump generated Firefox version to 8.0. - - - - - Only randomize the release date if the new --randomize-release-date - option is enabled. Firefox versions after 4 use a fixed date string - without meaning. - - - - - - + functionality. On Windows only, the Privoxy + program has two new command line arguments to install and uninstall + Privoxy as a service. + + + + Arguments: + + + --install[:service_name] + + + --uninstall[:service_name] + + + + + + After invoking Privoxy with + --install, you will need to bring up the + Windows service console to assign the user you + want Privoxy to run under, and whether or not you + want it to run whenever the system starts. You can start the + Windows services console with the following + command: services.msc. If you do not take the manual step + of modifying Privoxy's service settings, it will + not start. Note too that you will need to give Privoxy a user account that + actually exists, or it will not be permitted to + write to its log and configuration files. + + + + + +OS/2 + + + First, make sure that no previous installations of + Junkbuster and / or + Privoxy are left on your + system. Check that no Junkbuster + or Privoxy objects are in + your startup folder. + + + + + Then, just double-click the WarpIN self-installing archive, which will + guide you through the installation process. A shadow of the + Privoxy executable will be placed in your + startup folder so it will start automatically whenever OS/2 starts. + + + + The directory you choose to install Privoxy + into will contain all of the configuration files. + + + + +Mac OS X + + Installation instructions for the OS X platform depend upon whether + you downloaded a ready-built installation package (.pkg or .mpkg) or have + downloaded the source code. + + + +Installation from ready-built package + + The downloaded file will either be a .pkg (for OS X 10.5 upwards) or a bzipped + .mpkg file (for OS X 10.4). The former can be double-clicked as is and the + installation will start; double-clicking the latter will unzip the .mpkg file + which can then be double-clicked to commence the installation. + + + The privoxy service will automatically start after a successful installation + (and thereafter every time your computer starts up) however you will need to + configure your web browser(s) to use it. To do so, configure them to use a + proxy for HTTP and HTTPS at the address 127.0.0.1:8118. + + + To prevent the privoxy service from automatically starting when your computer + starts up, remove or rename the file /Library/LaunchDaemons/org.ijbswa.privoxy.plist + (on OS X 10.5 and higher) or the folder named + /Library/StartupItems/Privoxy (on OS X 10.4 'Tiger'). + + + To manually start or stop the privoxy service, use the scripts startPrivoxy.sh + and stopPrivoxy.sh supplied in /Applications/Privoxy. They must be run from an + administrator account, using sudo. + + + To uninstall, run /Applications/Privoxy/uninstall.command as sudo from an + administrator account. + + + +Installation from source + + To build and install the Privoxy source code on OS X you will need to obtain + the macsetup module from the Privoxy Sourceforge CVS repository (refer to + Sourceforge help for details of how to set up a CVS client to have read-only + access to the repository). This module contains scripts that leverage the usual + open-source tools (available as part of Apple's free of charge Xcode + distribution or via the usual open-source software package managers for OS X + (MacPorts, Homebrew, Fink etc.) to build and then install the privoxy binary + and associated files. The macsetup module's README file contains complete + instructions for its use. + + + The privoxy service will automatically start after a successful installation + (and thereafter every time your computer starts up) however you will need to + configure your web browser(s) to use it. To do so, configure them to use a + proxy for HTTP and HTTPS at the address 127.0.0.1:8118. + + + To prevent the privoxy service from automatically starting when your computer + starts up, remove or rename the file /Library/LaunchDaemons/org.ijbswa.privoxy.plist + (on OS X 10.5 and higher) or the folder named + /Library/StartupItems/Privoxy (on OS X 10.4 'Tiger'). + + + To manually start or stop the privoxy service, use the Privoxy Utility + for Mac OS X (also part of the macsetup module). This application can start + and stop the privoxy service and display its log and configuration files. + + + To uninstall, run the macsetup module's uninstall.sh as sudo from an + administrator account. + + + + +FreeBSD + + + Privoxy is part of FreeBSD's Ports Collection, you can build and install + it with cd /usr/ports/www/privoxy; make install clean. + + + + + + +Building from Source + + + The most convenient way to obtain the Privoxy sources + is to download the source tarball from our + project download + page. + + + + If you like to live on the bleeding edge and are not afraid of using + possibly unstable development versions, you can check out the up-to-the-minute + version directly from the + CVS repository. + + + + +&buildsource; + + + + +Keeping your Installation Up-to-Date + + + If you wish to receive an email notification whenever we release updates of + Privoxy or the actions file, subscribe + to our announce mailing list, privoxy-announce@lists.privoxy.org. + + + + In order not to lose your personal changes and adjustments when updating + to the latest default.action file we strongly + recommend that you use user.action and + user.filter for your local + customizations of Privoxy. See the Chapter on actions files for details. + + + + + + + + + +What's New in this Release + +&changelog; @@ -1562,12 +443,6 @@ How to install the binary packages depends on your operating system: files, thinking you will want to do that yourself. - - - standard.action has been merged into - the default.action file. - - In the default configuration only fatal errors are logged now. @@ -1746,18 +621,6 @@ How to install the binary packages depends on your operating system: - - Please see the section Contacting the @@ -2185,39 +1048,40 @@ How to install the binary packages depends on your operating system: directory. Except on Win32 where it will try config.txt. - -Red Hat and Fedora + +Debian - A default Red Hat installation may not start &my-app; upon boot. It will use - the file /etc/privoxy/config as its main configuration + We use a script. Note that Debian typically starts &my-app; upon booting per + default. It will use the file + /etc/privoxy/config as its main configuration file. - # /etc/rc.d/init.d/privoxy start + # /etc/init.d/privoxy start + + + +FreeBSD and ElectroBSD - Or ... + To start Privoxy upon booting, add + "privoxy_enable='YES'" to /etc/rc.conf. + Privoxy will use + /usr/local/etc/privoxy/config as its main + configuration file. - - # service privoxy start - + If you installed Privoxy into a jail, the + paths above are relative to the jail root. - - - -Debian - We use a script. Note that Debian typically starts &my-app; upon booting per - default. It will use the file - /etc/privoxy/config as its main configuration - file. + To start Privoxy manually, run: - # /etc/init.d/privoxy start + # service privoxy onestart @@ -2241,15 +1105,21 @@ Click on the &my-app; Icon to start Privoxy. If no co -Solaris, NetBSD, FreeBSD, HP-UX and others +Generic instructions for Unix derivates (Solaris, NetBSD, HP-UX etc.) Example Unix startup command: - # /usr/sbin/privoxy /etc/privoxy/config + # /usr/sbin/privoxy --user privoxy /etc/privoxy/config + + Note that if you installed Privoxy through + a package manager, the package will probably contain a platform-specific + script or configuration file to start Privoxy + upon boot. + @@ -2265,72 +1135,25 @@ Example Unix startup command: Mac OS X - After downloading the privoxy software, unzip the downloaded file by - double-clicking on the zip file icon. Then, double-click on the - installer package icon and follow the installation process. - - - The privoxy service will automatically start after a successful - installation. In addition, the privoxy service will automatically - start every time your computer starts up. - - - To prevent the privoxy service from automatically starting when your - computer starts up, remove or rename the folder named - /Library/StartupItems/Privoxy. - - - A simple application named Privoxy Utility has been created which - enables administrators to easily start and stop the privoxy service. - - - In addition, the Privoxy Utility presents a simple way for - administrators to edit the various privoxy config files. A method - to uninstall the software is also available. - - - An administrator username and password must be supplied in order for - the Privoxy Utility to perform any of the tasks. - - - - - -AmigaOS - - Start Privoxy (with RUN <>NIL:) in your - startnet script (AmiTCP), in - s:user-startup (RoadShow), as startup program in your - startup script (Genesis), or as startup action (Miami and MiamiDx). - Privoxy will automatically quit when you quit your - TCP/IP stack (just ignore the harmless warning your TCP/IP stack may display that - Privoxy is still running). - - - - -Gentoo - - A script is again used. It will use the file /etc/privoxy/config - as its main configuration file. - - - - /etc/init.d/privoxy start - + The privoxy service will automatically start after a successful installation + (and thereafter every time your computer starts up) however you will need to + configure your web browser(s) to use it. To do so, configure them to use a + proxy for HTTP and HTTPS at the address 127.0.0.1:8118. - Note that Privoxy is not automatically started at - boot time by default. You can change this with the rc-update - command. + To prevent the privoxy service from automatically starting when your computer + starts up, remove or rename the file /Library/LaunchDaemons/org.ijbswa.privoxy.plist + (on OS X 10.5 and higher) or the folder named + /Library/StartupItems/Privoxy (on OS X 10.4 'Tiger'). - - rc-update add privoxy default - + To manually start or stop the privoxy service, use the scripts startPrivoxy.sh + and stopPrivoxy.sh supplied in /Applications/Privoxy. They must be run from an + administrator account, using sudo. + - + Controlling Privoxy with Your Web Browser Privoxy's user interface can be reached through the special @@ -2615,7 +1439,7 @@ for details.         ▪  Documentation + url="https://www.privoxy.org/&p-version;/user-manual/">Documentation @@ -2637,10 +1461,7 @@ for details. it as a test to see whether it is Privoxy causing the problem or not. Privoxy continues to run as a proxy in this case, but all manipulation is disabled, i.e. - Privoxy acts like a normal forwarding proxy. There - is even a toggle Bookmarklet offered, so - that you can toggle Privoxy with one click from - your browser. + Privoxy acts like a normal forwarding proxy. @@ -3049,7 +1870,7 @@ for details. - + Finding the Right Mix Note that some actions, like cookie suppression @@ -3074,7 +1895,7 @@ for details. - + How to Edit The easiest way to edit the actions files is with a browser by @@ -3164,23 +1985,23 @@ for details. Generally, an URL pattern has the form - <domain><port>/<path>, where the - <domain>, the <port> + <host><port>/<path>, where the + <host>, the <port> and the <path> are optional. (This is why the special / pattern matches all URLs). Note that the protocol portion of the URL pattern (e.g. http://) should not be included in the pattern. This is assumed already! - The pattern matching syntax is different for the domain and path parts of - the URL. The domain part uses a simple globbing type matching technique, + The pattern matching syntax is different for the host and path parts of + the URL. The host part uses a simple globbing type matching technique, while the path part uses more flexible Regular Expressions (POSIX 1003.2). The port part of a pattern is a decimal port number preceded by a colon - (:). If the domain part contains a numerical IPv6 address, + (:). If the host part contains a numerical IPv6 address, it has to be put into angle brackets (<, >). @@ -3190,7 +2011,7 @@ for details. www.example.com/ - is a domain-only pattern and will match any request to www.example.com, + is a host-only pattern and will match any request to www.example.com, regardless of which document on that server is requested. So ALL pages in this domain would be covered by the scope of this action. Note that a simple example.com is different and would NOT match. @@ -3201,7 +2022,7 @@ for details. www.example.com - means exactly the same. For domain-only patterns, the trailing / may + means exactly the same. For host-only patterns, the trailing / may be omitted. @@ -3250,6 +2071,15 @@ for details. + + 10.0.0.1/ + + + Matches any URL with the host address 10.0.0.1. + (Note that the real URL uses plain brackets, not angle brackets.) + + + <2001:db8::1>/ @@ -3273,11 +2103,13 @@ for details. -The Domain Pattern +The Host Pattern - The matching of the domain part offers some flexible options: if the - domain starts or ends with a dot, it becomes unanchored at that end. + The matching of the host part offers some flexible options: if the + host pattern starts or ends with a dot, it becomes unanchored at that end. + The host pattern is often referred to as domain pattern as it is usually + used to match domain names and not IP addresses. For example: @@ -3384,7 +2216,7 @@ for details. -The Path Pattern +The Path Pattern Privoxy uses modern POSIX 1003.2 @@ -3484,18 +2316,18 @@ for details. -The Tag Pattern +The Request Tag Pattern - Tag patterns are used to change the applying actions based on the - request's tags. Tags can be created with either the - client-header-tagger + Request tag patterns are used to change the applying actions based on the + request's tags. Tags can be created based on HTTP headers with either + the client-header-tagger or the server-header-tagger action. - Tag patterns have to start with TAG:, so &my-app; - can tell them apart from URL patterns. Everything after the colon + Request tag patterns have to start with TAG:, so &my-app; + can tell them apart from other patterns. Everything after the colon including white space, is interpreted as a regular expression with path pattern syntax, except that tag patterns aren't left-anchored automatically (&my-app; doesn't silently add a ^, @@ -3511,15 +2343,15 @@ for details. - Sections can contain URL and tag patterns at the same time, - but tag patterns are checked after the URL patterns and thus + Sections can contain URL and request tag patterns at the same time, + but request tag patterns are checked after the URL patterns and thus always overrule them, even if they are located before the URL patterns. - Once a new tag is added, Privoxy checks right away if it's matched by one - of the tag patterns and updates the action settings accordingly. As a result - tags can be used to activate other tagger actions, as long as these other + Once a new request tag is added, Privoxy checks right away if it's matched by one + of the request tag patterns and updates the action settings accordingly. As a result + request tags can be used to activate other tagger actions, as long as these other taggers look for headers that haven't already be parsed. @@ -3543,6 +2375,79 @@ for details. + +The Negative Request Tag Patterns + + + To match requests that do not have a certain request tag, specify a negative tag pattern + by prefixing the tag pattern line with either NO-REQUEST-TAG: + or NO-RESPONSE-TAG: instead of TAG:. + + + + Negative request tag patterns created with NO-REQUEST-TAG: are checked + after all client headers are scanned, the ones created with NO-RESPONSE-TAG: + are checked after all server headers are scanned. In both cases all the created + tags are considered. + + + +The Client Tag Pattern + + + + + + This is an experimental feature. The syntax is likely to change in future versions. + + + + + Client tag patterns are not set based on HTTP headers but based on + the client's IP address. Users can enable them themselves, but the + Privoxy admin controls which tags are available and what their effect + is. + + + + After a client-specific tag has been defined with the + client-specific-tag, + directive, action sections can be activated based on the tag by using a + CLIENT-TAG pattern. The CLIENT-TAG pattern is evaluated at the same priority + as URL patterns, as a result the last matching pattern wins. Tags that + are created based on client or server headers are evaluated later on + and can overrule CLIENT-TAG and URL patterns! + + + The tag is set for all requests that come from clients that requested + it to be set. Note that "clients" are differentiated by IP address, + if the IP address changes the tag has to be requested again. + + + Clients can request tags to be set by using the CGI interface http://config.privoxy.org/client-tags. + + + + Example: + + + + +# If the admin defined the client-specific-tag circumvent-blocks, +# and the request comes from a client that previously requested +# the tag to be set, overrule all previous +block actions that +# are enabled based on URL to CLIENT-TAG patterns. +{-block} +CLIENT-TAG:^circumvent-blocks$ + +# This section is not overruled because it's located after +# the previous one. +{+block{Nobody is supposed to request this.}} +example.org/blocked-example-page + + @@ -3734,7 +2639,16 @@ for details. Example usage: - +add-header{X-User-Tracking: sucks} + # Add a DNT ("Do not track") header to all requests, +# event to those that already have one. +# +# This is just an example, not a recommendation. +# +# There is no reason to believe that user-tracking websites care +# about the DNT header and depending on the User-Agent, adding the +# header may make user-tracking easier. +{+add-header{DNT: 1}} +/ @@ -3944,7 +2858,7 @@ for details. Type: - Parameterized. + Multi-value. @@ -4031,7 +2945,7 @@ for details. Type: - Parameterized. + Multi-value. @@ -4752,6 +3666,94 @@ problem-host.example.com + + +external-filter + + + + Typical use: + + Modify content using a programming language of your choice. + + + + + Effect: + + + All instances of text-based type, most notably HTML and JavaScript, to which + this action applies, can be filtered on-the-fly through the specified external + filter. + By default plain text documents are exempted from filtering, because web + servers often use the text/plain MIME type for all files + whose type they don't know.) + + + + + + Type: + + + Multi-value. + + + + + Parameter: + + + The name of an external content filter, as defined in the + filter file. + External filters can be defined in one or more files as defined by the + filterfile + option in the config file. + + + When used in its negative form, + and without parameters, all filtering with external + filters is completely disabled. + + + + + + Notes: + + + External filters are scripts or programs that can modify the content in + case common filters + aren't powerful enough. With the exception that this action doesn't + use pcrs-based filters, the notes in the + filter section apply. + + + + Currently external filters are executed with &my-app;'s privileges. + Only use external filters you understand and trust. + + + + This feature is experimental, the syntax + may change in the future. + + + + + + + Example usage: + + + +external-filter{fancy-filter} + + + + + + fast-redirects @@ -4906,7 +3908,7 @@ problem-host.example.com Type: - Parameterized. + Multi-value. @@ -5014,7 +4016,7 @@ problem-host.example.com - +filter{js-events} # Kill all JS event bindings and timers (Radically destructive! Only for extra nasty sites). + +filter{js-events} # Kill JavaScript event bindings and timers (Radically destructive! Only for extra nasty sites). @@ -5026,15 +4028,15 @@ problem-host.example.com - +filter{refresh-tags} # Kill automatic refresh tags (for dial-on-demand setups). + +filter{refresh-tags} # Kill automatic refresh tags if refresh time is larger than 9 seconds. - +filter{unsolicited-popups} # Disable only unsolicited pop-up windows. Useful if your browser lacks this ability. + +filter{unsolicited-popups} # Disable only unsolicited pop-up windows. - +filter{all-popups} # Kill all popups in JavaScript and HTML. Useful if your browser lacks this ability. + +filter{all-popups} # Kill all popups in JavaScript and HTML. @@ -5064,6 +4066,10 @@ problem-host.example.com +filter{frameset-borders} # Give frames a border and make them resizable. + + + +filter{iframes} # Removes all detected iframes. Should only be enabled for individual sites. + +filter{demoronizer} # Fix MS's non-standard use of standard charsets. @@ -5219,7 +4225,7 @@ new action Type: - Multi-value. + Parameterized. @@ -5252,6 +4258,32 @@ new action for socks5 connections (with remote DNS resolution). + + + forward-webserver 127.0.0.1:80 to use the HTTP + server listening at 127.0.0.1 port 80 without adjusting the + request headers. + + + This makes it more convenient to use Privoxy to make + existing websites available as onion services as well. + + + Many websites serve content with hardcoded URLs and + can't be easily adjusted to change the domain based + on the one used by the client. + + + Putting Privoxy between Tor and the webserver (or an stunnel + that forwards to the webserver) allows to rewrite headers and + content to make client and server happy at the same time. + + + Using Privoxy for webservers that are only reachable through + onion addresses and whose location is supposed to be secret + is not recommended and should not be necessary anyway. + + @@ -5274,7 +4306,8 @@ new action If the ports are missing or invalid, default values will be used. This might change in the future and you shouldn't rely on it. Otherwise incorrect syntax causes Privoxy - to exit. + to exit. Due to design limitations, invalid parameter syntax isn't detected until the + action is used the first time. Use the show-url-info CGI page @@ -5289,15 +4322,17 @@ new action -# Always use direct connections for requests previously tagged as +# Use an ssh tunnel for requests previously tagged as # User-Agent: fetch libfetch/2.0 and make sure # resuming downloads continues to work. +# # This way you can continue to use Tor for your normal browsing, # without overloading the Tor network with your FreeBSD ports updates # or downloads of bigger files like ISOs. +# # Note that HTTP headers are easy to fake and therefore their # values are as (un)trustworthy as your clients and users. -{+forward-override{forward .} \ +{+forward-override{forward-socks5 10.0.0.2:2222 .} \ -hide-if-modified-since \ -overwrite-last-modified \ } @@ -6427,9 +5462,15 @@ new action filter file section. - This action will be ignored if you use it together with - block. - It can be combined with + Requests can't be blocked and redirected at the same time, + applying this action together with + block + is a configuration error. Currently the request is blocked + and an error message logged, the behavior may change in the + future and result in Privoxy rejecting the action file. + + + This action can be combined with fast-redirects{check-decoded-url} to redirect to a decoded version of a rewritten URL. @@ -6454,8 +5495,8 @@ new action example.com/stylesheet\.css # Create a short, easy to remember nickname for a favorite site -# (relies on the browser accept and forward invalid URLs to &my-app;) -{ +redirect{http://www.privoxy.org/user-manual/actions-file.html} } +# (relies on the browser to accept and forward invalid URLs to &my-app;) +{ +redirect{https://www.privoxy.org/user-manual/actions-file.html} } a # Always use the expanded view for Undeadly.org articles @@ -6472,6 +5513,19 @@ undeadly.org/cgi\?action=article&sid=\d*$ {+redirect{s@^http://[^/]*/results\.aspx\?q=([^&]*).*@http://search.yahoo.com/search?p=$1@}} search.msn.com//results\.aspx\?q= +# Redirect http://example.com/&bla=fasel&toChange=foo (and any other value but "bar") +# to http://example.com/&bla=fasel&toChange=bar +# +# The URL pattern makes sure that the following request isn't redirected again. +{+redirect{s@toChange=[^&]+@toChange=bar@}} +example.com/.*toChange=(?!bar) + +# Add a shortcut to look up illumos bugs +{+redirect{s@^http://i([0-9]+)/.*@https://www.illumos.org/issues/$1@}} +# Redirected URL = http://i4974/ +# Redirect Destination = https://www.illumos.org/issues/4974 +i[0-9][0-9][0-9][0-9]*/ + # Redirect remote requests for this manual # to the local version delivered by Privoxy {+redirect{s@^http://www@http://config@}} @@ -6512,7 +5566,7 @@ www.privoxy.org/user-manual/ Type: - Parameterized. + Multi-value. @@ -6595,7 +5649,7 @@ example.org/instance-that-is-delivered-as-xml-but-is-not Type: - Parameterized. + Multi-value. @@ -6640,6 +5694,14 @@ example.org/instance-that-is-delivered-as-xml-but-is-not # Tag every request with the content type declared by the server {+server-header-tagger{content-type}} / + +# If the response has a tag starting with 'image/' enable an external +# filter that only applies to images. +# +# Note that the filter is not available by default, it's just a +# silly example. +{+external-filter{rotate-image} +force-text-mode} +TAG:^image/ @@ -6858,7 +5920,7 @@ example.org/instance-that-is-delivered-as-xml-but-is-not - + Summary Note that many of these actions have the potential to cause a page to @@ -7001,7 +6063,7 @@ hal stop here and user.action file and see how all these pieces come together: - + match-all.action Remember all actions are disabled when matching starts, @@ -7044,7 +6106,7 @@ hal stop here - + default.action @@ -7333,7 +6395,7 @@ wiki. -user.action +user.action So far we are painting with a broad brush by setting general policies, @@ -7600,7 +6662,7 @@ stupid-server.example.com/ - &my-app; supports three different filter actions: + &my-app; supports three different pcrs-based filter actions: filter to rewrite the content that is send to the client, client-header-filter @@ -7620,6 +6682,13 @@ stupid-server.example.com/ applying actions through sections with tag-patterns. + + Finally &my-app; supports the + external-filter action + to enable external filters + written in proper programming languages. + + Multiple filter files can be defined through the in a syntax that imitates Perl's s/// operator. If you are familiar with Perl, you will find this to be quite intuitive, and may want to look at the - PCRS documentation for the subtle differences to Perl behaviour. Most - notably, the non-standard option letter U is supported, - which turns the default to ungreedy matching. + PCRS documentation for the subtle differences to Perl behaviour. + + + + Most notably, the non-standard option letter U is supported, + which turns the default to ungreedy matching (add ? to + quantifiers to turn them greedy again). + + + + The non-standard option letter D (dynamic) allows + to use the variables $host, $origin (the IP address the request came from), + $path, $url and $listen-address (the address on which Privoxy accepted the + client request. Example: 127.0.0.1:8118). + They will be replaced with the value they refer to before the filter + is executed. + + + + Note that '$' is a bad choice for a delimiter in a dynamic filter as you + might end up with unintended variables if you use a variable name + directly after the delimiter. Variables will be resolved without + escaping anything, therefore you also have to be careful not to chose + delimiters that appear in the replacement text. For example '<' should + be save, while '?' will sooner or later cause conflicts with $url. + + + + The non-standard option letter T (trivial) prevents + parsing for backreferences in the substitute. Use it if you want to include + text like '$&' in your substitute without quoting. @@ -7714,7 +6811,7 @@ stupid-server.example.com/ -Filter File Tutorial +Filter File Tutorial Now, let's complete our foo content filter. We have already defined the heading, but the jobs are still missing. Since all it does is to replace @@ -8411,6 +7508,79 @@ pre-defined filters for your convenience: + + +External filter syntax + + External filters are scripts or programs that can modify the content in + case common filters + aren't powerful enough. + + + External filters can be written in any language the platform &my-app; runs + on supports. + + + They are controlled with the + external-filter action + and have to be defined in the filterfile + first. + + + The header looks like any other filter, but instead of pcrs jobs, external + filters contain a single job which can be a program or a shell script (which + may call other scripts or programs). + + + External filters read the content from STDIN and write the rewritten + content to STDOUT. + The environment variables PRIVOXY_URL, PRIVOXY_PATH, PRIVOXY_HOST, + PRIVOXY_ORIGIN, PRIVOXY_LISTEN_ADDRESS can be used to get some details + about the client request. + + + &my-app; will temporary store the content to filter in the + temporary-directory. + + + +EXTERNAL-FILTER: cat Pointless example filter that doesn't actually modify the content +/bin/cat + +# Incorrect reimplementation of the filter above in POSIX shell. +# +# Note that it's a single job that spans multiple lines, the line +# breaks are not passed to the shell, thus the semicolons are required. +# +# If the script isn't trivial, it is recommended to put it into an external file. +# +# In general, writing external filters entirely in POSIX shell is not +# considered a good idea. +EXTERNAL-FILTER: cat2 Pointless example filter that despite its name may actually modify the content +while read line; \ +do \ + echo "$line"; \ +done + +EXTERNAL-FILTER: rotate-image Rotate an image by 180 degree. Test filter with limited value. +/usr/local/bin/convert - -rotate 180 - + +EXTERNAL-FILTER: citation-needed Adds a "[citation needed]" tag to an image. The coordinates may need adjustment. +/usr/local/bin/convert - -pointsize 16 -fill white -annotate +17+418 "[citation needed]" - + + + + + + Currently external filters are executed with &my-app;'s privileges! + Only use external filters you understand and trust. + + + + External filters are experimental and the syntax may change in the future. + + + @@ -8531,11 +7701,20 @@ Requests ©right; + + Privoxy is free software; you can + redistribute it and/or modify it under the terms of the + GNU General Public License, version 2, + as published by the Free Software Foundation and included in + the next section. + + -License - - &license; - +License + + + + @@ -8803,7 +7982,7 @@ Requests - + Privoxy's Internal Pages @@ -8920,84 +8099,6 @@ Requests - - These may be bookmarked for quick reference. See next. - - - - -Bookmarklets - - Below are some bookmarklets to allow you to easily access a - mini version of some of Privoxy's - special pages. They are designed for MS Internet Explorer, but should work - equally well in Netscape, Mozilla, and other browsers which support - JavaScript. They are designed to run directly from your bookmarks - not by - clicking the links below (although that should work for testing). - - - To save them, right-click the link and choose Add to Favorites - (IE) or Add Bookmark (Netscape). You will get a warning that - the bookmark may not be safe - just click OK. Then you can run the - Bookmarklet directly from your favorites/bookmarks. For even faster access, - you can put them on the Links bar (IE) or the Personal - Toolbar (Netscape), and run them with a single click. - - - - - - - - Privoxy - Enable - - - - - - Privoxy - Disable - - - - - - Privoxy - Toggle Privoxy (Toggles between enabled and disabled) - - - - - - Privoxy- View Status - - - - - - Privoxy - Why? - - - - - - - Credit: The site which gave us the general idea for these bookmarklets is - www.bookmarklets.com. They - have more information about bookmarklets. - - - - - @@ -9149,8 +8250,7 @@ Requests One quick test to see if Privoxy is causing a problem or not, is to disable it temporarily. This should be the first troubleshooting - step. See the Bookmarklets section on a quick - and easy way to do this (be sure to flush caches afterward!). Looking at the + step (be sure to flush caches afterward!). Looking at the logs is a good idea too. (Note that both the toggle feature and logging are enabled via config file settings, and may need to be turned on.)