Amuro Namie
Mark Nelson
Tobias Netzel
+ John Palkovic
Adam Piggott
Petr Písar
Dan Price
14) Allow to filter POST parameters.
-16) Filter SSL encrypted content as well.
-
- At the beginning we could use a unencrypted connection between
- client and Privoxy, and use an encrypted connection between
- Privoxy and the server.
-
- This should be good enough for most of the content the
- user would want to filter.
-
- Interested donors: 2.
-
19) enable-forward-fallback. Syntax? Suggested by K.R.
21) User Manual delivery doesn't accept multiple slashes. Should it?
Interested donors: 1.
-54) Move away from CVS to a more modern revision control system.
- The move to git is work in progress:
- https://sourceforge.net/p/ijbswa/mailman/message/34994343/
-
58) Move more template strings from the code into the actual templates.
59) Import the German template translation.
go through all filters and skip the filter types the caller isn't
interested in.
-98) When showing action section on the CGI pages, properly escape
+98) When showing action sections on the CGI pages, properly escape
line breaks so they can be copy&pasted into action files without
adjustments.
122) Allow customized log messages.
-123) Evaluate if the voluntarily-disclose-session-keys option in Firefox
- (and other browsers) can be leveraged. Probably depends on #16.
-
124) Add support for the "lightweight OS capability and sandbox framework"
Capsicum. http://www.cl.cam.ac.uk/research/security/capsicum/
Interested donors: 1.
performance point of view, using a single thread should reduce Privoxy's
memory footprint a bit which may be noticeable in case of multi-user setups
with hundreds of idle connections.
+
+159) Support Brotli compression.
+
##########################################################################
Hosting wish list (relevant for #53)
* File : $Source: /cvsroot/ijbswa/current/cgi.c,v $
*
* Purpose : Declares functions to intercept request, generate
- * html or gif answers, and to compose HTTP resonses.
+ * html or gif answers, and to compose HTTP responses.
* This only contains the framework functions, the
* actual handler functions are declared elsewhere.
*
- * Copyright : Written by and Copyright (C) 2001-2017
+ * Copyright : Written by and Copyright (C) 2001-2020
* members of the Privoxy team. https://www.privoxy.org/
*
* Based on the Internet Junkbuster originally written
#if defined(FEATURE_CGI_EDIT_ACTIONS) || defined(FEATURE_TOGGLE)
#include "cgiedit.h"
#endif /* defined(FEATURE_CGI_EDIT_ACTIONS) || defined (FEATURE_TOGGLE) */
+#ifdef FEATURE_HTTPS_INSPECTION
+#include "ssl.h"
+#endif
/* loadcfg.h is for global_toggle_state only */
#include "loadcfg.h"
/* jcc.h is for mutex semaphore globals only */
#include "jcc.h"
+static char *make_menu(const struct client_state *csp, const char *self);
+
/*
* List of CGI functions: name, handler, description
* Note: Do NOT use single quotes in the description;
"\000\000\000\000\111\105\116\104\256\102\140\202";
/*
- * 1x1 transparant PNG.
+ * 1x1 transparent PNG.
*/
const char image_blank_data[] =
"\211\120\116\107\015\012\032\012\000\000\000\015\111\110\104\122"
"\270\005\000\073";
/*
- * 1x1 transparant GIF.
+ * 1x1 transparent GIF.
*/
const char image_blank_data[] =
"GIF89a\001\000\001\000\200\000\000\377\377\377\000\000"
static char *grep_cgi_referrer(const struct client_state *csp)
{
struct list_entry *p;
+ struct list_entry *first_header =
+#ifdef FEATURE_HTTPS_INSPECTION
+ client_use_ssl(csp) ? csp->https_headers->first :
+#endif
+ csp->headers->first;
- for (p = csp->headers->first; p != NULL; p = p->next)
+ for (p = first_header; p != NULL; p = p->next)
{
if (p->str == NULL) continue;
if (strncmpic(p->str, "Referer: ", 9) == 0)
{
char *referrer;
static const char alternative_prefix[] = "http://" CGI_SITE_1_HOST "/";
+ static const char alt_prefix_https[] = "https://" CGI_SITE_1_HOST "/";
const char *trusted_cgi_referrer = csp->config->trusted_cgi_referrer;
referrer = grep_cgi_referrer(csp);
log_error(LOG_LEVEL_ERROR, "Denying access to %s. No referrer found.",
csp->http->url);
}
- else if ((0 == strncmp(referrer, CGI_PREFIX, sizeof(CGI_PREFIX)-1)
- || (0 == strncmp(referrer, alternative_prefix, strlen(alternative_prefix)))))
+ else if ((0 == strncmp(referrer, CGI_PREFIX, sizeof(CGI_PREFIX)-1))
+#ifdef FEATURE_HTTPS_INSPECTION
+ || (0 == strncmp(referrer, CGI_PREFIX_HTTPS, sizeof(CGI_PREFIX_HTTPS)-1))
+ || (0 == strncmp(referrer, alt_prefix_https, strlen(alt_prefix_https)))
+#endif
+ || (0 == strncmp(referrer, alternative_prefix, strlen(alternative_prefix))))
{
/* Trustworthy referrer */
log_error(LOG_LEVEL_CGI, "Granting access to %s, referrer %s is trustworthy.",
*
* Function : get_string_param
*
- * Description : Get a string paramater, to be used as an
- * ACTION_STRING or ACTION_MULTI paramater.
+ * Description : Get a string parameter, to be used as an
+ * ACTION_STRING or ACTION_MULTI parameter.
* Validates the input to prevent stupid/malicious
* users from corrupting their action file.
*
* Parameters :
* 1 : parameters = map of cgi parameters
* 2 : param_name = The name of the parameter to read
- * 3 : pparam = destination for paramater. Allocated as
+ * 3 : pparam = destination for parameter. Allocated as
* part of the map "parameters", so don't free it.
* Set to NULL if not specified.
*
- * Returns : JB_ERR_OK on success, or if the paramater
+ * Returns : JB_ERR_OK on success, or if the parameter
* was not specified.
* JB_ERR_MEMORY on out-of-memory.
- * JB_ERR_CGI_PARAMS if the paramater is not valid.
+ * JB_ERR_CGI_PARAMS if the parameter is not valid.
*
*********************************************************************/
jb_err get_string_param(const struct map *parameters,
* In this context, "unexpected" means "anything other
* than JB_ERR_MEMORY or JB_ERR_CGI_PARAMS" - CGIs are
* expected to handle all other errors internally,
- * since they can give more relavent error messages
+ * since they can give more relevant error messages
* that way.
*
* Note this is not a true CGI, it takes an error
* unless the client asked for HTTP/1.0.
*/
snprintf(buf, sizeof(buf), "%s %s",
- strcmpic(csp->http->ver, "HTTP/1.0") ? "HTTP/1.1" : "HTTP/1.0",
+ strcmpic(csp->http->version, "HTTP/1.0") ? "HTTP/1.1" : "HTTP/1.0",
rsp->status ? rsp->status : "200 OK");
err = enlist_first(rsp->headers, buf);
if (!err) err = map(exports, "my-hostname", 1, html_encode(hostname ? hostname : "unknown"), 0);
freez(hostname);
if (!err) err = map(exports, "homepage", 1, html_encode(HOME_PAGE_URL), 0);
- if (!err) err = map(exports, "default-cgi", 1, html_encode(CGI_PREFIX), 0);
- if (!err) err = map(exports, "menu", 1, make_menu(caller, csp->config->feature_flags), 0);
+ if (!err)
+ {
+ err = map(exports, "default-cgi", 1, html_encode(
+#ifdef FEATURE_HTTPS_INSPECTION
+ client_use_ssl(csp) ? CGI_PREFIX_HTTPS :
+#endif
+ CGI_PREFIX), 0);
+ }
+ if (!err) err = map(exports, "menu", 1, make_menu(csp, caller), 0);
if (!err) err = map(exports, "code-status", 1, CODE_STATUS, 1);
if (!strncmpic(csp->config->usermanual, "file://", 7) ||
!strncmpic(csp->config->usermanual, "http", 4))
else
{
/* Manual is delivered by Privoxy. */
- if (!err) err = map(exports, "user-manual", 1, html_encode(CGI_PREFIX"user-manual/"), 0);
+ if (!err)
+ {
+ err = map(exports, "user-manual", 1, html_encode(
+#ifdef FEATURE_HTTPS_INSPECTION
+ client_use_ssl(csp) ? CGI_PREFIX_HTTPS"user-manual/" :
+#endif
+ CGI_PREFIX"user-manual/"), 0);
+ }
}
if (!err) err = map(exports, "actions-help-prefix", 1, ACTIONS_HELP_PREFIX ,1);
#ifdef FEATURE_TOGGLE
* and the toggle CGI if toggling is disabled.
*
* Parameters :
- * 1 : self = name of CGI to leave out, can be NULL for
+ * 1 : csp = Current client state (buffers, headers, etc...)
+ * 2 : self = name of CGI to leave out, can be NULL for
* complete listing.
- * 2 : feature_flags = feature bitmap from csp->config
- *
*
* Returns : menu string, or NULL on out-of-memory error.
*
*********************************************************************/
-char *make_menu(const char *self, const unsigned feature_flags)
+char *make_menu(const struct client_state *csp, const char *self)
{
const struct cgi_dispatcher *d;
char *result = strdup("");
{
#ifdef FEATURE_TOGGLE
- if (!(feature_flags & RUNTIME_FEATURE_CGI_TOGGLE) && !strcmp(d->name, "toggle"))
+ if (!(csp->config->feature_flags & RUNTIME_FEATURE_CGI_TOGGLE) && !strcmp(d->name, "toggle"))
{
/*
* Suppress the toggle link if remote toggling is disabled.
* the "blocked" template's JavaScript.
*/
string_append(&result, "<li><a href=\"");
- html_encoded_prefix = html_encode(CGI_PREFIX);
+ html_encoded_prefix = html_encode(
+#ifdef FEATURE_HTTPS_INSPECTION
+ client_use_ssl(csp) ? CGI_PREFIX_HTTPS :
+#endif
+ CGI_PREFIX);
if (html_encoded_prefix == NULL)
{
return NULL;
* File : $Source: /cvsroot/ijbswa/current/cgi.h,v $
*
* Purpose : Declares functions to intercept request, generate
- * html or gif answers, and to compose HTTP resonses.
+ * html or gif answers, and to compose HTTP responses.
*
* Functions declared include:
*
*/
extern void get_http_time(int time_offset, char *buf, size_t buffer_size);
extern char *add_help_link(const char *item, struct configuration_spec *config);
-extern char *make_menu(const char *self, const unsigned feature_flags);
extern char *dump_map(const struct map *the_map);
/*
/* alias_list contains the aliases defined in this file.
* It might be better to use the "file_line.data" fields
- * in the relavent places instead.
+ * in the relevant places instead.
*/
cur_line = file->lines;
* browsers (BR #1570678).
*
* The config option split-large-forms works around this browser
- * bug (HTTP has no URL length limitation) by deviding the action
+ * bug (HTTP has no URL length limitation) by dividing the action
* list form into multiple smaller ones. It means the URLs are shorter
* and work in broken browsers as well, but the user can no longer change
* all actions with one submit.
#ifndef FEATURE_EXTERNAL_FILTERS
if (!err) err = map_block_killer(exports, "external-content-filters");
#endif
+#ifndef FEATURE_HTTP_INSPECTION
+ if (!err) err = map_block_killer(exports, "https-inspection");
+#endif
if (err)
{
if (url_param[0] == '\0')
{
- /* URL paramater not specified, display query form only. */
+ /* URL parameter not specified, display query form only. */
free(url_param);
if (map_block_killer(exports, "url-given")
|| map(exports, "url", 1, "", 1))
}
/*
- * We have a warning about SSL paths. Hide it for unencrypted sites.
+ * We have a warning about SSL paths. Hide it for unencrypted sites
+ * and unconditionally if https inspection is enabled.
*/
+#ifndef FEATURE_HTTPS_INSPECTION
if (!url_to_query->ssl)
+#endif
{
- if (map_block_killer(exports, "https"))
+ if (map_block_killer(exports, "https-and-no-https-inspection"))
{
free_current_action(action);
free_map(exports);
*
* Function : show_defines
*
- * Description : Add to a map the state od all conditional #defines
+ * Description : Add to a map the state of all conditional #defines
* used when building
*
* Parameters :
* File : $Source: /cvsroot/ijbswa/current/cgisimple.h,v $
*
* Purpose : Declares functions to intercept request, generate
- * html or gif answers, and to compose HTTP resonses.
+ * html or gif answers, and to compose HTTP responses.
*
* Functions declared include:
*
#
# The available debug levels are:
#
-# debug 1 # Log the destination for each request Privoxy let through. See also debug 1024.
+# debug 1 # Log the destination for each request. See also debug 1024.
# debug 2 # show each connection status
# debug 4 # show I/O status
# debug 8 # show header parsing
# you read the log messages, you may even be able to solve the
# problem on your own.
#
-#debug 1 # Log the destination for each request Privoxy let through. See also debug 1024.
+#debug 1 # Log the destination for each request.
#debug 1024 # Log the destination for requests Privoxy didn't let through, and the reason why.
#debug 4096 # Startup banner and warnings
#debug 8192 # Non-fatal errors
# whole destination part are optional.
#
# If your system implements RFC 3493, then src_addr and dst_addr
-# can be IPv6 addresses delimeted by brackets, port can be a
+# can be IPv6 addresses delimited by brackets, port can be a
# number or a service name, and src_masklen and dst_masklen can
# be a number from 0 to 128.
#
# Notes:
#
# Under high load incoming connection may queue up before
-# Privoxy gets around to serve them. The queue length is
-# limitted by the operating system. Once the queue is full,
-# additional connections are dropped before Privoxy can accept
-# and serve them.
+# Privoxy gets around to serve them. The queue length is limited
+# by the operating system. Once the queue is full, additional
+# connections are dropped before Privoxy can accept and serve
+# them.
#
# Increasing the queue length allows Privoxy to accept more
-# incomming connections that arrive roughly at the same time.
+# incoming connections that arrive roughly at the same time.
#
# Note that Privoxy can only request a certain queue length,
# whether or not the requested length is actually used depends
* an (optional) image block and an arbitrary number
* of image extension blocks, produce an output GIF with
* only one image block that contains the last image
- * (extenstion) block of the original.
- * Also strip Comments, Application extenstions, etc.
+ * (extension) block of the original.
+ * Also strip Comments, Application extensions, etc.
*
* Parameters :
* 1 : src = Pointer to the source binbuffer
-privoxy (3.0.29~git6a358e-1) UNRELEASED; urgency=medium
+privoxy (3.0.29~gitec5b42-1) UNRELEASED; urgency=medium
* New upstream developement version.
* Update all patches to new version.
* 36_trusted-cgi-referer-example and 37_ppedit-tests403 are now
incorporated upstream.
+ * d/copyright: use ./ prefix to upstream filenames, because orig.tar.gz
+ is build with a ./ prefix.
- -- Roland Rosenfeld <roland@debian.org> Sat, 30 Nov 2019 12:44:20 +0100
+ -- Roland Rosenfeld <roland@debian.org> Sat, 04 Apr 2020 14:46:51 +0200
+
+privoxy (3.0.28-3) unstable; urgency=medium
+
+ * Add buildlog to salsa-ci test pipeline.
+ * d/salsa-ci.yml stripped down using pipline-jobs.yml
+ * Build-depend on debhelper-compat (= 12) instead of using d/compat.
+ * Upgrade to Standards-Version 4.5.0 (no changes).
+ * Add upstream metadata.
+ * privoxy.service: run After=network-online.target, this fixes
+ LP#1870101.
+
+ -- Roland Rosenfeld <roland@debian.org> Thu, 02 Apr 2020 18:04:44 +0200
privoxy (3.0.28-2) unstable; urgency=medium
Priority: optional
Maintainer: Roland Rosenfeld <roland@debian.org>
Build-Depends: autoconf,
- debhelper (>= 12~),
+ debhelper-compat (= 12),
docbook,
docbook-dsssl,
docbook-utils,
sgmlspl,
w3m,
zlib1g-dev
-Standards-Version: 4.3.0
+Standards-Version: 4.5.0
Homepage: https://www.privoxy.org/
Vcs-Git: https://salsa.debian.org/debian/privoxy.git
Vcs-Browser: https://salsa.debian.org/debian/privoxy
Junkbusters Corporation. http://www.junkbusters.com
License: GPL-2+
-Files: regression-tests.action tools/uagen.pl tools/privoxy-regression-test.pl
- tools/privoxy-log-parser.pl tools/url-pattern-translator.pl utils/changelog2doc.pl
- utils/create-package-feed.pl
+Files: ./regression-tests.action ./tools/uagen.pl ./tools/privoxy-regression-test.pl
+ ./tools/privoxy-log-parser.pl ./tools/url-pattern-translator.pl ./utils/changelog2doc.pl
+ ./utils/create-package-feed.pl
Copyright:
Copyright (c) 2006-2017 Fabian Keil <fk@fabiankeil.de>
License: ISC
-Files: strptime.h
+Files: ./strptime.h
Copyright:
Copyright (C) 1996, 1997, 1998, 1999, 2000 Free Software Foundation, Inc.
Comment:
2002-2018 Roland Rosenfeld <roland@debian.org>
License: GPL-2+
-Files: pcre/*
+Files: ./pcre/*
Copyright: Copyright (c) 1997-2000 University of Cambridge
Comment:
Written by: Philip Hazel <ph10@cam.ac.uk>
--- a/default.action.master
+++ b/default.action.master
-@@ -2433,6 +2433,9 @@ schneegans.de/sv/\?url=referer
+@@ -2435,6 +2435,9 @@ schneegans.de/sv/\?url=referer
#MASTER# REMARKS: Exclude per Debian bug report #377843
# URL = http://blogs.msdn.com/wga/archive/2006/07/16/667063.aspx
blogs.msdn.com
{-filter{unsolicited-popups}}
# Sticky Actions = -filter{unsolicited-popups}
-@@ -2491,6 +2494,11 @@ tr.anp.se/
+@@ -2493,6 +2496,11 @@ tr.anp.se/
{+filter{tiny-textforms}}
.sourceforge.net/tracker
[Unit]
Description=Privacy enhancing HTTP Proxy
Documentation=man:privoxy(8) https://www.privoxy.org/user-manual/
-After=network.target
+After=network-online.target
[Service]
Environment=PIDFILE=/run/privoxy.pid
Tests: privoxy-regression-test
-Depends: @, curl
+Depends: curl, @
--- /dev/null
+Archive: SourceForge
+Bug-Database: https://sourceforge.net/p/ijbswa/bugs/
+Bug-Submit: https://sourceforge.net/p/ijbswa/bugs/new/
+Changelog: https://www.privoxy.org/gitweb/?p=privoxy.git;a=blob_plain;f=ChangeLog
+Documentation: https://www.privoxy.org/user-manual/
+Donation: https://www.privoxy.org/faq/general.html#DONATE
+FAQ: https://www.privoxy.org/faq/
+Repository: https://www.privoxy.org/git/privoxy.git
+Repository-Browse: https://www.privoxy.org/gitweb/?p=privoxy.git
+Security-Contact: Fabian Keil <fk@fabiankeil.de>
# Compulsory line, this is a version 3 file
version=3
-#http://sourceforge.net/project/showfiles.php?group_id=11118 http://prdownloads.sourceforge.net/ijbswa/privoxy-(.*)-stable-src.tar.gz\?download
+#https://sourceforge.net/project/showfiles.php?group_id=11118 https://prdownloads.sourceforge.net/ijbswa/privoxy-(.*)-stable-src.tar.gz\?download
opts=pgpsigurlmangle=s/$/.asc/ https://sf.net/ijbswa/privoxy-(\d+.*)-(?:stable|beta)-src.tar.gz
# and enhancements are better placed in user.action,
# the match-all section has been moved to match-all.action.
#
-# Copyright : Written by and Copyright (C) 2001-2019 the
+# Copyright : Written by and Copyright (C) 2001-2020 the
# Privoxy team. https://www.privoxy.org/
#
# Feedback welcome, for details please have a look at:
# https://www.privoxy.org/user-manual/contact.html
#
-# The current development version of this file is located:
-# http://ijbswa.cvs.sourceforge.net/viewvc/ijbswa/current/default.action.master
+# The current development version of this file is located at:
+# https://www.privoxy.org/gitweb/?p=privoxy.git;a=blob_plain;f=default.action.master;hb=HEAD
#
#############################################################################
# Syntax
# Client-header filters predefined in the supplied default.filter include:
#
# hide-tor-exit-notation: Removes the Tor exit node notation in Host and Referer headers.
+# no-brotli-accepted: Strips "br" from the Accept-Encoding header
# privoxy-control: Removes X-Privoxy-Control headers.
#
# +client-header-tagger{string}
#
{ \
+change-x-forwarded-for{block} \
++client-header-filter{no-brotli-accepted} \
+client-header-tagger{css-requests} \
+client-header-tagger{image-requests} \
+client-header-tagger{range-requests} \
#
{ \
+change-x-forwarded-for{block} \
++client-header-filter{no-brotli-accepted} \
+client-header-tagger{css-requests} \
+client-header-tagger{image-requests} \
+client-header-tagger{range-requests} \
#
{ \
+change-x-forwarded-for{block} \
++client-header-filter{no-brotli-accepted} \
+client-header-tagger{css-requests} \
+client-header-tagger{image-requests} \
+client-header-tagger{range-requests} \
disqus.com/
# URL = http://www.youtube.com/embed/blafasel?feature=oembed&enablejsapi=1&origin=http://www.example.com&wmode=opaque
.youtube.com/.*origin=http
+# URL = https://oss-fuzz.com/login?dest=https%3A%2F%2Foss-fuzz.com%2Ftestcase-detail%2F5727799779524608
+#MASTER# REMARKS: Only match based on the host so the test works even without https-inspection enabled.
+oss-fuzz.com/
{+redirect{s@.*url=@http://@} -block}
# Sticky Actions = +redirect -block
#
# Purpose : Rules to process the content of web pages
#
-# Copyright : Written by and Copyright (C) 2001-2018 the
+# Copyright : Written by and Copyright (C) 2001-2020 the
# Privoxy team. https://www.privoxy.org/
#
# This program is free software; you can redistribute it
s@^((?:Referer|Host):\s*(?:https?://)?[^/]*)\.[^\./]*?\.exit@$1@i
+#################################################################################
+#
+# no-brotli-accepted: Strips "br" from the Accept-Encoding header as Privoxy
+# currently doesn't support Brotli.
+#
+#################################################################################
+CLIENT-HEADER-FILTER: no-brotli-accepted Strip "br" from Accept-Encoding header
+
+s@(^Accept-Encoding:.*?)(?:br, |, br|br)@$1@i
+
#################################################################################
#
# less-download-windows: Prevents annoying download windows for content types
Purpose : Entity included in other project documents.
- Copyright (C) 2001-2019 Privoxy Developers https://www.privoxy.org/
+ Copyright (C) 2001-2020 Privoxy Developers https://www.privoxy.org/
See LICENSE.
======================================================================
-->
<para>
- Copyright &my-copy; 2001-2019 by Privoxy Developers <email>privoxy-devel@lists.privoxy.org</email>
+ Copyright &my-copy; 2001-2020 by Privoxy Developers <email>privoxy-devel@lists.privoxy.org</email>
</para>
<para>
The main compilation takes place with IBM Visual Age C++.
Some ancillary work takes place with GNU tools, available from
various sources like hobbes.nmsu.edu.
- Specificially, you will need <filename>autoheader</filename>,
+ Specifically, you will need <filename>autoheader</filename>,
<filename>autoconf</filename> and <filename>sh</filename> tools.
The packaging takes place with WarpIN, available from various sources, including
its home page: <ulink url="http://www.xworkplace.org/">xworkplace</ulink>.
It is not a substitute for the
<ulink url="../user-manual/index.html"><citetitle>Privoxy User Manual</citetitle></ulink>.
<!--
- This works, at least in some situtations:
+ This works, at least in some situations:
Test: <ulink url="privoxy-user-manual.pdf"><citetitle>User Manual</citetitle></ulink>.
-->
</para>
this is not considered a Privoxy bug.
</para>
<para>
- To prevent the crashes you can rewrite your filter to use less ressources,
+ To prevent the crashes you can rewrite your filter to use less resources,
increase the relevant memory limit or recompile pcre to use less stack space.
For details please see the
<ulink url="http://pcre.org/original/doc/html/pcrestack.html">pcrestack man page</ulink>
Amuro Namie
Mark Nelson
Tobias Netzel
+ John Palkovic
Adam Piggott
Petr Písar
Dan Price
The available debug levels are:
</para>
<programlisting>
- debug 1 # Log the destination for each request &my-app; let through. See also debug 1024.
+ debug 1 # Log the destination for each request. See also debug 1024.
debug 2 # show each connection status
debug 4 # show I/O status
debug 8 # show header parsing
If your system implements
<ulink url="http://tools.ietf.org/html/rfc3493">RFC 3493</ulink>, then
<replaceable class="parameter">src_addr</replaceable> and <replaceable
- class="parameter">dst_addr</replaceable> can be IPv6 addresses delimeted by
+ class="parameter">dst_addr</replaceable> can be IPv6 addresses delimited by
brackets, <replaceable class="parameter">port</replaceable> can be a number
or a service name, and
<replaceable class="parameter">src_masklen</replaceable> and
<listitem>
<para>
Under high load incoming connection may queue up before Privoxy
- gets around to serve them. The queue length is limitted by the
+ gets around to serve them. The queue length is limited by the
operating system. Once the queue is full, additional connections
are dropped before Privoxy can accept and serve them.
</para>
<para>
Increasing the queue length allows Privoxy to accept more
- incomming connections that arrive roughly at the same time.
+ incoming connections that arrive roughly at the same time.
</para>
<para>
Note that Privoxy can only request a certain queue length,
<!entity % p-not-stable "INCLUDE">
<!entity % p-stable "IGNORE">
<!entity % p-text "IGNORE"> <!-- define we are not a text only doc -->
-<!entity % p-authors-formal "IGNORE"> <!-- exclude additional formating -->
+<!entity % p-authors-formal "IGNORE"> <!-- exclude additional formatting -->
<!entity my-copy "(C)"> <!-- db2man barfs on copyright symbol -->
<!entity % seealso-extra "IGNORE"> <!-- for excluding sections of seealso -->
]>
looks for the string <quote>http://</quote>, either in plain text
(invalid but often used) or encoded as <quote>http%3a//</quote>.
Some sites use their own URL encoding scheme, encrypt the address
- of the target server or replace it with a database id. In theses cases
+ of the target server or replace it with a database id. In these cases
<literal>fast-redirects</literal> is fooled and the request reaches the
redirection server where it probably gets logged.
</para>
<para>
Note that some (rare) ill-configured sites don't handle requests for uncompressed
documents correctly. Broken PHP applications tend to send an empty document body,
- some IIS versions only send the beginning of the content. If you enable
- <literal>prevent-compression</literal> per default, you might want to add
- exceptions for those sites. See the example for how to do that.
+ some IIS versions only send the beginning of the content and some content delivery
+ networks let the connection time out.
+ If you enable <literal>prevent-compression</literal> per default, you might
+ want to add exceptions for those sites. See the example for how to do that.
</para>
</listitem>
</varlistentry>
sometimes appear on some pages, or user agents that don't correct for this on
the fly.
<!--
- My version of Mozilla (ancient) shows litte square boxes for quote
+ My version of Mozilla (ancient) shows little square boxes for quote
characters, and apostrophes on moronized pages. So many pages have this, I
can read them fine now. HB 08/27/06
-->
</tr>
</table>
<p>You will need a mix of development tools. The main compilation takes place with IBM Visual Age C++. Some
- ancillary work takes place with GNU tools, available from various sources like hobbes.nmsu.edu. Specificially,
+ ancillary work takes place with GNU tools, available from various sources like hobbes.nmsu.edu. Specifically,
you will need <tt class="FILENAME">autoheader</tt>, <tt class="FILENAME">autoconf</tt> and <tt class=
"FILENAME">sh</tt> tools. The packaging takes place with WarpIN, available from various sources, including its
home page: <a href="http://www.xworkplace.org/" target="_top">xworkplace</a>.</p>
</div>
<div class="SECT1">
<h1 class="SECT1"><a name="COPYRIGHT" id="COPYRIGHT">7. Privoxy Copyright, License and History</a></h1>
- <p>Copyright © 2001-20
19 by Privoxy Developers <code class="EMAIL"><<a href=
+ <p>Copyright © 2001-20
20 by Privoxy Developers <code class="EMAIL"><<a href=
"mailto:privoxy-devel@lists.privoxy.org">privoxy-devel@lists.privoxy.org</a>></code></p>
<p>Some source code is based on code Copyright © 1997 by Anonymous Coders and Junkbusters, Inc. and licensed
under the <i class="CITETITLE">GNU General Public License</i>.</p>
sufficient.</p>
<p>Unless the problem occurs with the filters available in the default configuration, this is not considered a
Privoxy bug.</p>
- <p>To prevent the crashes you can rewrite your filter to use less ressources, increase the relevant memory limit
+ <p>To prevent the crashes you can rewrite your filter to use less resources, increase the relevant memory limit
or recompile pcre to use less stack space. For details please see the <a href=
"http://pcre.org/original/doc/html/pcrestack.html" target="_top">pcrestack man page</a> and the documentation of
your operating system.</p>
<p><a href="https://www.top10vpn.com">https://www.top10vpn.com</a></p>
<h3>Bronze sponsor</h3>
<p><a href="https://www.betrugstest.com/">https://www.betrugstest.com/</a></p>
+ <p><a href="https://www.vpncompare.co.uk/">https://www.vpncompare.co.uk/</a></p>
<h3>Becoming a Privoxy sponsor</h3>
<p>If you want to become a sponsor, please have a look at the <a title=
"FAQ: How can I become a sponsor and get my logo or link on privoxy.org?" href="../faq/general.html#SPONSOR">sponsor
<p>The default profiles, and their associated actions, as pre-defined in <tt class=
"FILENAME">default.action</tt> are:</p>
<div class="TABLE">
- <a name="AEN3105" id="AEN3105"></a>
+ <a name="AEN3104" id="AEN3104"></a>
<p><b>Table 1. Default Configurations</b></p>
<table border="1" frame="border" rules="all" class="CALSTABLE">
<col width="1*" title="C1">
<p>To detect a redirection URL, <tt class="LITERAL">fast-redirects</tt> only looks for the string
<span class="QUOTE">"http://"</span>, either in plain text (invalid but often used) or encoded as
<span class="QUOTE">"http%3a//"</span>. Some sites use their own URL encoding scheme, encrypt the address
- of the target server or replace it with a database id. In theses cases <tt class=
+ of the target server or replace it with a database id. In these cases <tt class=
"LITERAL">fast-redirects</tt> is fooled and the request reaches the redirection server where it probably
gets logged.</p>
</dd>
action settings.</p>
<p>Note that some (rare) ill-configured sites don't handle requests for uncompressed documents correctly.
Broken PHP applications tend to send an empty document body, some IIS versions only send the beginning of
- the content. If you enable <tt class="LITERAL">prevent-compression</tt> per default, you might want to
- add exceptions for those sites. See the example for how to do that.</p>
+ the content and some content delivery networks let the connection time out. If you enable <tt class=
+ "LITERAL">prevent-compression</tt> per default, you might want to add exceptions for those sites. See the
+ example for how to do that.</p>
</dd>
<dt>Example usage (sections):</dt>
<dd>
these. If not, you will get a friendly error message. Internet access is not necessary either.</p>
<ul>
<li>
- <p>Privoxy main page:</p><a name="AEN6290" id="AEN6290"></a>
+ <p>Privoxy main page:</p><a name="AEN6289" id="AEN6289"></a>
<blockquote class="BLOCKQUOTE">
<p><a href="http://config.privoxy.org/" target="_top">http://config.privoxy.org/</a></p>
</blockquote>
"APPLICATION">Privoxy</span>)</p>
</li>
<li>
- <p>View and toggle client tags:</p><a name="AEN6298" id="AEN6298"></a>
+ <p>View and toggle client tags:</p><a name="AEN6297" id="AEN6297"></a>
<blockquote class="BLOCKQUOTE">
<p><a href="http://config.privoxy.org/client-tags" target=
"_top">http://config.privoxy.org/client-tags</a></p>
</li>
<li>
<p>Show information about the current configuration, including viewing and editing of actions
- files:</p><a name="AEN6303" id="AEN6303"></a>
+ files:</p><a name="AEN6302" id="AEN6302"></a>
<blockquote class="BLOCKQUOTE">
<p><a href="http://config.privoxy.org/show-status" target=
"_top">http://config.privoxy.org/show-status</a></p>
</blockquote>
</li>
<li>
- <p>Show the browser's request headers:</p><a name="AEN6308" id="AEN6308"></a>
+ <p>Show the browser's request headers:</p><a name="AEN6307" id="AEN6307"></a>
<blockquote class="BLOCKQUOTE">
<p><a href="http://config.privoxy.org/show-request" target=
"_top">http://config.privoxy.org/show-request</a></p>
</blockquote>
</li>
<li>
- <p>Show which actions apply to a URL and why:</p><a name="AEN6313" id="AEN6313"></a>
+ <p>Show which actions apply to a URL and why:</p><a name="AEN6312" id="AEN6312"></a>
<blockquote class="BLOCKQUOTE">
<p><a href="http://config.privoxy.org/show-url-info" target=
"_top">http://config.privoxy.org/show-url-info</a></p>
<li>
<p>Toggle Privoxy on or off. This feature can be turned off/on in the main <tt class="FILENAME">config</tt>
file. When toggled <span class="QUOTE">"off"</span>, <span class="QUOTE">"Privoxy"</span> continues to run,
- but only as a pass-through proxy, with no actions taking place:</p><a name="AEN6321" id="AEN6321"></a>
+ but only as a pass-through proxy, with no actions taking place:</p><a name="AEN6320" id="AEN6320"></a>
<blockquote class="BLOCKQUOTE">
<p><a href="http://config.privoxy.org/toggle" target="_top">http://config.privoxy.org/toggle</a></p>
</blockquote>
- <p>Short cuts. Turn off, then on:</p><a name="AEN6325" id="AEN6325"></a>
+ <p>Short cuts. Turn off, then on:</p><a name="AEN6324" id="AEN6324"></a>
<blockquote class="BLOCKQUOTE">
<p><a href="http://config.privoxy.org/toggle?set=disable" target=
"_top">http://config.privoxy.org/toggle?set=disable</a></p>
- </blockquote><a name="AEN6328" id="AEN6328"></a>
+ </blockquote><a name="AEN6327" id="AEN6327"></a>
<blockquote class="BLOCKQUOTE">
<p><a href="http://config.privoxy.org/toggle?set=enable" target=
"_top">http://config.privoxy.org/toggle?set=enable</a></p>
<table border="0" bgcolor="#E0E0E0" width="90%">
<tr>
<td>
- <pre class="PROGRAMLISTING"> debug 1 # Log the destination for each request <span class=
- "APPLICATION">Privoxy</span> let through. See also debug 1024.
+ <pre class=
+ "PROGRAMLISTING"> debug 1 # Log the destination for each request. See also debug 1024.
debug 2 # show each connection status
debug 4 # show I/O status
debug 8 # show header parsing
destination part are optional.</p>
<p>If your system implements <a href="http://tools.ietf.org/html/rfc3493" target="_top">RFC 3493</a>,
then <tt class="REPLACEABLE"><i>src_addr</i></tt> and <tt class="REPLACEABLE"><i>dst_addr</i></tt> can be
- IPv6 addresses delimeted by brackets, <tt class="REPLACEABLE"><i>port</i></tt> can be a number or a
+ IPv6 addresses delimited by brackets, <tt class="REPLACEABLE"><i>port</i></tt> can be a number or a
service name, and <tt class="REPLACEABLE"><i>src_masklen</i></tt> and <tt class=
"REPLACEABLE"><i>dst_masklen</i></tt> can be a number from 0 to 128.</p>
</dd>
<dt>Notes:</dt>
<dd>
<p>Under high load incoming connection may queue up before Privoxy gets around to serve them. The queue
- length is limitted by the operating system. Once the queue is full, additional connections are dropped
+ length is limited by the operating system. Once the queue is full, additional connections are dropped
before Privoxy can accept and serve them.</p>
- <p>Increasing the queue length allows Privoxy to accept more incomming connections that arrive roughly at
+ <p>Increasing the queue length allows Privoxy to accept more incoming connections that arrive roughly at
the same time.</p>
<p>Note that Privoxy can only request a certain queue length, whether or not the requested length is
actually used depends on the operating system which may use a different length instead.</p>
</div>
<div class="SECT1">
<h1 class="SECT1"><a name="COPYRIGHT">12. Privoxy Copyright, License and History</a></h1>
- <p>Copyright © 2001-20
19 by Privoxy Developers <code class="EMAIL"><<a href=
+ <p>Copyright © 2001-20
20 by Privoxy Developers <code class="EMAIL"><<a href=
"mailto:privoxy-devel@lists.privoxy.org">privoxy-devel@lists.privoxy.org</a>></code></p>
<p>Some source code is based on code Copyright © 1997 by Anonymous Coders and Junkbusters, Inc. and licensed
under the <i class="CITETITLE">GNU General Public License</i>.</p>
Amuro Namie<br>
Mark Nelson<br>
Tobias Netzel<br>
+ John Palkovic<br>
Adam Piggott<br>
Petr Písar<br>
Dan Price<br>
case LOG_LEVEL_FATAL:
log_level_string = "Fatal error";
break;
- case LOG_LEVEL_GPC:
+ case LOG_LEVEL_REQUEST:
log_level_string = "Request";
break;
case LOG_LEVEL_CONNECT:
* the taskbar icon animate. (There is an option to disable
* this but checking that is handled inside LogShowActivity()).
*/
- if ((loglevel == LOG_LEVEL_GPC) || (loglevel == LOG_LEVEL_CRUNCH))
+ if ((loglevel == LOG_LEVEL_REQUEST) || (loglevel == LOG_LEVEL_CRUNCH))
{
LogShowActivity();
}
break;
case 'c':
/*
- * Note that char paramaters are converted to int, so we need to
+ * Note that char parameters are converted to int, so we need to
* pass "int" to va_arg. (See K&R, 2nd ed, section A7.3.2, page 202)
*/
tempbuf[0] = (char) va_arg(ap, int);
/* Debug level for errors */
-/* XXX: Should be renamed. */
-#define LOG_LEVEL_GPC 0x0001
+#define LOG_LEVEL_REQUEST 0x0001
#define LOG_LEVEL_CONNECT 0x0002
#define LOG_LEVEL_IO 0x0004
#define LOG_LEVEL_HEADER 0x0008
return 0;
}
- /* TODO: Optimize by checking by words insted of octets */
+ /* TODO: Optimize by checking by words instead of octets */
for (i = 0; (i < addr_len) && netmask_addr[i]; i++)
{
if ((network_addr[i] & netmask_addr[i]) !=
*
* Description : Check to see if CONNECT requests to the destination
* port of this request are forbidden. The check is
- * independend of the actual request method.
+ * independent of the actual request method.
*
* Parameters :
* 1 : csp = Current client state (buffers, headers, etc...)
* Function : fuzz_socks
*
* Description : Treat the fuzzed input as a socks response.
- * XXX: This is pretty useless as parsing socks repsonse
+ * XXX: This is pretty useless as parsing socks response
* is trivial.
*
* Parameters :
return;
}
+ assert(slot < SZ(reusable_connection));
assert(NULL != connection->host);
reusable_connection[slot].host = strdup_or_die(connection->host);
reusable_connection[slot].sfd = connection->sfd;
#endif
int connect_failed;
/*
- * XXX: Initializeing it here is only necessary
+ * XXX: Initializing it here is only necessary
* because not all situations are properly
* covered yet.
*/
#endif
jb_socket afd;
#if defined(_WIN32) || defined(__OS2__)
- /* Wierdness - fix a warning. */
+ /* Weirdness - fix a warning. */
int c_length;
#else
socklen_t c_length;
#ifdef FEATURE_HTTPS_INSPECTION
if (client_use_ssl(csp))
{
- ssl_send_data(&(csp->mbedtls_client_attr.ssl),
- (const unsigned char *)response, strlen(response));
+ ssl_send_data_delayed(&(csp->mbedtls_client_attr.ssl),
+ (const unsigned char *)response, strlen(response),
+ get_write_delay(csp));
}
else
#endif
/* Log that the request was crunched and why. */
log_applied_actions(csp->action);
- log_error(LOG_LEVEL_CRUNCH, "%s: %s", crunch_reason(rsp), http->url);
- log_error(LOG_LEVEL_CLF, "%s - - [%T] \"%s\" %s %u",
- csp->ip_addr_str, http->ocmd, status_code, rsp->content_length);
-
+#ifdef FEATURE_HTTPS_INSPECTION
+ if (client_use_ssl(csp))
+ {
+ log_error(LOG_LEVEL_CRUNCH, "%s: https://%s%s", crunch_reason(rsp),
+ http->hostport, http->path);
+ log_error(LOG_LEVEL_CLF, "%s - - [%T] \"%s https://%s%s %s\" %s %llu",
+ csp->ip_addr_str, http->gpc, http->hostport, http->path,
+ http->version, status_code, rsp->content_length);
+ }
+ else
+#endif
+ {
+ log_error(LOG_LEVEL_CRUNCH, "%s: %s", crunch_reason(rsp), http->url);
+ log_error(LOG_LEVEL_CLF, "%s - - [%T] \"%s\" %s %u",
+ csp->ip_addr_str, http->ocmd, status_code, rsp->content_length);
+ }
/* Write the answer to the client */
#ifdef FEATURE_HTTPS_INSPECTION
if (client_use_ssl(csp))
{
- if ((ssl_send_data(&(csp->mbedtls_client_attr.ssl),
- (const unsigned char *)rsp->head, rsp->head_length) < 0)
- || (ssl_send_data(&(csp->mbedtls_client_attr.ssl),
- (const unsigned char *)rsp->body, rsp->content_length) < 0))
+ if ((ssl_send_data_delayed(&(csp->mbedtls_client_attr.ssl),
+ (const unsigned char *)rsp->head, rsp->head_length,
+ get_write_delay(csp)) < 0)
+ || (ssl_send_data_delayed(&(csp->mbedtls_client_attr.ssl),
+ (const unsigned char *)rsp->body, rsp->content_length,
+ get_write_delay(csp)) < 0))
{
/* There is nothing we can do about it. */
log_error(LOG_LEVEL_CONNECT, "Couldn't deliver the error message "
* if +downgrade action applies.
*/
if ((csp->action->flags & ACTION_DOWNGRADE)
- && (!strcmpic(http->ver, "HTTP/1.1")))
+ && (!strcmpic(http->version, "HTTP/1.1")))
{
- freez(http->ver);
- http->ver = strdup_or_die("HTTP/1.0");
+ freez(http->version);
+ http->version = strdup_or_die("HTTP/1.0");
}
/*
string_append(request_line, http->path);
}
string_append(request_line, " ");
- string_append(request_line, http->ver);
+ string_append(request_line, http->version);
if (*request_line == NULL)
{
if (strcmp(fuzz_input_file, "-") != 0)
{
log_error(LOG_LEVEL_FATAL,
- "Fuzzed client requests can currenty only be read from stdin (-).");
+ "Fuzzed client requests can currently only be read from stdin (-).");
}
err = receive_client_request(csp);
if (err != JB_ERR_OK)
#ifdef FEATURE_CONNECTION_KEEP_ALIVE
if ((csp->config->feature_flags & RUNTIME_FEATURE_CONNECTION_KEEP_ALIVE)
- && (!strcmpic(csp->http->ver, "HTTP/1.1"))
+ && (!strcmpic(csp->http->version, "HTTP/1.1"))
&& (csp->http->ssl == 0))
{
/* Assume persistence until further notice */
* Returns : 0 on success, anything else is an error.
*
*********************************************************************/
-static jb_err receive_and_send_encrypted_post_data(struct client_state *csp)
+static int receive_and_send_encrypted_post_data(struct client_state *csp)
{
int content_length_known = csp->expected_client_content_length != 0;
- while (is_ssl_pending(&(csp->mbedtls_client_attr.ssl)))
+ while (is_ssl_pending(&(csp->mbedtls_client_attr.ssl))
+ || (content_length_known && csp->expected_client_content_length != 0))
{
unsigned char buf[BUFFER_SIZE];
int len;
/* XXX: Does this actually happen? */
break;
}
- log_error(LOG_LEVEL_HEADER, "Forwarding %d bytes of encrypted POST data",
+ log_error(LOG_LEVEL_CONNECT, "Forwarding %d bytes of encrypted POST data",
len);
len = ssl_send_data(&(csp->mbedtls_server_attr.ssl), buf, (size_t)len);
if (len == -1)
}
if (csp->expected_client_content_length == 0)
{
- log_error(LOG_LEVEL_HEADER, "Forwarded the last %d bytes", len);
+ log_error(LOG_LEVEL_CONNECT, "Forwarded the last %d bytes", len);
break;
}
}
}
- log_error(LOG_LEVEL_HEADER, "Done forwarding encrypted POST data");
+ log_error(LOG_LEVEL_CONNECT, "Done forwarding encrypted POST data");
return 0;
/* XXX: Also used for JB_ERR_MEMORY */
log_error(LOG_LEVEL_ERROR, "Failed to receive encrypted request: %s",
jb_err_to_string(err));
- ssl_send_data(&(csp->mbedtls_client_attr.ssl),
- (const unsigned char *)CHEADER, strlen(CHEADER));
+ ssl_send_data_delayed(&(csp->mbedtls_client_attr.ssl),
+ (const unsigned char *)CHEADER, strlen(CHEADER), get_write_delay(csp));
return err;
}
if (request_line == NULL)
{
log_error(LOG_LEVEL_ERROR, "Failed to get the encrypted request line");
- ssl_send_data(&(csp->mbedtls_client_attr.ssl),
- (const unsigned char *)CHEADER, strlen(CHEADER));
+ ssl_send_data_delayed(&(csp->mbedtls_client_attr.ssl),
+ (const unsigned char *)CHEADER, strlen(CHEADER), get_write_delay(csp));
return JB_ERR_PARSE;
}
assert(*request_line != '\0');
freez(request_line);
if (JB_ERR_OK != err)
{
- ssl_send_data(&(csp->mbedtls_client_attr.ssl),
- (const unsigned char *)CHEADER, strlen(CHEADER));
+ ssl_send_data_delayed(&(csp->mbedtls_client_attr.ssl),
+ (const unsigned char *)CHEADER, strlen(CHEADER), get_write_delay(csp));
/* XXX: Use correct size */
log_error(LOG_LEVEL_CLF, "%s - - [%T] \"Invalid request\" 400 0", csp->ip_addr_str);
log_error(LOG_LEVEL_ERROR,
*/
log_error(LOG_LEVEL_ERROR,
"Failed to get the encrypted request destination");
- ssl_send_data(&(csp->mbedtls_client_attr.ssl),
- (const unsigned char *)CHEADER, strlen(CHEADER));
+ ssl_send_data_delayed(&(csp->mbedtls_client_attr.ssl),
+ (const unsigned char *)CHEADER, strlen(CHEADER), get_write_delay(csp));
return JB_ERR_PARSE;
}
err = sed_https(csp);
if (JB_ERR_OK != err)
{
- ssl_send_data(&(csp->mbedtls_client_attr.ssl),
- (const unsigned char *)CHEADER, strlen(CHEADER));
+ ssl_send_data_delayed(&(csp->mbedtls_client_attr.ssl),
+ (const unsigned char *)CHEADER, strlen(CHEADER), get_write_delay(csp));
log_error(LOG_LEVEL_ERROR, "Failed to parse client request from %s.",
csp->ip_addr_str);
log_error(LOG_LEVEL_CLF, "%s - - [%T] \"%s\" 400 0",
log_error(LOG_LEVEL_HEADER, "Encrypted request processed");
log_applied_actions(csp->action);
+ log_error(LOG_LEVEL_REQUEST, "https://%s%s", csp->http->hostport,
+ csp->http->path);
return err;
}
+
+/*********************************************************************
+ *
+ * Function : cgi_page_requested
+ *
+ * Description : Checks if a request is for an internal CGI page.
+ *
+ * Parameters :
+ * 1 : host = The host requested by the client.
+ *
+ * Returns : 1 if a CGI page has been requested, 0 otherwise
+ *
+ *********************************************************************/
+static int cgi_page_requested(const char *host)
+{
+ if ((0 == strcmpic(host, CGI_SITE_1_HOST))
+ || (0 == strcmpic(host, CGI_SITE_1_HOST "."))
+ || (0 == strcmpic(host, CGI_SITE_2_HOST))
+ || (0 == strcmpic(host, CGI_SITE_2_HOST ".")))
+ {
+ return 1;
+ }
+
+ return 0;
+
+}
+
#endif
#endif /* def FEATURE_CONNECTION_KEEP_ALIVE */
#ifdef FEATURE_HTTPS_INSPECTION
- /*
- * Reading data from standard or secured connection (HTTP/HTTPS)
- */
if (client_use_ssl(csp))
{
- /*
- * Receiving HTTP request from client over TLS/SSL and sending
- * it to server over TLS/SSL.
- */
- len = ssl_recv_data(&(csp->mbedtls_client_attr.ssl),
- (unsigned char *)csp->receive_buffer, (size_t)max_bytes_to_read);
-
- if (len <= 0)
- {
- mark_server_socket_tainted(csp);
- break;
- }
-
- ret = ssl_send_data(&(csp->mbedtls_server_attr.ssl),
- (const unsigned char *)csp->receive_buffer, (size_t)len);
-
- if (ret < 0)
- {
- log_error(LOG_LEVEL_ERROR,
- "Send request over TLS/SSL to: %s failed", http->host);
- mark_server_socket_tainted(csp);
- close_client_and_server_ssl_connections(csp);
- return;
- }
+ log_error(LOG_LEVEL_CONNECT, "Breaking with TLS/SSL.");
+ break;
}
else
#endif /* def FEATURE_HTTPS_INSPECTION */
*/
if (client_use_ssl(csp))
{
- if ((ssl_send_data(&(csp->mbedtls_client_attr.ssl),
- (const unsigned char *)hdr, strlen(hdr)) < 0)
- || (ssl_send_data(&(csp->mbedtls_client_attr.ssl),
+ if ((ssl_send_data_delayed(&(csp->mbedtls_client_attr.ssl),
+ (const unsigned char *)hdr, strlen(hdr),
+ get_write_delay(csp)) < 0)
+ || (ssl_send_data_delayed(&(csp->mbedtls_client_attr.ssl),
(const unsigned char *) ((p != NULL) ? p : csp->iob->cur),
- csp->content_length) < 0))
+ csp->content_length, get_write_delay(csp)) < 0))
{
log_error(LOG_LEVEL_ERROR, "write modified content to "
"client over TLS/SSL failed");
*/
if (client_use_ssl(csp))
{
- if ((ssl_send_data(&(csp->mbedtls_client_attr.ssl),
- (const unsigned char *)hdr, hdrlen) < 0)
+ if ((ssl_send_data_delayed(&(csp->mbedtls_client_attr.ssl),
+ (const unsigned char *)hdr, hdrlen, get_write_delay(csp)) < 0)
|| ((flushed = ssl_flush_socket(&(csp->mbedtls_client_attr.ssl),
csp->iob)) < 0)
- || (ssl_send_data(&(csp->mbedtls_client_attr.ssl),
- (const unsigned char *)csp->receive_buffer, (size_t)len) < 0))
+ || (ssl_send_data_delayed(&(csp->mbedtls_client_attr.ssl),
+ (const unsigned char *)csp->receive_buffer, (size_t)len,
+ get_write_delay(csp)) < 0))
{
log_error(LOG_LEVEL_CONNECT,
"Flush header and buffers to client failed");
*/
if (client_use_ssl(csp))
{
- ret = ssl_send_data(&(csp->mbedtls_client_attr.ssl),
- (const unsigned char *)csp->receive_buffer, (size_t)len);
+ ret = ssl_send_data_delayed(&(csp->mbedtls_client_attr.ssl),
+ (const unsigned char *)csp->receive_buffer, (size_t)len,
+ get_write_delay(csp));
if (ret < 0)
{
log_error(LOG_LEVEL_ERROR,
*/
if (client_use_ssl(csp))
{
- ssl_send_data(&(csp->mbedtls_client_attr.ssl),
+ ssl_send_data_delayed(&(csp->mbedtls_client_attr.ssl),
(const unsigned char *)INVALID_SERVER_HEADERS_RESPONSE,
- strlen(INVALID_SERVER_HEADERS_RESPONSE));
+ strlen(INVALID_SERVER_HEADERS_RESPONSE), get_write_delay(csp));
}
else
#endif /* def FEATURE_HTTPS_INSPECTION */
*/
if (client_use_ssl(csp))
{
- ssl_send_data(&(csp->mbedtls_client_attr.ssl),
+ ssl_send_data_delayed(&(csp->mbedtls_client_attr.ssl),
(const unsigned char *)INVALID_SERVER_HEADERS_RESPONSE,
- strlen(INVALID_SERVER_HEADERS_RESPONSE));
+ strlen(INVALID_SERVER_HEADERS_RESPONSE),
+ get_write_delay(csp));
}
else
#endif /* def FEATURE_HTTPS_INSPECTION */
*/
if (client_use_ssl(csp))
{
- ssl_send_data(&(csp->mbedtls_client_attr.ssl),
+ ssl_send_data_delayed(&(csp->mbedtls_client_attr.ssl),
(const unsigned char *)INVALID_SERVER_HEADERS_RESPONSE,
- strlen(INVALID_SERVER_HEADERS_RESPONSE));
+ strlen(INVALID_SERVER_HEADERS_RESPONSE),
+ get_write_delay(csp));
}
else
#endif
#ifdef FEATURE_HTTPS_INSPECTION
if (client_use_ssl(csp))
{
- if ((ssl_send_data(&(csp->mbedtls_client_attr.ssl),
- (const unsigned char *)hdr, strlen(hdr)) < 0)
+ if ((ssl_send_data_delayed(&(csp->mbedtls_client_attr.ssl),
+ (const unsigned char *)hdr, strlen(hdr),
+ get_write_delay(csp)) < 0)
|| (len = ssl_flush_socket(&(csp->mbedtls_client_attr.ssl),
csp->iob) < 0))
{
*/
if (client_use_ssl(csp))
{
- ssl_send_data(&(csp->mbedtls_client_attr.ssl),
+ ssl_send_data_delayed(&(csp->mbedtls_client_attr.ssl),
(const unsigned char *)INVALID_SERVER_HEADERS_RESPONSE,
- strlen(INVALID_SERVER_HEADERS_RESPONSE));
+ strlen(INVALID_SERVER_HEADERS_RESPONSE),
+ get_write_delay(csp));
}
else
#endif /* def FEATURE_HTTPS_INSPECTION */
}
#endif
- log_error(LOG_LEVEL_CLF, "%s - - [%T] \"%s\" 200 %llu",
- csp->ip_addr_str, http->ocmd, csp->content_length);
-
+#ifdef FEATURE_HTTPS_INSPECTION
+ if (client_use_ssl(csp))
+ {
+ log_error(LOG_LEVEL_CLF, "%s - - [%T] \"%s https://%s%s %s\" 200 %llu",
+ csp->ip_addr_str, http->gpc, http->hostport, http->path,
+ http->version, csp->content_length);
+ }
+ else
+#endif
+ {
+ log_error(LOG_LEVEL_CLF, "%s - - [%T] \"%s\" 200 %llu",
+ csp->ip_addr_str, http->ocmd, csp->content_length);
+ }
csp->server_connection.timestamp = time(NULL);
}
{
return;
}
+#ifdef FEATURE_HTTPS_INSPECTION
+ /*
+ * Log the request unless we're https inspecting
+ * in which case we don't have the path yet and
+ * will log the request later.
+ */
+ if (!client_use_ssl(csp))
+#endif
+ {
+ log_error(LOG_LEVEL_REQUEST, "%s%s", http->hostport, http->path);
+ }
/* decide how to route the HTTP request */
fwd = forward_url(csp, http);
* Setting flags to use old solution with SSL tunnel and to disable
* certificates verification.
*/
- if (csp->http->ssl && !(csp->action->flags & ACTION_HTTPS_INSPECTION))
+ if (csp->http->ssl && !(csp->action->flags & ACTION_HTTPS_INSPECTION)
+ && !cgi_page_requested(csp->http->host))
{
use_ssl_tunnel = 1;
}
}
log_applied_actions(csp->action);
- log_error(LOG_LEVEL_GPC, "%s%s", http->hostport, http->path);
-
if (fwd->forward_host)
{
log_error(LOG_LEVEL_CONNECT, "via [%s]:%d to: %s",
int ret;
/*
* Creating an SSL proxy. If forwarding is disabled, we must send
- * CSUCCEED mesage to client. Then TLS/SSL connection with client
+ * CSUCCEED message to client. Then TLS/SSL connection with client
* is created.
*/
* Description : Check that a string list is valid. The intended
* usage is "assert(list_is_valid(the_list))".
* Currently this checks that "the_list->last"
- * is correct, and that the list dosn't contain
+ * is correct, and that the list doesn't contain
* circular references. It is likely to crash if
* it's passed complete garbage.
*
* 2 : name = name parameter to look for
*
* Returns : the value if found, else the empty string.
- * Return value is alloced as part of the map, so
+ * Return value is allocated as part of the map, so
* it is freed when the map is destroyed. Caller
* must not free or modify it.
*
/*
* This takes the "cryptic" hash of each keyword and aliases them to
* something a little more readable. This also makes changing the
- * hash values easier if they should change or the hash algorthm changes.
+ * hash values easier if they should change or the hash algorithm changes.
* Use the included "hash" program to find out what the hash will be
* for any string supplied on the command line. (Or just put it in the
* config file and read the number from the error message in the log).
/*
* If this is the first filter block, chain it
- * to the file_list rather than its (nonexistant)
+ * to the file_list rather than its (nonexistent)
* predecessor
*/
if (fs->f == NULL)
#############################################################################
{ \
+change-x-forwarded-for{block} \
++client-header-filter{no-brotli-accepted} \
+client-header-tagger{css-requests} \
+client-header-tagger{image-requests} \
+client-header-tagger{range-requests} \
struct list headers;
/*
- * Temporarly replace csp->headers with csp->https_headers
+ * Temporarily replace csp->headers with csp->https_headers
* to trick sed() into filtering the https headers.
*/
headers.first = csp->headers->first;
csp->headers->last = csp->https_headers->last;
/*
- * Start with fresh tags. Already exising tags may
+ * Start with fresh tags. Already existing tags may
* be set again. This is necessary to overrule
* URL-based patterns.
*/
/*
* We want client header filters and taggers
- * so temporarly remove the flag.
+ * so temporarily remove the flag.
*/
csp->flags &= ~CSP_FLAG_CLIENT_HEADER_PARSING_DONE;
err = sed(csp, FILTER_CLIENT_HEADERS);
csp->flags |= CSP_FLAG_CLIENT_HEADER_PARSING_DONE;
+ /*
+ * Update the last header which may have changed
+ * due to header additions,
+ */
+ csp->https_headers->last = csp->headers->last;
+
csp->headers->first = headers.first;
csp->headers->last = headers.last;
if ((csp->config->feature_flags & RUNTIME_FEATURE_CONNECTION_SHARING)
&& !(csp->flags & CSP_FLAG_SERVER_SOCKET_TAINTED))
{
- if (!strcmpic(csp->http->ver, "HTTP/1.1"))
+ if (!strcmpic(csp->http->version, "HTTP/1.1"))
{
log_error(LOG_LEVEL_HEADER,
"Removing \'%s\' to imply keep-alive.", *header);
*
* Description : Remove the Content-Encoding header if the
* decompression was successful and the content
- * has been modifed.
+ * has been modified.
*
* Parameters :
* 1 : csp = Current client state (buffers, headers, etc...)
#ifdef FEATURE_CONNECTION_KEEP_ALIVE
if ((csp->config->feature_flags & RUNTIME_FEATURE_CONNECTION_KEEP_ALIVE)
&& !(csp->flags & CSP_FLAG_SERVER_SOCKET_TAINTED)
- && !strcmpic(csp->http->ver, "HTTP/1.1"))
+ && !strcmpic(csp->http->version, "HTTP/1.1"))
{
csp->flags |= CSP_FLAG_CLIENT_CONNECTION_KEEP_ALIVE;
return JB_ERR_OK;
string_append(&http->cmd, " ");
string_append(&http->cmd, http->url);
string_append(&http->cmd, " ");
- string_append(&http->cmd, http->ver);
+ string_append(&http->cmd, http->version);
if (http->cmd == NULL)
{
return JB_ERR_MEMORY;
string_append(&http->cmd, " ");
string_append(&http->cmd, http->url);
string_append(&http->cmd, " ");
- string_append(&http->cmd, http->ver);
+ string_append(&http->cmd, http->version);
if (http->cmd == NULL)
{
return JB_ERR_MEMORY;
/**
* This macro is used to free a pointer that may be NULL.
* It also sets the variable to NULL after it's been freed.
- * The paramater should be a simple variable without side effects.
+ * The parameter should be a simple variable without side effects.
*/
#define freez(X) { if(X) { free((void*)X); X = NULL ; } }
/**
* A map from a string to another string.
- * This is used for the paramaters passed in a HTTP GET request, and
+ * This is used for the parameters passed in a HTTP GET request, and
* to store the exports when the CGI interface is filling in a template.
*/
struct map
char *ocmd; /**< Backup of original cmd for CLF logging */
char *gpc; /**< HTTP method: GET, POST, ... */
char *url; /**< The URL */
- char *ver; /**< Protocol version */
+ char *version; /**< Protocol version */
int status; /**< HTTP Status */
char *host; /**< Host part of URL */
#endif /* ndef FEATURE_EXTENDED_HOST_PATTERNS */
#ifdef FEATURE_HTTPS_INSPECTION
- int client_ssl; /**< Flag if we should comunicate with slient over ssl */
- int server_ssl; /**< Flag if we should comunicate with server over ssl */
+ int client_ssl; /**< Flag if we should communicate with client over ssl */
+ int server_ssl; /**< Flag if we should communicate with server over ssl */
unsigned char hash_of_host_hex[(HASH_OF_HOST_BUF_SIZE * 2) + 1]; /**< chars for hash in hex string and one for '\0' */
unsigned char hash_of_host[HASH_OF_HOST_BUF_SIZE+1]; /**< chars for bytes of hash and one for '\0' */
#endif
* Struct for linked list containing certificates
*/
typedef struct certs_chain {
- char text_buf[CERT_INFO_BUF_SIZE]; /* text info about properties of certificate */
+ char info_buf[CERT_INFO_BUF_SIZE]; /* text info about properties of certificate */
char file_buf[CERT_FILE_BUF_SIZE]; /* buffer for whole certificate - format to save in file */
struct certs_chain *next; /* next certificate in chain of trust */
} certs_chain_t;
unsigned long flags;
/**
- * Paramaters for those actions that require them.
+ * Parameters for those actions that require them.
* Each entry is valid if & only if the corresponding entry in "flags" is
* set.
*/
#define CSP_FLAG_UNSUPPORTED_CLIENT_EXPECTATION 0x02000000U
/**
- * Flag for csp->flags: Set if we answered the request ourselve.
+ * Flag for csp->flags: Set if we answered the request ourselves.
*/
#define CSP_FLAG_CRUNCHED 0x04000000U
/*
* Server certificate chain of trust including strings with certificates
- * informations and string with whole certificate file
+ * information and string with whole certificate file
*/
struct certs_chain server_certs_chain;
#endif
* INCLUDES the trailing slash.
*/
#define CGI_PREFIX "http://" CGI_SITE_2_HOST CGI_SITE_2_PATH "/"
+#define CGI_PREFIX_HTTPS "https://" CGI_SITE_2_HOST CGI_SITE_2_PATH "/"
#endif /* ndef PROJECT_H_INCLUDED */
#
#############################################################################
#
-# Copyright (c) 2007-2018 Fabian Keil <fk@fabiankeil.de>
+# Copyright (c) 2007-2020 Fabian Keil <fk@fabiankeil.de>
#
# Permission to use, copy, modify, and distribute this software for any
# purpose with or without fee is hereby granted, provided that the above
# Redirected URL = http://redirect2.example.net/blafasel?x=http://redirected.example.net/¶meter1=part-of-the-redirected-url1¶meter2=part-of-the-redirected-url1
# Redirect Destination = http://redirected.example.net/¶meter1=part-of-the-redirected-url1¶meter2=part-of-the-redirected-url1
redirect2.example.net/
+
+{+delay-response{1}}
+# Fetch Test = http://config.privoxy.org/show-url-info?url=http%3A%2F%2Fwww.example.com%2Fprivoxy-test-delay-response
+config.privoxy.org/show-url-info\?url=http%3A%2F%2Fwww\.example\.com%2Fprivoxy-test-delay-response
#include "errlog.h"
#include "jcc.h"
#include "ssl.h"
+#include "encode.h"
/*
*
* Function : is_ssl_pending
*
- * Description : Tests if there are some waiting data on ssl connection
+ * Description : Tests if there are some waiting data on ssl connection.
+ * Only considers data that has actually been received
+ * locally and ignores data that is still on the fly
+ * or has not yet been sent by the remote end.
*
* Parameters :
* 1 : ssl = SSL context to test
}
+/*********************************************************************
+ *
+ * Function : ssl_send_data_delayed
+ *
+ * Description : Sends the contents of buf (for n bytes) to given SSL
+ * connection, optionally delaying the operation.
+ *
+ * Parameters :
+ * 1 : ssl = SSL context to send data to
+ * 2 : buf = Pointer to data to be sent
+ * 3 : len = Length of data to be sent to the SSL context
+ * 4 : delay = Delay in milliseconds.
+ *
+ * Returns : 0 on success (entire buffer sent).
+ * nonzero on error.
+ *
+ *********************************************************************/
+extern int ssl_send_data_delayed(mbedtls_ssl_context *ssl,
+ const unsigned char *buf, size_t len,
+ unsigned int delay)
+{
+ size_t i = 0;
+
+ if (delay == 0)
+ {
+ if (ssl_send_data(ssl, buf, len) < 0)
+ {
+ return -1;
+ }
+ else
+ {
+ return 0;
+ }
+ }
+
+ while (i < len)
+ {
+ size_t write_length;
+ enum { MAX_WRITE_LENGTH = 10 };
+
+ if ((i + MAX_WRITE_LENGTH) > len)
+ {
+ write_length = len - i;
+ }
+ else
+ {
+ write_length = MAX_WRITE_LENGTH;
+ }
+
+ privoxy_millisleep(delay);
+
+ if (ssl_send_data(ssl, buf + i, write_length) < 0)
+ {
+ return -1;
+ }
+ i += write_length;
+ }
+
+ return 0;
+
+}
+
+
/*********************************************************************
*
* Function : ssl_recv_data
{
/*
* We can't use function mbedtls_net_free, because this function
- * inter alia close TCP connection on setted fd. Instead of this
+ * inter alia close TCP connection on set fd. Instead of this
* function, we change fd to -1, which is the same what does
* rest of mbedtls_net_free function.
*/
{
log_error(LOG_LEVEL_ERROR,
"mbedtls_ssl_handshake with server failed: %s", err_buf);
+ free_certificate_chain(csp);
ret = -1;
}
goto exit;
{
/*
* We can't use function mbedtls_net_free, because this function
- * inter alia close TCP connection on setted fd. Instead of this
+ * inter alia close TCP connection on set fd. Instead of this
* function, we change fd to -1, which is the same what does
* rest of mbedtls_net_free function.
*/
int ret = 0;
struct certs_chain *cert = NULL;
- /* Header of message with certificate informations */
+ /* Header of message with certificate information */
const char message_begin[] =
"HTTP/1.1 200 OK\r\n"
"Content-Type: text/html\r\n"
{
size_t base64_len = 4 * ((strlen(cert->file_buf) + 2) / 3) + 1;
- message_len += strlen(cert->
text_buf) + strlen("<pre></pre>\n")
+ message_len += strlen(cert->
info_buf) + strlen("<pre></pre>\n")
+ base64_len + strlen("<a href=\"data:application"
"/x-x509-ca-cert;base64,\">Download certificate</a>");
cert = cert->next;
}
strlcat(message, "<pre>", message_len);
- strlcat(message, cert->text_buf, message_len);
+ strlcat(message, cert->info_buf, message_len);
strlcat(message, "</pre>\n", message_len);
if (ret == 0)
* Function : ssl_verify_callback
*
* Description : This is a callback function for certificate verification.
- * It's called for all certificates in server certificate
- * trusted chain and it's preparing information about this
- * certificates. Prepared informations can be used to inform
- * user about invalid certificates.
+ * It's called once for each certificate in the server's
+ * certificate trusted chain and prepares information about
+ * the certificate. The information can be used to inform
+ * the user about invalid certificates.
*
* Parameters :
* 1 : csp_void = Current client state (buffers, headers, etc...)
*/
last->next = malloc_or_die(sizeof(struct certs_chain));
last->next->next = NULL;
- memset(last->next->text_buf, 0, sizeof(last->next->text_buf));
+ memset(last->next->info_buf, 0, sizeof(last->next->info_buf));
memset(last->next->file_buf, 0, sizeof(last->next->file_buf));
/*
crt->raw.p, crt->raw.len, (unsigned char *)last->file_buf,
sizeof(last->file_buf)-1, &olen)) != 0)
{
+ char err_buf[ERROR_BUF_SIZE];
+
+ mbedtls_strerror(ret, err_buf, sizeof(err_buf));
+ log_error(LOG_LEVEL_ERROR, "mbedtls_pem_write_buffer() failed: %s",
+ err_buf);
+
return(ret);
}
/*
* Saving certificate information into buffer
*/
- mbedtls_x509_crt_info(last->text_buf, sizeof(last->text_buf) - 1,
- CERT_INFO_PREFIX, crt);
+ {
+ char buf[CERT_INFO_BUF_SIZE];
+ char *encoded_text;
+
+ mbedtls_x509_crt_info(buf, sizeof(buf), CERT_INFO_PREFIX, crt);
+ encoded_text = html_encode(buf);
+ strlcpy(last->info_buf, encoded_text, sizeof(last->info_buf));
+ freez(encoded_text);
+ }
return 0;
}
* Function : free_certificate_chain
*
* Description : Frees certificates linked list. This linked list is
- * used to save informations about certificates in
+ * used to save information about certificates in
* trusted chain.
*
* Parameters :
struct certs_chain *cert = csp->server_certs_chain.next;
/* Cleaning buffers */
- memset(csp->server_certs_chain.text_buf, 0,
- sizeof(csp->server_certs_chain.text_buf));
+ memset(csp->server_certs_chain.info_buf, 0,
+ sizeof(csp->server_certs_chain.info_buf));
memset(csp->server_certs_chain.file_buf, 0,
sizeof(csp->server_certs_chain.file_buf));
csp->server_certs_chain.next = NULL;
/* Freeing memory in whole linked list */
- if (cert != NULL)
+ while (cert != NULL)
{
- do
- {
- struct certs_chain *cert_for_free = cert;
- cert = cert->next;
- freez(cert_for_free);
- } while (cert != NULL);
+ struct certs_chain *cert_for_free = cert;
+ cert = cert->next;
+ freez(cert_for_free);
}
}
* Function : tunnel_established_successfully
*
* Description : Check if parent proxy server response contains
- * informations about successfully created connection with
+ * information about successfully created connection with
* destination server. (HTTP/... 2xx ...)
*
* Parameters :
/* Functions for sending and receiving data over TLS/SSL connections */
extern int ssl_send_data(mbedtls_ssl_context *ssl, const unsigned char *buf, size_t len);
+extern int ssl_send_data_delayed(mbedtls_ssl_context *ssl, const unsigned char *buf,
+ size_t len, unsigned int delay);
extern int ssl_recv_data(mbedtls_ssl_context *ssl, unsigned char *buf, size_t maxLen);
extern long ssl_flush_socket(mbedtls_ssl_context *ssl, struct iob *iob);
extern void ssl_send_certificate_error(struct client_state *csp);
}
/* Any character but `%' must be matched by the same character
- in the iput string. */
+ in the input string. */
if (*fmt != '%')
{
match_char (*fmt++, *rp++);
# particular section in an actions file.
#
#
-# Copyright : Written by and Copyright (C) 2001-2014 the
+# Copyright : Written by and Copyright (C) 2001-2020 the
# Privoxy team. https://www.privoxy.org/
#
# Original Author: Copyright (C) 2001 Jonathan Foster
<input type="text" name="hide_user_agent_mode" size="40"
value="@hide-user-agent-param@"></td>
</tr>
+
+<!-- @if-https-inspection-start -->
+ <tr class="bg1" align="left" valign="top">
+ <td class="en1" align="center" valign="middle"><input type="radio"
+ name="https_inspection" value="Y" @https-inspection-y@
+ ></td>
+ <td class="dis1" align="center" valign="middle"><input type="radio"
+ name="https_inspection" value="N" @https-inspection-n@
+ ></td>
+ <td class="noc1" align="center" valign="middle"><input type="radio"
+ name="https_inspection" value="X" @https-inspection-x@
+ ></td>
+ <td class="action"><a href="@user-manual@@actions-help-prefix@HTTPS-INSPECTION">https-inspection</a></td>
+ <td>
+ Enable content and header filtering for https requests.
+ </td>
+ </tr>
+
+ <tr class="bg1" align="left" valign="top">
+ <td class="en1" align="center" valign="middle"><input type="radio"
+ name="ignore_certificate_errors" value="Y" @ignore-certificate-errors-y@
+ ></td>
+ <td class="dis1" align="center" valign="middle"><input type="radio"
+ name="ignore_certificate_errors" value="N" @ignore-certificate-errors-n@
+ ></td>
+ <td class="noc1" align="center" valign="middle"><input type="radio"
+ name="ignore_certificate_errors" value="X" @ignore-certificate-errors-x@
+ ></td>
+ <td class="action"><a href="@user-manual@@actions-help-prefix@IGNORE-CERTIFICATE-ERRORS">ignore-certificate-errors</a></td>
+ <td>
+ Don't verify certificates when forwarding encrypted requests.
+ </td>
+ </tr>
+
+<!-- if-https-inspection-end@ -->
+
<tr class="bg1" align="left" valign="top">
<td class="en1" align="center" valign="middle"><input type="radio"
name="limit_connect" id="limit_connect_y" value="Y" @limit-connect-y@
</tr>
<tr>
<td class="box">
-<!-- @if-https-start -->
+<!-- @if-https-and-no-https-inspection-start -->
<h2>NOTE:</h2>
<p>This is a HTTPS URL, so the part after the "/" is ignored
as Privoxy doesn't see the path for real HTTPS requests either.</p>
-<!-- if-https-end@ -->
+<!-- if-https-and-no-https-inspection-end@ -->
<!-- if-valid-url-end@ -->
<h2>Matches for <a href="@url@">@url@</a>:</h2>
@matches@
# hash key as input.
# - Add --compress and --decompress options.
#
-# Copyright (c) 2007-2017 Fabian Keil <fk@fabiankeil.de>
+# Copyright (c) 2007-2020 Fabian Keil <fk@fabiankeil.de>
#
# Permission to use, copy, modify, and distribute this software for any
# purpose with or without fee is hereby granted, provided that the above
use Getopt::Long;
use constant {
- PRIVOXY_LOG_PARSER_VERSION => '0.9',
+ PRIVOXY_LOG_PARSER_VERSION => '0.9.1',
# Feel free to mess with these ...
DEFAULT_BACKGROUND => 'black', # Choose registered colour (like 'black')
DEFAULT_TEXT_COLOUR => 'white', # Choose registered colour (like 'black')
my $shorten_thread_ids;
my $line_end;
-sub prepare_our_stuff () {
+sub prepare_our_stuff() {
# Syntax Higlight hash
@all_colours = (
init_stats();
}
-sub paint_it ($) {
+sub paint_it($) {
###############################################################
# Takes a colour string and returns an ANSI escape sequence
# (unless --no-syntax-highlighting is used).
return $colour_code;
}
-sub get_semantic_html_markup ($) {
+sub get_semantic_html_markup($) {
###############################################################
# Takes a string and returns a span element
###############################################################
return $code;
}
-sub cli_option_is_set ($) {
+sub cli_option_is_set($) {
our %cli_options;
my $cli_option = shift;
return $cli_options{$cli_option};
}
-sub get_html_title () {
+sub get_html_title() {
our %cli_options;
return $cli_options{'title'};
);
}
-sub get_css_colour ($) {
+sub get_css_colour($) {
our %css_colours;
my $colour = shift;
return '#' . $css_colours{$colour};
}
-sub get_css_line ($) {
+sub get_css_line($) {
my $class = shift;
my $css_line;
return $css_line;
}
-sub get_css_line_for_colour ($) {
+sub get_css_line_for_colour($) {
my $colour = shift;
my $css_line;
}
# XXX: Wrong solution
-sub get_missing_css_lines () {
+sub get_missing_css_lines() {
my $css_line;
return $css_line;
}
-sub get_css () {
+sub get_css() {
our %css_colours; #XXX: Wrong solution
return $css;
}
-sub print_intro () {
+sub print_intro() {
my $intro = '';
}
}
-sub print_outro () {
+sub print_outro() {
my $outro = '';
}
}
-sub get_line_end () {
+sub get_line_end() {
return cli_option_is_set('html-output') ? "<br>\n" : "\n";
}
-sub get_colour_html_markup ($) {
+sub get_colour_html_markup($) {
###############################################################
# Takes a colour string a span element. XXX: WHAT?
# XXX: This function shouldn't be necessary, the
return $code;
}
-sub default_colours () {
+sub default_colours() {
# XXX: Properly
our $bg_code;
return reset_colours();
}
-sub show_colours () {
+sub show_colours() {
# XXX: Implement
}
-sub reset_colours () {
+sub reset_colours() {
return ESCAPE . "0m";
}
-sub set_background ($){
+sub set_background($) {
my $colour = shift;
our $bg_code;
}
}
-sub get_background (){
+sub get_background() {
return our $bg_code;
}
-sub prepare_highlight_hash ($) {
+sub prepare_highlight_hash($) {
my $ref = shift;
foreach my $key (keys %$ref) {
}
}
-sub prepare_colour_array ($) {
+sub prepare_colour_array($) {
my $ref = shift;
foreach my $i (0 ... @$ref - 1) {
}
}
-sub found_unknown_content ($) {
+sub found_unknown_content($) {
my $unknown = shift;
my $message;
die "Unworthy content parser" if PUNISH_MISSING_LOG_KNOWLEDGE_WITH_DEATH;
}
-sub log_parse_error ($) {
+sub log_parse_error($) {
my $message = shift;
}
}
-sub debug_message (@) {
+sub debug_message(@) {
my @message = @_;
print $h{'debug'} . "@message" . $h{'Standard'} . "\n";
# highlighter functions that aren't loglevel-specific
################################################################################
-sub h ($) {
+sub h($) {
# Get highlight marker
my $highlight = shift; # XXX: Stupid name;
return $result;
}
-sub highlight_known_headers ($) {
+sub highlight_known_headers($) {
my $content = shift;
return $content;
}
-sub highlight_matched_request_line ($$) {
+sub highlight_matched_request_line($$) {
my $result = shift; # XXX: Stupid name;
my $regex = shift;
return $result;
}
-sub highlight_request_line ($) {
+sub highlight_request_line($) {
my $rl = shift;
my ($method, $url, $http_version);
return $rl;
}
-sub highlight_response_line ($) {
+sub highlight_response_line($) {
my $rl = shift;
my ($http_version, $status_code, $status_message);
return $rl;
}
-sub highlight_matched_url ($$) {
+sub highlight_matched_url($$) {
my $result = shift; # XXX: Stupid name;
my $regex = shift;
return $result;
}
-sub highlight_matched_host ($$) {
+sub highlight_matched_host($$) {
my ($result, $regex) = @_; # XXX: result ist stupid name;
return $result;
}
-sub highlight_matched_pattern ($$$) {
+sub highlight_matched_pattern($$$) {
my $result = shift; # XXX: Stupid name;
my $key = shift;
return $result;
}
-sub highlight_matched_path ($$) {
+sub highlight_matched_path($$) {
my $result = shift; # XXX: Stupid name;
my $regex = shift;
return $result;
}
-sub highlight_url ($) {
+sub highlight_url($) {
my $url = shift;
return $url;
}
-sub update_header_highlight_regex ($) {
+sub update_header_highlight_regex($) {
my $header = shift;
my $headers = join ('|', keys %header_colours);
# loglevel-specific highlighter functions
################################################################################
-sub handle_loglevel_header ($) {
+sub handle_loglevel_header($) {
my $c = shift;
return $c;
}
-sub handle_loglevel_re_filter ($) {
+sub handle_loglevel_re_filter($) {
my $content = shift;
my $c = $content;
return $content;
}
-sub handle_loglevel_redirect ($) {
+sub handle_loglevel_redirect($) {
my $c = shift;
return $c;
}
-sub handle_loglevel_gif_deanimate ($) {
+sub handle_loglevel_gif_deanimate($) {
my $content = shift;
return $content;
}
-sub handle_loglevel_request ($) {
+sub handle_loglevel_request($) {
my $content = shift;
return $content;
}
-sub handle_loglevel_crunch ($) {
+sub handle_loglevel_crunch($) {
my $content = shift;
return $content;
}
-sub handle_loglevel_connect ($) {
+sub handle_loglevel_connect($) {
my $c = shift;
$c =~ s@(?<=Shifting )(\d+)@$h{'Number'}$1$h{'Standard'}@;
$c =~ s@(?<=by )(\d+)@$h{'Number'}$1$h{'Standard'}@;
+ } elsif ($c =~ m/^Flushed (\d+) bytes of request body while expecting (\d+)/) {
+
+ # Flushed 30 bytes of request body while expecting 30
+ $c =~ s@(?<=Flushed )(\d+)@$h{'Number'}$1$h{'Standard'}@;
+ $c =~ s@(?<=expecting )(\d+)@$h{'Number'}$1$h{'Standard'}@;
+
+ } elsif ($c =~ m/^Performing the TLS\/SSL handshake with client. Hash of host:/) {
+
+ # Performing the TLS/SSL handshake with client. Hash of host: bab5296b25e256c7b06b92b17b56bcae
+ $c = highlight_matched_host($c, '(?<=Hash of host: ).+');
+
} elsif ($c =~ m/^Looks like we / or
$c =~ m/^Unsetting keep-alive flag/ or
$c =~ m/^No connections to wait/ or
}
-sub handle_loglevel_info ($) {
+sub handle_loglevel_info($) {
my $c = shift;
return $c;
}
-sub handle_loglevel_cgi ($) {
+sub handle_loglevel_cgi($) {
my $c = shift;
return $c;
}
-sub handle_loglevel_force ($) {
+sub handle_loglevel_force($) {
my $c = shift;
return $c;
}
-sub handle_loglevel_error ($) {
+sub handle_loglevel_error($) {
my $c = shift;
}
-sub handle_loglevel_ignore ($) {
+sub handle_loglevel_ignore($) {
return shift;
}
-sub gather_loglevel_request_stats ($$) {
+sub gather_loglevel_clf_stats($) {
+
+ my $content = shift;
+ my ($method, $resource, $http_version, $status_code, $size);
+ our %stats;
+ our %cli_options;
+
+ # +0200] "GET https://www.youtube.com/watch?v=JmcA9LIIXWw HTTP/1.1" 200 68004
+ $content =~ m/^[+-]\d{4}\] "(\w+) (.+) (HTTP\/\d\.\d)" (\d+) (\d+)/;
+ $method = $1;
+ $resource = $2;
+ $http_version = $3;
+ $status_code = $4;
+ $size = $5;
+
+ unless (defined $method) {
+ print("Failed to parse: $content\n");
+ return;
+ }
+ $stats{'method'}{$method}++;
+ if ($cli_options{'url-statistics-threshold'} != 0) {
+ $stats{'resource'}{$resource}++;
+ }
+ $stats{'http-version'}{$http_version}++;
+
+ if ($cli_options{'host-statistics-threshold'} != 0) {
+ $resource =~ m@(?:http[s]://)([^/]+)/?@;
+ $stats{'hosts'}{$1}++;
+ }
+ $stats{'content-size-total'} += $size;
+ $stats{'status-code'}{$status_code}++;
+ $stats{requests_clf}++;
+}
+
+sub gather_loglevel_request_stats($$) {
my $c = shift;
my $thread = shift;
our %stats;
$stats{requests}++;
}
-sub gather_loglevel_crunch_stats ($$) {
+sub gather_loglevel_crunch_stats($$) {
my $c = shift;
my $thread = shift;
our %stats;
- $stats{requests}++;
$stats{crunches}++;
if ($c =~ m/^Redirected:/) {
}
-sub gather_loglevel_error_stats ($$) {
+sub gather_loglevel_error_stats($$) {
my $c = shift;
my $thread = shift;
}
}
-sub gather_loglevel_connect_stats ($$) {
+sub gather_loglevel_connect_stats($$) {
my ($c, $thread) = @_;
our %thread_data;
}
}
-sub gather_loglevel_header_stats ($$) {
+sub gather_loglevel_header_stats($$) {
my ($c, $thread) = @_;
our %stats;
# A HTTP/1.1 response without Connection header implies keep-alive.
# Keeping the server header 'Connection: keep-alive' around.
$stats{'server-keep-alive'}++;
-
- } elsif ($c =~ m/^scan: ((\w+) (.+) (HTTP\/\d\.\d))/) {
-
- # scan: HTTP/1.1 200 OK
- $stats{'method'}{$2}++;
- if ($cli_options{'url-statistics-threshold'} != 0) {
- $stats{'resource'}{$3}++;
- }
- $stats{'http-version'}{$4}++;
-
- } elsif ($cli_options{'host-statistics-threshold'} != 0 and
- $c =~ m/^scan: Host: ([^\s]+)/) {
-
- # scan: Host: p.p
- $stats{'hosts'}{$1}++;
}
}
-sub init_stats () {
+sub init_stats() {
our %stats = (
requests => 0,
+ requests_clf => 0,
crunches => 0,
'server-keep-alive' => 0,
'reused-connections' => 0,
'reused-connections' => 0,
'server-keep-alive' => 0,
'closed-client-connections' => 0,
+ 'content-size-total' => 0,
);
$stats{'client-requests-on-connection'}{1} = 0;
}
-sub get_percentage ($$) {
+sub get_percentage($$) {
my $big = shift;
my $small = shift;
return sprintf("%.2f%%", $small / $big * 100);
}
-sub print_stats () {
+sub print_stats() {
our %stats;
our %cli_options;
my $new_connections = $stats{requests} - $stats{crunches} - $stats{'reused-connections'};
- my $outgoing_requests = $stats{requests} - $stats{crunches};
my $client_requests_checksum = 0;
+ if ($stats{requests_clf} && $stats{requests}
+ && $stats{requests_clf} != $stats{requests}) {
+ print "Inconsistent request counts: " . $stats{requests} . "/" . $stats{requests_clf} . "\n";
+ }
+ if ($stats{requests_clf} && $stats{requests} eq 0) {
+ $stats{requests} = $stats{requests_clf};
+ }
+
if ($stats{requests} eq 0) {
print "No requests yet.\n";
return;
}
print "Client requests total: " . $stats{requests} . "\n";
- print "Crunches: " . $stats{crunches} . " (" .
- get_percentage($stats{requests}, $stats{crunches}) . ")\n";
- print "Blocks: " . $stats{'blocked'} . " (" .
- get_percentage($stats{requests}, $stats{'blocked'}) . ")\n";
- print "Fast redirections: " . $stats{'fast-redirections'} . " (" .
- get_percentage($stats{requests}, $stats{'fast-redirections'}) . ")\n";
- print "Connection timeouts: " . $stats{'connection-timeout'} . " (" .
- get_percentage($stats{requests}, $stats{'connection-timeout'}) . ")\n";
- print "Connection failures: " . $stats{'connection-failure'} . " (" .
- get_percentage($stats{requests}, $stats{'connection-failure'}) . ")\n";
- print "Outgoing requests: " . $outgoing_requests . " (" .
- get_percentage($stats{requests}, $outgoing_requests) . ")\n";
+ if ($stats{crunches}) {
+ my $outgoing_requests = $stats{requests} - $stats{crunches};
+ print "Crunches: " . $stats{crunches} . " (" .
+ get_percentage($stats{requests}, $stats{crunches}) . ")\n";
+ print "Blocks: " . $stats{'blocked'} . " (" .
+ get_percentage($stats{requests}, $stats{'blocked'}) . ")\n";
+ print "Fast redirections: " . $stats{'fast-redirections'} . " (" .
+ get_percentage($stats{requests}, $stats{'fast-redirections'}) . ")\n";
+ print "Connection timeouts: " . $stats{'connection-timeout'} . " (" .
+ get_percentage($stats{requests}, $stats{'connection-timeout'}) . ")\n";
+ print "Connection failures: " . $stats{'connection-failure'} . " (" .
+ get_percentage($stats{requests}, $stats{'connection-failure'}) . ")\n";
+ print "Outgoing requests: " . $outgoing_requests . " (" .
+ get_percentage($stats{requests}, $outgoing_requests) . ")\n";
+ } else {
+ print "No crunches detected. Is 'debug 1024' enabled?\n";
+ }
+
print "Server keep-alive offers: " . $stats{'server-keep-alive'} . " (" .
get_percentage($stats{requests}, $stats{'server-keep-alive'}) . ")\n";
print "New outgoing connections: " . $new_connections . " (" .
get_percentage($stats{requests}, $stats{'empty-responses-on-reused-connections'}) .
")\n";
print "Client connections: " . $stats{'closed-client-connections'} . "\n";
-
+ if ($stats{'content-size-total'}) {
+ print "Bytes transfered excluding headers: " . $stats{'content-size-total'} . "\n";
+ }
my $lines_printed = 0;
print "Client requests per connection distribution:\n";
foreach my $client_requests (sort {
printf "%8d : %-8s\n", $stats{'method'}{$method}, $method;
}
} else {
- print "Method distribution unknown. No response headers parsed yet. Is 'debug 8' enabled?\n";
+ print "Method distribution unknown. No CLF message parsed yet. Is 'debug 512' enabled?\n";
+ }
+ if (exists $stats{'http-version'}) {
+ print "Client HTTP versions:\n";
+ foreach my $http_version (sort {$stats{'http-version'}{$b} <=> $stats{'http-version'}{$a}} keys %{$stats{'http-version'}}) {
+ printf "%8d : %-8s\n", $stats{'http-version'}{$http_version}, $http_version;
+ }
+ } else {
+ print "HTTP version distribution unknown. No CLF message parsed yet. Is 'debug 512' enabled?\n";
}
- print "Client HTTP versions:\n";
- foreach my $http_version (sort {$stats{'http-version'}{$b} <=> $stats{'http-version'}{$a}} keys %{$stats{'http-version'}}) {
- printf "%d : %s\n", $stats{'http-version'}{$http_version}, $http_version;
+ if (exists $stats{'status-code'}) {
+ print "HTTP status codes:\n";
+ foreach my $status_code (sort {$stats{'status-code'}{$b} <=> $stats{'status-code'}{$a}} keys %{$stats{'status-code'}}) {
+ printf "%8d : %-8d\n", $stats{'status-code'}{$status_code}, $status_code;
+ }
+ } else {
+ print "Status code distribution unknown. No CLF message parsed yet. Is 'debug 512' enabled?\n";
}
if ($cli_options{'url-statistics-threshold'} == 0) {
# Functions that actually print stuff
################################################################################
-sub print_clf_message () {
+sub print_clf_message() {
our ($ip, $timestamp, $request_line, $status_code, $size);
my $output = '';
print $output;
}
-sub print_non_clf_message ($) {
+sub print_non_clf_message($) {
my $content = shift;
my $msec_string = $no_msecs_mode ? '' : '.' . $req{$t}{'msecs'};
. $line_end;
}
-sub shorten_thread_id ($) {
+sub shorten_thread_id($) {
my $thread_id = shift;
return $short_thread_ids{$thread_id}
}
-sub parse_loop () {
+sub parse_loop() {
my ($day, $time_stamp, $thread, $log_level, $content, $c, $msecs);
my $last_msecs = 0;
}
}
-sub stats_loop () {
+sub stats_loop() {
my ($day, $time_stamp, $msecs, $thread, $log_level, $content);
my $strict_checks = cli_option_is_set('strict-checks');
while (<>) {
(undef, $time_stamp, $thread, $log_level, $content) = split(/ /, $_, 5);
- # Skip LOG_LEVEL_CLF
- next if (not defined($log_level) or $time_stamp eq "-");
- if (defined($log_level_handlers{$log_level})) {
+ next if (not defined($log_level));
+
+ if ($time_stamp eq "-") {
+
+ gather_loglevel_clf_stats($content);
+
+ } elsif (defined($log_level_handlers{$log_level})) {
$content = $log_level_handlers{$log_level}($content, $thread);
print $version_message;
}
-sub get_cli_options () {
+sub get_cli_options() {
our %cli_options = (
'html-output' => CLI_OPTION_DEFAULT_TO_HTML_OUTPUT,
$line_end = get_line_end();
}
-sub help () {
+sub help() {
our %cli_options;
################################################################################
# main
################################################################################
-sub main () {
+sub main() {
get_cli_options();
set_background(DEFAULT_BACKGROUND);
Overwrite conditions are an alternative and can be added in any action
file as long as the come after the test that is expected to fail.
-They causes all previous tests a matching the condition to be skipped.
+They cause all previous tests that match the condition to be skipped.
It is recommended to put the overwrite condition below the custom Privoxy
section that causes the expected test failure and before the custom test
freez(http->url);
freez(http->hostport);
freez(http->path);
- freez(http->ver);
+ freez(http->version);
freez(http->host_ip_addr_str);
#ifndef FEATURE_EXTENDED_HOST_PATTERNS
freez(http->dbuffer);
/*
* Got a path.
*
- * NOTE: The following line ignores the path for HTTPS URLS.
- * This means that you get consistent behaviour if you type a
- * https URL in and it's parsed by the function. (When the
- * URL is actually retrieved, SSL hides the path part).
+ * If FEATURE_HTTPS_INSPECTION isn't available, ignore the
+ * path for https URLs so that we get consistent behaviour
+ * if a https URL is parsed. When the URL is actually
+ * retrieved, https hides the path part.
*/
- http->path = strdup_or_die(http->ssl ? "/" : url_path);
+ http->path = strdup_or_die(
+#ifndef FEATURE_HTTPS_INSPECTION
+ http->ssl ? "/" :
+#endif
+ url_path
+ );
*url_path = '\0';
http->hostport = strdup_or_die(url_noproto);
}
*/
http->cmd = strdup_or_die(req);
http->gpc = strdup_or_die(v[0]);
- http->ver = strdup_or_die(v[2]);
+ http->version = strdup_or_die(v[2]);
http->ocmd = strdup_or_die(http->cmd);
freez(buf);
* File : $Source: /cvsroot/ijbswa/current/w32log.c,v $
*
* Purpose : Functions for creating and destroying the log window,
- * ouputting strings, processing messages and so on.
+ * outputting strings, processing messages and so on.
*
* Copyright : Written by and Copyright (C) 2001-2009 members of
* the Privoxy team. https://www.privoxy.org/
* File : $Source: /cvsroot/ijbswa/current/w32log.h,v $
*
* Purpose : Functions for creating and destroying the log window,
- * ouputting strings, processing messages and so on.
+ * outputting strings, processing messages and so on.
*
* Copyright : Written by and Copyright (C) 2001-2009 members of
* the Privoxy team. https://www.privoxy.org/
#if 0
/*
* Cheat in parsing the command line. We only ever have at most one
- * paramater, which may optionally be specified inside double quotes.
+ * parameter, which may optionally be specified inside double quotes.
*/
if (lpCmdLine != NULL)
projects / privoxy.git / commitdiff