1 Announcing Privoxy 3.0.27 stable
2 --------------------------------------------------------------------
4 Privoxy 3.0.27 stable scales better in multi-user environments
5 and brings a couple of tuning directives.
6 --------------------------------------------------------------------
8 --------------------------------------------------------------------
9 - General improvements:
10 - Add a receive-buffer-size directive which can be used to
11 set the size of the previously statically allocated buffer
12 in handle_established_connection().
13 Increasing the buffer size increases Privoxy's memory usage but
14 can lower the number of context switches and thereby reduce the
15 CPU usage and potentially increase the throughput.
16 This is mostly relevant for fast network connections and
17 large downloads that don't require filtering.
18 Sponsored by: Robert Klemme
19 - Add a listen-backlog directive which specifies the backlog
20 value passed to listen().
21 Sponsored by: Robert Klemme
22 - Add an enable-accept-filter directive which allows to
23 toggle accept filter support at run time when compiled
24 with FEATURE_ACCEPT_FILTER support.
25 It makes testing more convenient and now that it's
26 optional we can emit an error message if enabling
27 the accept filter fails.
28 Sponsored by: Robert Klemme
29 - Add a delay-response{} action.
30 This is useful to tar pit JavaScript requests that
31 are endlessly retried in case of blocks. It can also
32 be used to simulate a slow Internet connection.
33 Sponsored by: Robert Klemme
34 - Add a 'trusted-cgi-referrer' directive.
35 It allows to configure another page or site that can be used
36 to reach sensitive CGI resources.
37 Sponsored by: Robert Klemme
38 - Add a --fuzz mode which exposes Privoxy internals to input
40 Mainly tested with American Fuzzy Lop. For details see:
41 https://www.fabiankeil.de/talks/fuzzing-on-freebsd/
42 This work was partially funded with donations and done
43 as part of the Privoxy month in 2015.
44 - Consistently use the U(ngreedy) flag in the 'img-reorder' filter.
45 - listen_loop(): Reuse a single thread attribute object
46 The object doesn't change and creating a new one for
47 every thread is a waste of (CPU) time.
48 Sponsored by: Robert Klemme
49 - Free csp resources in the thread that belongs to the csp instead
50 of the main thread which has enough on its plate already.
51 Sponsored by: Robert Klemme
52 - Improve 'socket timeout reached' message.
53 Log the timeout that was triggered and downgrade the
54 log level to LOG_LEVEL_CONNECT to reduce the log noise
55 with common debug settings.
56 The timeout isn't necessary the result of an error and
57 usually merely indicates that Privoxy's socket timeout
58 is lower than the relevant timeouts used by client and
60 Sponsored by: Robert Klemme
61 - Explicitly taint the server socket in case of CONNECT requests.
62 This doesn't fix any known problems, but makes
63 some log messages less confusing.
64 - Let write_pid_file() terminate if the pid file can't be opened.
65 Logging the issue at info level is unlikely to help.
66 - log_error(): Reduce the mutex-protected area by not using a
67 heap-allocated buffer that is shared between all threads.
68 This increases performance and reduces the latency with
69 verbose debug settings and multiple concurrent connections.
70 Sponsored by: Robert Klemme
71 - Let zalloc() use calloc() if it's available.
72 In some situations using calloc() can be faster than
73 malloc() + memset() and it should never be slower.
74 In the real world the impact of this change is not
75 expected to be noticeable.
76 Sponsored by: Robert Klemme
77 - Never use select() when poll() is available.
78 On most platforms select() is limited by FD_SETSIZE while
79 poll() is not. This was a scaling issue for multi-user setups.
80 Using poll() has no downside other than the usual risk
81 that code modifications may introduce new bugs that have
82 yet to be found and fixed.
83 At least in theory this commit could also reduce the latency
84 when there are lots of connections and select() would use
85 "bit fields in arrays of integers" to store file descriptors.
86 Another side effect is that Privoxy no longer has to stop
87 monitoring the client sockets when pipelined requests are
88 waiting but can't be read yet.
89 This code keeps the select()-based code behind ifdefs for
90 now but hopefully it can be removed soonish to make the
92 Sponsored by: Robert Klemme
93 - Add a 'reproducible-tarball-dist' target.
94 It's currently separate from the "tarball-dist" target
95 because it requires a tar implementation with mtree spec
97 It's far from being perfect and does not enforce a
98 reproducible mode, but it's better than nothing.
99 - Use arc4random() if it's available.
100 While Privoxy doesn't need high quality pseudo-random numbers
101 there's no reason not to use them when we can and this silences
102 a warning emitted by code checkers that can't tell whether or not
104 - Show the FEATURE_EXTERNAL_FILTERS status on the status page.
105 Better late than never. Previously a couple of tests weren't
106 executed as Privoxy-Regression-Test couldn't detect that the
107 FEATURE_EXTERNAL_FILTERS dependency was satisfied.
108 - Ditch FEATURE_IMAGE_DETECT_MSIE.
109 It's an obsolete workaround we inherited from Junkbuster
110 and was already disabled by default.
111 Users that feel the urge to work around issues with
112 image requests coming from an Internet Explorer version
113 from more than 15 years ago can still do this using tags.
114 - Consistently use strdup_or_die() instead of strdup() in
115 cases where allocation failures aren't expected.
116 Using strdup_or_die() allows to remove a couple of explicit
117 error checks which slightly reduces the size of the binary.
118 - Insert a refresh tag into the /client-tags CGI page when
119 serving it while a client-specific tag is temporarily enabled.
120 This makes it less likely that the user ends up
121 looking at tag state that is out of date.
122 - Use absolute URLs in the client-tag forms.
123 It's more consistent with the rest of the CGI page
124 URLs and makes it more convenient to copy the forms
126 - cgi_error_disabled(): Use status code 403 and an appropriate response line
127 - Use a dedicated CGI handler to deal with tag-toggle requests
128 As a result the /client-tags page is now safe to reach without
129 trusted Referer header which makes bookmarking or linking to
131 Finally, refreshing the /client-tags page to show the
132 current state can no longer unintentionally repeat the
133 previous toggle request.
134 - Don't add a "Connection" header for CONNECT requests.
135 Explicitly sending "Connection: close" is not necessary and
136 apparently it causes problems with some forwarding proxies
137 that will close the connection prematurely.
138 Reported by Marc Thomas.
139 - Fix compiler warnings.
142 - rfc2553_connect_to(): Properly detect and log when poll()
143 reached the time out. Previously this was logged as:
144 Could not connect to [...]: No error: 0.
145 which isn't very helpful.
146 Sponsored by: Robert Klemme
147 - add_tag_for_client(): Set time_to_live properly.
148 Previously the time_to_live was always set for the first tag.
149 Attempts to temporarily enable a tag would result in enabling
150 it permanently unless no tag was enabled already.
151 - Revert r1.165 which didn't perform as advertised.
152 While the idea was to use "https:// when creating links
153 for the user manual on the website", the actual effect
154 was to use "https://" when Privoxy was supposed to serve
155 the user manual itself.
156 Reported by Yossi Zahn on Privoxy-devel@.
157 - socks5_connect(): Fail in case of unsupported address types.
158 Previously they would not be detected right away and
159 Privoxy would fail later on with an error message that
160 didn't make it obvious that the problem was socks-related.
161 So far, no such problems have actually been reported.
162 - socks5_connect(): Properly deal with socks replies that
163 contain IPv6 addresses.
164 Previously parts of the reply were left unread and
165 later on treated as invalid HTTP response data.
166 Fixes #904 reported by Danny Goossen who also provided
167 the initial version of this patch.
169 - Action file improvements:
170 - Unblock 'msdn.microsoft.com/'.
171 It (presumably) isn't used to serve the kind of ads Privoxy should
172 block by default but happens to serve lots of pages with URLs that
173 are likely to result in false positives.
174 Reported by bugreporter1694 in AF#939.
175 - Disable gif deanimation for requests tagged with CSS-REQUEST.
176 The action will ignore content that isn't considered text
177 anyway and explicitly disabling it makes this more obvious
178 if "action" debugging (debug 65536) is enabled while
179 "gif deanimation" debugging (debug 256) isn't.
180 - Explicitly disable HTML filters for requests with CSS-REQUEST tag.
181 The filters are unlikely to break CSS files but executing
182 them without (intentionally) getting any hits is a waste of
183 cpu time and makes the log more noisy when running with
185 - Unblock 'adventofcode.com/'.
186 Reported by Clint Adams in Debian bug #848211.
187 Fixes Roland's AF#937.
188 - Unblock 'adlibris.com'.
189 Reported by Wyrex in #935
190 - Unblock .golang.org/
191 - Add fast-redirects exception for '.youtube.com/.*origin=http'
193 - Privoxy-Log-Parser:
194 - Don't gather host and resource statistics if they aren't requested.
195 While the performance impact seems negligible this significantly
196 reduces the memory usage if there are lots of requests.
197 - Bump version as the behaviour (slightly) changed.
198 - Count connection failures as well in statistics mode.
199 Sponsored by: Robert Klemme
200 - Count connection timeouts as well in statistics mode.
201 Sponsored by: Robert Klemme
202 - Fix an 'uninitialized value' warning when generating
203 statistics for a log file without response headers.
204 While privoxy-log-parser was supposed to detect this already,
205 the check was flawed and the message the user didn't see was
206 somewhat confusing anyway.
207 Now the message is less confusing, more helpful and actually printed.
208 Reported by: Robert Klemme
210 - Documentation improvements:
211 - Refer to the git sources instead of CVS.
212 - Use GNU/Linux when referring to the OS instead of the kernel.
213 - Add FAQ entry for what to do if editing the config file is access denied.
214 - Add brief HTTP/2 FAQ.
215 - Add a small fuzzing section to the developer documentation.
216 - Add a client-header-tagger{client-ip-address} example.
217 - Stop suggesting that Privoxy is an anonymizing proxy.
218 The term could lead to Privoxy users overestimating
219 what it can do on its own (without Tor).
220 - Make it more obvious that SPI accepts Paypal, too.
221 Currently most donations are made through the Paypal account
222 managed by Zwiebelfreunde e.V. and a more even distribution
224 - Suggest to log applying actions as well when reproducing problems.
225 - Explicitly mention that Privoxy binaries are built by individuals
226 on their own systems. Buyer beware!
227 - Mention the release feed on the homepage.
228 - Remove a mysterious comment with a GNU FDL link as it isn't
229 useful and could confuse license scanners.
230 In May 2002 it was briefly claimed that "this document" was covered
231 by the GNU FDL. The commit message (r1.5) doesn't explain the motivation
232 or whether all copyright holders were actually asked and agreed to the
233 declared license change.
234 It's thus hard to tell whether or not the license change was legit,
235 but luckily two days later the "doc license" was "put" "back to GPL"
237 At the same time the offending comment with a link to the FDL
238 (not the GPL) was added for no obvious reason.
242 - Bump for-privoxy-version to 3.0.27 as we now rely on untrusted
243 CGI request being rejected with status code 403 (instead of 200).
244 - Update test for /send-stylesheet and add another one
247 - Consistently use https:// when linking to the Privoxy website.
248 - Remove SourceForge references in Copyright header.
249 - Remove a couple of SourceForge references in a comment.
250 While at it, fix the grammar.
251 - Move the site-specific documentation block before the generic one.
252 While most Privoxy installations don't have a site-specific
253 documentation block, in cases were it exists it's likely to
254 be more relevant than the generic one.
255 Showing it first makes it less likely that users stop reading
256 before they reach it, especially on pages that don't fit on
259 - Build system improvements:
260 - Prefer openjade to jade. On some systems Jade produces
261 HTML with unescaped ampersands in URLs.
262 - Prefer OpenSP to SP to be consistent.
263 - Have Docbook generated HTML files be straight ASCII.
264 Dealing with a mixture of ISO-8859 and UTF-8 files is problematic.
265 - Echo the filename to stderr for 'make dok-tidy'.
266 Make it a bit easier to find errors in docbook generated HTML.
267 - Warn when still using select().
268 - Warn when compiling without calloc().
269 - Make it more obvious that the --with-fdsetsize configure switch
270 is pointless if poll() is available.
271 - Remove support for AmigaOS.
272 - Update windows build system to use supported software.
273 The cygwin gcc -mno-cygwin option is no longer supported, so
274 convert the windows build system to use the cygwin cross-compiler
275 to build "native" code.
276 - Add --enable-static-linking option for configure
277 does the same thing as LDFLAGS=-static; ./configure
278 but nicer than mixing evars and configure options.
280 -----------------------------------------------------------------
282 -----------------------------------------------------------------
284 Privoxy is a non-caching web proxy with advanced filtering capabilities for
285 enhancing privacy, modifying web page data and HTTP headers, controlling
286 access, and removing ads and other obnoxious Internet junk. Privoxy has a
287 flexible configuration and can be customized to suit individual needs and
288 tastes. It has application for both stand-alone systems and multi-user
291 Privoxy is Free Software and licensed under the GNU GPLv2.
293 Our TODO list is rather long. Helping hands and donations are welcome:
295 * https://www.privoxy.org/faq/general.html#PARTICIPATE
297 * https://www.privoxy.org/faq/general.html#DONATE
299 At present, Privoxy is known to run on Windows 95 and later versions
300 (98, ME, 2000, XP, Vista, Windows 7 etc.), GNU/Linux (RedHat, SuSE,
301 Debian, Fedora, Gentoo, Slackware and others), Mac OS X (10.4 and
302 upwards on PPC and Intel processors), OS/2, Haiku, DragonFly, ElectroBSD,
303 FreeBSD, NetBSD, OpenBSD, Solaris, and various other flavors of Unix.
305 In addition to the core features of ad blocking and cookie management,
306 Privoxy provides many supplemental features, that give the end-user
307 more control, more privacy and more freedom:
309 * Supports "Connection: keep-alive". Outgoing connections can be kept
310 alive independently from the client. Currently not available on all
313 * Supports IPv6, provided the operating system does so too,
314 and the configure script detects it.
316 * Supports tagging which allows to change the behaviour based on client
319 * Can be run as an "intercepting" proxy, which obviates the need to
320 configure browsers individually.
322 * Sophisticated actions and filters for manipulating both server and
325 * Can be chained with other proxies.
327 * Integrated browser based configuration and control utility at
328 http://config.privoxy.org/ (shortcut: http://p.p/). Browser-based
329 tracing of rule and filter effects. Remote toggling.
331 * Web page filtering (text replacements, removes banners based on size,
332 invisible "web-bugs" and HTML annoyances, etc.)
334 * Modularized configuration that allows for standard settings and user
335 settings to reside in separate files, so that installing updated actions
336 files won't overwrite individual user settings.
338 * Support for Perl Compatible Regular Expressions in the configuration
339 files, and a more sophisticated and flexible configuration syntax.
343 * Bypass many click-tracking scripts (avoids script redirection).
345 * User-customizable HTML templates for most proxy-generated pages (e.g.
348 * Auto-detection and re-reading of config file changes.
350 * Most features are controllable on a per-site or per-location basis.
354 https://www.privoxy.org/
356 - Privoxy Developers <privoxy-devel@lists.privoxy.org>