<!entity p-intro SYSTEM "privoxy.sgml">
<!entity history SYSTEM "history.sgml">
<!entity seealso SYSTEM "seealso.sgml">
-<!entity p-version "3.0.26">
-<!entity p-status "stable">
-<!entity % p-not-stable "IGNORE">
-<!entity % p-stable "INCLUDE">
+<!entity p-version "3.0.27">
+<!entity p-status "UNRELEASED">
+<!entity % p-not-stable "INCLUDE">
+<!entity % p-stable "IGNORE">
<!entity % p-text "IGNORE"> <!-- define we are not a text only doc -->
<!entity % p-doc "INCLUDE"> <!-- and we are a formal doc -->
<!entity % seealso-extra "INCLUDE"> <!-- extra stuff from seealso.sgml -->
This file belongs into
ijbswa.sourceforge.net:/home/groups/i/ij/ijbswa/htdocs/
- $Id: developer-manual.sgml,v 2.74 2016/08/22 20:19:40 ler762 Exp $
+ $Id: developer-manual.sgml,v 2.83 2017/06/08 13:08:39 fabiankeil Exp $
Copyright (C) 2001-2016 Privoxy Developers https://www.privoxy.org/
See LICENSE.
</pubdate>
- <pubdate>$Id: developer-manual.sgml,v 2.74 2016/08/22 20:19:40 ler762 Exp $</pubdate>
+ <pubdate>$Id: developer-manual.sgml,v 2.83 2017/06/08 13:08:39 fabiankeil Exp $</pubdate>
<!--
<para>
Basic Guidelines, for all branches:
</para>
- <para>
<itemizedlist>
<listitem><para>
Please don't commit even
</para>
</listitem>
</itemizedlist>
- </para>
<!--
<para>
</para>
<para>
How do you update the webserver (i.e. the pages on privoxy.org)?
+ </para>
<orderedlist numeration="arabic">
<listitem><para>
sourceforge webserver via scp.
</para></listitem>
</orderedlist>
- </para>
<para>
Finished docs should be occasionally submitted to CVS
Some common elements that you likely will use:
</para>
-<para>
<simplelist>
<member>
<emphasis><para></para></emphasis>, paragraph delimiter. Most
<emphasis><quote></quote></emphasis>, for, doh, quoting text.
</member>
</simplelist>
-</para>
<para>
Look at any of the existing docs for examples of all these and more.
</para>
<para>
- You might also find <quote><ulink
- url="http://opensource.bureau-cornavin.com/crash-course/index.html">Writing Documentation
- Using DocBook - A Crash Course</ulink></quote> useful.
+ You might also find
+ <!-- <quote><ulink url="http://opensource.bureau-cornavin.com/crash-course/index.html">
+ domain no longer exists so link to the wayback archive -->
+ <quote><ulink url="https://web.archive.org/web/20160315230758/http://opensource.bureau-cornavin.com/crash-course/index.html">
+ Writing Documentation Using DocBook - A Crash Course</ulink></quote> useful.
</para>
</sect2>
<para>
Here it is:
</para>
- <para>
<itemizedlist>
<listitem>
<para>
<para>
Tags delimiting a <emphasis>block</emphasis> of text (even small
blocks) should be on their own line. Like:
+ </para>
<literallayout>
<para>
Some text goes here.
</para>
</literallayout>
+ <para>
Tags marking individual words, or few words, should be in-line:
+ </para>
<literallayout>
Just to <emphasis>emphasize</emphasis>, some text goes here.
</literallayout>
- </para>
</listitem>
<listitem>
<para>
Tags should be nested and step indented for block text like: (except
in-line tags)
+ </para>
<literallayout>
<para>
<itemizedlist>
</itemizedlist>
</para>
</literallayout>
+ <para>
This makes it easier to find the text amongst the tags ;-)
</para>
</listitem>
</listitem>
</itemizedlist>
- </para>
</sect2>
custom entities are listed below. See any of the main docs for examples.
</para>
- <para>
<itemizedlist>
<listitem>
<para>
</simplelist>
</listitem>
</itemizedlist>
- </para>
<para>
There are others in various places that are defined for a specific
purpose. Read the source!
<para><emphasis>Instead of:</emphasis></para>
- <para>
<programlisting>
int msiis5hack = 0; int msIis5Hack = 0;
</programlisting>
-</para>
<para><emphasis>Instead of:</emphasis></para>
- <para>
<programlisting>
int loadsomefile(struct client_state *csp)
int loadSomeFile(struct client_state *csp)
</programlisting>
-</para>
</sect3>
(.h) extern int load_aclfile(struct client_state *csp);
(.c) int load_aclfile(struct client_state *csp)</programlisting>
- <para><emphasis>Instead of:</emphasis>
+ <para><emphasis>Instead of:</emphasis></para>
<programlisting>
(.h) extern int load_aclfile(struct client_state *); or
(.h) extern int load_aclfile();
(.c) int load_aclfile(struct client_state *csp)
</programlisting>
-</para>
</sect3>
<para><emphasis>Instead of:</emphasis></para>
- <para>
<programlisting>
#define USE_IMG_LST 1 or
#define _USE_IMAGE_LIST 1 or
#define use_image_list 1 or
#define UseImageList 1
</programlisting>
-</para>
</sect3>
<para><emphasis>Exception:</emphasis></para>
- <para>
<programlisting>
/* This is not a local include, but requires a path element. */
#include <sys/fileName.h>
</programlisting>
-</para>
<para><emphasis>Note:</emphasis> Please! do not add "-I." to the Makefile
without a _very_ good reason. This duplicates the #include
</programlisting>
<para><emphasis>Note:</emphasis> This declares the rcs variables that should be
- added to the "show-proxy-args" page. If this is a brand new
+ added to the "show-version" page. If this is a brand new
creation by you, you are free to change the "Copyright" section
to represent the rights you wish to maintain.</para>
<sect2 id="testing-plan"><title>Testplan for releases</title>
<para>
Explain release numbers. major, minor. developer releases. etc.
-
+ </para>
<orderedlist numeration="arabic">
<listitem><para>
Remove any existing rpm with rpm -e
</para></listitem>
<listitem><para>
Remove any file that was left over. This includes (but is not limited to)
+ </para>
<itemizedlist>
<listitem><para>/var/log/privoxy</para></listitem>
<listitem><para>/etc/privoxy</para></listitem>
<listitem><para>/etc/init.d/privoxy</para></listitem>
<listitem><para>/usr/doc/privoxy*</para></listitem>
</itemizedlist>
-</para></listitem>
+ </listitem>
<listitem><para>
Install the rpm. Any error messages?
</para></listitem>
<listitem><para>Start browsing. Does <application>Privoxy</application> work? Logfile written?</para></listitem>
<listitem><para>Remove the rpm. Any error messages? All files removed?</para></listitem>
</orderedlist>
-</para>
</sect2>
<!-- XXX: Document how to write test reports and where to send them -->
+
+ <!-- ~~~~~ New section ~~~~~ -->
+ <sect2 id="fuzzing"><title>Fuzzing Privoxy</title>
+ <para>
+ To make fuzzing more convenient, Privoxy can be configured
+ with --enable-fuzz which will result in the --fuzz option
+ becoming available.
+ </para>
+ <para>
+ Example (tested on ElectroBSD):
+ </para>
+ <programlisting>
+# Compile Privoxy with instrumentation for afl
+$ export CC=afl-clang
+$ export CFLAGS="-fsanitize=address -ggdb"
+$ export CPPFLAGS=-I/usr/local/include/
+$ export LDFLAGS="-fsanitize=address -L/usr/local/lib"
+$ export AFL_USE_ASAN=1
+$ export AFL_HARDEN=1
+$ ./configure --with-debug --enable-extended-host-patterns --enable-accept-filter --enable-no-gifs --enable-compression --enable-strptime-sanity-checks --enable-external-filters --enable-fuzz
+
+$ ./privoxy --fuzz
+Privoxy version 3.0.24 (http://www.privoxy.org/)
+Usage: ./privoxy [--config-test] [--chroot] [--help] [--no-daemon] [--pidfile pidfile] [--pre-chroot-nslookup hostname] [--user user[.group]] [--version] [configfile]
+ ./privoxy --fuzz fuzz-mode ./path/to/fuzzed/input [--stfu]
+
+Supported fuzz modes and the expected input:
+ action: Text to parse as action file.
+ client-request: Client request to parse. Currently incomplete
+ client-header: Client header to parse.
+ chunked-transfer-encoding: Chunk-encoded data to dechunk.
+ deflate: deflate-compressed data to decompress.
+ filter: Text to parse as filter file.
+ gif: gif to deanimate.
+ gzip: gzip-compressed data to decompress.
+ pcrs-substitute: A pcrs-substitute to compile. Not a whole pcrs job! Example: Bla $1 bla C $3 blah.
+ server-header: Server header to parse.
+ server-response: Server response to parse.
+
+The following fuzz modes read data from stdin if the 'file' is '-'
+ client-request
+ client-header
+ chunked-transfer-encoding
+ deflate
+ gif
+ gzip
+ pcrs-substitute
+ server-header
+ server-response
+
+Aborting
+
+$ export ASAN_OPTIONS='abort_on_error=1'
+$ mkdir input output
+$ echo '$1 bla fasel $2' > input/pcrs
+$ afl-fuzz -i input -o output -m none ~/git/privoxy/privoxy --fuzz pcrs-substitute - --stfu
+
+$ cat >input/pcrs.txt
+FILTER: bla fasel
+s@(.{1})[432](\d+)@$1$2$hostname@UgisT
+
+$ afl-fuzz -i input/ -o output/ -f bla.filter -m none privoxy --fuzz filter bla.filter --stfu
+</programlisting>
+ </sect2>
</sect1>
<!-- ~~~~~ New section ~~~~~ -->
First you need to determine which version number the release will have.
<application>Privoxy</application> version numbers consist of three numbers,
separated by dots, like in X.Y.Z (e.g. 3.0.0), where:
+ </para>
<itemizedlist>
<listitem>
<para>
</para>
</listitem>
</itemizedlist>
- </para>
<para>
In summary, the main CVS trunk is the development branch where new
features are being worked on for the next stable series. This should
The following <emphasis>must be done by one of the
developers</emphasis> prior to each new release.
</para>
- <para>
<itemizedlist>
<listitem>
<para>
<listitem>
<para>
Increment the version number (point from odd to even in development
- branches!) in <filename>configure.in</filename>. (RPM spec files
- will need to be incremented as well.)
+ branches!) in <filename>configure.in</filename> and update the code
+ status (CODE_STATUS="xxx") to one of "alpha", "beta" or "stable".
+ Rebuild configure and GNUMakefile to make sure the updated values are
+ being used.
</para>
</listitem>
<listitem>
<para>
- Update the code status (CODE_STATUS="xxx") to one of "alpha", "beta" or "stable".
+ Use the dok-release target to update the sgml documentation source files.
</para>
</listitem>
<listitem>
<para>
- If action file processing has changed and is not backward-compatable,
+ If action file processing has changed and is not backward-compatible,
make sure the "for-privoxy-version=x.y.z" minimum version number in
default.action.master has been updated:
+ </para>
<programlisting>
{{settings}}
#############################################################################
#MASTER# COMMENT: The minimum Privoxy version:
for-privoxy-version=3.0.11
</programlisting>
- </para>
- </listitem>
- <listitem>
- <para>
- If <filename>default.action</filename> has changed since last
- release (i.e. software release or standalone actions file release),
- bump up its version info to A.B in this line:
- </para>
- <para>
- <programlisting>
- {+add-header{X-Actions-File-Version: A.B} -filter -no-popups}
-</programlisting>
- </para>
- <para>
- Then change the version info in doc/webserver/actions/index.php,
- line: '$required_actions_file_version = "A.B";'
- </para>
</listitem>
<listitem>
<para>
without the ability to build these). Some docs may require
rather obscure processing tools. <filename>config</filename>,
the man page (and the html version of the man page)
- fall in this category. REAMDE, the man page, AUTHORS, and config
+ fall in this category. README, the man page, AUTHORS, and config
should all also be committed to CVS for other packagers. The
formal docs should be uploaded to the webserver. See the
Section "Updating the webserver" in this manual for details.
</para>
</listitem>
</itemizedlist>
- </para>
</sect2>
<sect2 id="therelease">
asked for a password):
</para>
- <para>
<programlisting>
mkdir dist # delete or choose different name if it already exists
cd dist
cvs -d:pserver:anonymous@ijbswa.cvs.sourceforge.net:/cvsroot/ijbswa login
cvs -z3 -d:pserver:anonymous@ijbswa.cvs.sourceforge.net:/cvsroot/ijbswa export -r v_X_Y_Z current
</programlisting>
- </para>
<para>
<emphasis>Do NOT change</emphasis> a single bit, including, but not limited to
Please keep these general guidelines in mind when putting together
your package. These apply to <emphasis>all</emphasis> platforms!
</para>
- <para>
<itemizedlist>
<listitem>
<para>
</listitem>
</itemizedlist>
- </para>
</sect3>
version into an empty directory</emphasis>. (See "Building and releasing
packages" above). Then run:
</para>
- <para>
<programlisting>
cd current
autoheader && autoconf && ./configure
</programlisting>
- </para>
<para>
Then do:
</para>
- <para>
<programlisting>
make tarball-dist
</programlisting>
- </para>
<para>
To upload the package to Sourceforge, simply issue
</para>
- <para>
<programlisting>
make tarball-upload
</programlisting>
- </para>
<para>
Go to the displayed URL and release the file publicly on Sourceforge.
For the change log field, use the relevant section of the
<para>
Then run:
</para>
- <para>
<programlisting>
cd current
autoheader && autoconf && ./configure
</programlisting>
- </para>
<para>
Then do
</para>
- <para>
<programlisting>
make <replaceable class="parameter">dist</replaceable>-dist
</programlisting>
- </para>
<para>
To upload the package to Sourceforge, simply issue
</para>
- <para>
<programlisting>
make <replaceable class="parameter">dist</replaceable>-upload <replaceable class="parameter">rpm_packagerev</replaceable>
</programlisting>
- </para>
<para>
where <replaceable class="parameter">rpm_packagerev</replaceable> is the
RPM release number as determined above.
version into an empty directory</emphasis>. (See "Building and releasing
packages" above). Then get the OS/2 Setup module:
</para>
- <para>
<programlisting>
cvs -z3 -d:pserver:anonymous@ijbswa.cvs.sourceforge.net:/cvsroot/ijbswa co os2setup
</programlisting>
- </para>
<para>
You will need a mix of development tools.
The main compilation takes place with IBM Visual Age C++.
Edit the os2build.cmd file to set the final executable filename.
For example,
</para>
- <para>
<programlisting>
installExeName='privoxyos2_setup_X.Y.Z.exe'
</programlisting>
- </para>
<para>
Next, edit the <filename>IJB.wis</filename> file so the release number matches
in the <filename>PACKAGEID</filename> section:
</para>
- <para>
<programlisting>
PACKAGEID="Privoxy Team\Privoxy\Privoxy Package\X\Y\Z"
</programlisting>
- </para>
<para>
You're now ready to build. Run:
</para>
- <para>
<programlisting>
os2build
</programlisting>
- </para>
<para>
You will find the WarpIN-installable executable in the
<filename>./files</filename> directory. Upload this anonymously to
<para>
Login to Sourceforge's compilefarm via ssh:
</para>
- <para>
<programlisting>
ssh cf.sourceforge.net
</programlisting>
- </para>
<para>
Choose the right operating system (not the Debian one).
When logged in, <emphasis>make sure that you have freshly exported the right
version into an empty directory</emphasis>. (See "Building and releasing
packages" above). Then run:
</para>
- <para>
<programlisting>
cd current
autoheader && autoconf && ./configure
</programlisting>
- </para>
<para>
Then run
</para>
- <para>
<programlisting>
gmake solaris-dist
</programlisting>
- </para>
<para>
which creates a gzip'ed tar archive. Sadly, you cannot use <command>make
solaris-upload</command> on the Sourceforge machine (no ncftpput). You now have
version into an empty directory</emphasis>. (See "Building and releasing
packages" above). Then get the Windows setup module:
</para>
- <para>
<programlisting>
cvs -z3 -d:pserver:anonymous@ijbswa.cvs.sourceforge.net:/cvsroot/ijbswa co winsetup
</programlisting>
- </para>
<para>
Then you can build the package. This is fully automated, and is
controlled by <filename>winsetup/GNUmakefile</filename>.
All you need to do is:
</para>
- <para>
<programlisting>
cd winsetup
make
</programlisting>
- </para>
<para>
Now you can manually rename <filename>privoxy_setup.exe</filename> to
<filename>privoxy_setup_X_Y_Z.exe</filename>, and upload it to
entry to <filename>debian/changelog</filename>, if it is not
already there, for example by running:
</para>
- <para>
<programlisting>
debchange -v &p-version;-&p-status;-1 "New upstream version"
</programlisting>
- </para>
<para>
Then, run:
</para>
- <para>
<programlisting>
dpkg-buildpackage -rfakeroot -us -uc -b
</programlisting>
- </para>
<para>
This will create
<filename>../privoxy_&p-version;-&p-status;-1_i386.deb</filename>
which can be uploaded. To upload the package to Sourceforge, simply
issue
</para>
- <para>
<programlisting>
make debian-upload
</programlisting>
- </para>
</sect3>
<sect3 id="newrelease-macosx"><title>Mac OS X</title>
The OSXPackageBuilder module generates OS X installer packages
supporting all Macs running OS X 10.4 and above. Obtain it from CVS as
follows into a folder parallel to the exported privoxy source:
+ </para>
<programlisting>
cvs -z3 -d:pserver:anonymous@ijbswa.cvs.sourceforge.net:/cvsroot/ijbswa co OSXPackageBuilder
</programlisting>
- </para>
<para>
The module contains complete instructions on its usage in the file
<filename>OS X Package Builder HOWTO.txt</filename>.
<para>
Check out the module from CVS as follows into a folder parallel to the
exported privoxy source:
+ </para>
<programlisting>
cvs -z3 -d:pserver:anonymous@ijbswa.cvs.sourceforge.net:/cvsroot/ijbswa co osxsetup
</programlisting>
- </para>
<para>
Then run:
</para>
- <para>
<programlisting>
cd osxsetup
build
</programlisting>
- </para>
<para>
This will run <filename>autoheader</filename>, <filename>autoconf</filename>
and <filename>configure</filename> as well as <filename>make</filename>.
package" button. If you specify ./Privoxy.pkg as the output package
name, you can then create the distributable zip file with the command:
</para>
- <para>
<programlisting>
zip -r privoxyosx_setup_x.y.z.zip Privoxy.pkg
</programlisting>
- </para>
<para>
You can then upload this file directly to the Files section of the
Sourceforge project in the Macintosh (OS X) folder. Each new version
<para>
Check out the module from CVS as follows into a folder parallel to the
exported privoxy source:
+ </para>
<programlisting>
cvs -z3 -d:pserver:anonymous@ijbswa.cvs.sourceforge.net:/cvsroot/ijbswa co macsetup
</programlisting>
- </para>
<para>
The module contains complete instructions on its usage in its
<filename>README</filename> file. The end result will be the
to SourceForge, and go through the release steps. The upload
is done via FTP:
</para>
- <para>
<itemizedlist>
<listitem>
<para>
</para>
</listitem>
</itemizedlist>
- </para>
<para>
Or use the <command>make</command> targets as described above.
</para>
If you have changed anything in the stable-branch documentation source
SGML files, do:
</para>
- <para>
<programlisting>
make dok
</programlisting>
- </para>
<para>
That will generate <filename>doc/webserver/user-manual</filename>,
<filename>doc/webserver/developer-manual</filename>,
Next, commit any changes from the above steps to CVS. All set?
If these are docs in the stable branch, then do:
</para>
- <para>
<programlisting>
make webserver
</programlisting>
- </para>
<para>
This will do the upload to <ulink url="https://www.privoxy.org/">the
webserver</ulink> (www.privoxy.org) and ensure all files and directories
projects / privoxy.git / blobdiff