X-Git-Url: http://www.privoxy.org/gitweb/?p=privoxy.git;a=blobdiff_plain;f=doc%2Fsource%2Fdeveloper-manual.sgml;h=89d0b004aff5809a6827589f9e92de3876c30473;hp=ce5b03d68808e1c5be1cc064f45906f079c55b6e;hb=352696e3ebdddaaf4d370ee35f2bab3ed3b18134;hpb=ab25b26c4bd9b2c6bbb1750cd9fd5f073c384c8e diff --git a/doc/source/developer-manual.sgml b/doc/source/developer-manual.sgml index ce5b03d6..89d0b004 100644 --- a/doc/source/developer-manual.sgml +++ b/doc/source/developer-manual.sgml @@ -5,10 +5,10 @@ - - - - + + + + @@ -21,7 +21,7 @@ This file belongs into ijbswa.sourceforge.net:/home/groups/i/ij/ijbswa/htdocs/ - $Id: developer-manual.sgml,v 2.74 2016/08/22 20:19:40 ler762 Exp $ + $Id: developer-manual.sgml,v 2.83 2017/06/08 13:08:39 fabiankeil Exp $ Copyright (C) 2001-2016 Privoxy Developers https://www.privoxy.org/ See LICENSE. @@ -49,7 +49,7 @@ - $Id: developer-manual.sgml,v 2.74 2016/08/22 20:19:40 ler762 Exp $ + $Id: developer-manual.sgml,v 2.83 2017/06/08 13:08:39 fabiankeil Exp $ + + Writing Documentation Using DocBook - A Crash Course useful. @@ -505,7 +503,6 @@ Hal. Here it is: - @@ -516,21 +513,24 @@ Hal. Tags delimiting a block of text (even small blocks) should be on their own line. Like: + <para> Some text goes here. </para> + Tags marking individual words, or few words, should be in-line: + Just to <emphasis>emphasize</emphasis>, some text goes here. - Tags should be nested and step indented for block text like: (except in-line tags) + <para> <itemizedlist> @@ -542,6 +542,7 @@ Hal. </itemizedlist> </para> + This makes it easier to find the text amongst the tags ;-) @@ -602,7 +603,6 @@ Hal. - @@ -634,7 +634,6 @@ Hal. custom entities are listed below. See any of the main docs for examples. - @@ -678,7 +677,6 @@ Hal. - There are others in various places that are defined for a specific purpose. Read the source! @@ -946,11 +944,9 @@ int ms_iis5_hack = 0; Instead of: - int msiis5hack = 0; int msIis5Hack = 0; - @@ -973,12 +969,10 @@ int load_some_file(struct client_state *csp) Instead of: - int loadsomefile(struct client_state *csp) int loadSomeFile(struct client_state *csp) - @@ -997,13 +991,12 @@ int loadSomeFile(struct client_state *csp) (.h) extern int load_aclfile(struct client_state *csp); (.c) int load_aclfile(struct client_state *csp) - Instead of: + Instead of: (.h) extern int load_aclfile(struct client_state *); or (.h) extern int load_aclfile(); (.c) int load_aclfile(struct client_state *csp) - @@ -1055,7 +1048,6 @@ int loadSomeFile(struct client_state *csp) Instead of: - #define USE_IMG_LST 1 or #define _USE_IMAGE_LIST 1 or @@ -1063,7 +1055,6 @@ int loadSomeFile(struct client_state *csp) #define use_image_list 1 or #define UseImageList 1 - @@ -1464,12 +1455,10 @@ for (size_t cnt = 0; cnt < len; cnt++) Exception: - /* This is not a local include, but requires a path element. */ #include <sys/fileName.h> - Note: Please! do not add "-I." to the Makefile without a _very_ good reason. This duplicates the #include @@ -1835,7 +1824,7 @@ const char FILENAME_h_rcs[] = FILENAME_H_VERSION; Note: This declares the rcs variables that should be - added to the "show-proxy-args" page. If this is a brand new + added to the "show-version" page. If this is a brand new creation by you, you are free to change the "Copyright" section to represent the rights you wish to maintain. @@ -1948,13 +1937,14 @@ int FUNCTION_NAME(void *param1, const char *x) Testplan for releases Explain release numbers. major, minor. developer releases. etc. - + Remove any existing rpm with rpm -e Remove any file that was left over. This includes (but is not limited to) + /var/log/privoxy /etc/privoxy @@ -1962,7 +1952,7 @@ Remove any file that was left over. This includes (but is not limited to) /etc/init.d/privoxy /usr/doc/privoxy* - + Install the rpm. Any error messages? @@ -1972,9 +1962,72 @@ Install the rpm. Any error messages? Start browsing. Does Privoxy work? Logfile written? Remove the rpm. Any error messages? All files removed? - + + + Fuzzing Privoxy + + To make fuzzing more convenient, Privoxy can be configured + with --enable-fuzz which will result in the --fuzz option + becoming available. + + + Example (tested on ElectroBSD): + + +# Compile Privoxy with instrumentation for afl +$ export CC=afl-clang +$ export CFLAGS="-fsanitize=address -ggdb" +$ export CPPFLAGS=-I/usr/local/include/ +$ export LDFLAGS="-fsanitize=address -L/usr/local/lib" +$ export AFL_USE_ASAN=1 +$ export AFL_HARDEN=1 +$ ./configure --with-debug --enable-extended-host-patterns --enable-accept-filter --enable-no-gifs --enable-compression --enable-strptime-sanity-checks --enable-external-filters --enable-fuzz + +$ ./privoxy --fuzz +Privoxy version 3.0.24 (http://www.privoxy.org/) +Usage: ./privoxy [--config-test] [--chroot] [--help] [--no-daemon] [--pidfile pidfile] [--pre-chroot-nslookup hostname] [--user user[.group]] [--version] [configfile] + ./privoxy --fuzz fuzz-mode ./path/to/fuzzed/input [--stfu] + +Supported fuzz modes and the expected input: + action: Text to parse as action file. + client-request: Client request to parse. Currently incomplete + client-header: Client header to parse. + chunked-transfer-encoding: Chunk-encoded data to dechunk. + deflate: deflate-compressed data to decompress. + filter: Text to parse as filter file. + gif: gif to deanimate. + gzip: gzip-compressed data to decompress. + pcrs-substitute: A pcrs-substitute to compile. Not a whole pcrs job! Example: Bla $1 bla C $3 blah. + server-header: Server header to parse. + server-response: Server response to parse. + +The following fuzz modes read data from stdin if the 'file' is '-' + client-request + client-header + chunked-transfer-encoding + deflate + gif + gzip + pcrs-substitute + server-header + server-response + +Aborting + +$ export ASAN_OPTIONS='abort_on_error=1' +$ mkdir input output +$ echo '$1 bla fasel $2' > input/pcrs +$ afl-fuzz -i input -o output -m none ~/git/privoxy/privoxy --fuzz pcrs-substitute - --stfu + +$ cat >input/pcrs.txt +FILTER: bla fasel +s@(.{1})[432](\d+)@$1$2$hostname@UgisT + +$ afl-fuzz -i input/ -o output/ -f bla.filter -m none privoxy --fuzz filter bla.filter --stfu + + @@ -2005,6 +2058,7 @@ Install the rpm. Any error messages? First you need to determine which version number the release will have. Privoxy version numbers consist of three numbers, separated by dots, like in X.Y.Z (e.g. 3.0.0), where: + @@ -2056,7 +2110,6 @@ Install the rpm. Any error messages? - In summary, the main CVS trunk is the development branch where new features are being worked on for the next stable series. This should @@ -2089,7 +2142,6 @@ Install the rpm. Any error messages? The following must be done by one of the developers prior to each new release. - @@ -2102,43 +2154,29 @@ Install the rpm. Any error messages? Increment the version number (point from odd to even in development - branches!) in configure.in. (RPM spec files - will need to be incremented as well.) + branches!) in configure.in and update the code + status (CODE_STATUS="xxx") to one of "alpha", "beta" or "stable". + Rebuild configure and GNUMakefile to make sure the updated values are + being used. - Update the code status (CODE_STATUS="xxx") to one of "alpha", "beta" or "stable". + Use the dok-release target to update the sgml documentation source files. - If action file processing has changed and is not backward-compatable, + If action file processing has changed and is not backward-compatible, make sure the "for-privoxy-version=x.y.z" minimum version number in default.action.master has been updated: + {{settings}} ############################################################################# #MASTER# COMMENT: The minimum Privoxy version: for-privoxy-version=3.0.11 - - - - - If default.action has changed since last - release (i.e. software release or standalone actions file release), - bump up its version info to A.B in this line: - - - - {+add-header{X-Actions-File-Version: A.B} -filter -no-popups} - - - - Then change the version info in doc/webserver/actions/index.php, - line: '$required_actions_file_version = "A.B";' - @@ -2147,7 +2185,7 @@ for-privoxy-version=3.0.11 without the ability to build these). Some docs may require rather obscure processing tools. config, the man page (and the html version of the man page) - fall in this category. REAMDE, the man page, AUTHORS, and config + fall in this category. README, the man page, AUTHORS, and config should all also be committed to CVS for other packagers. The formal docs should be uploaded to the webserver. See the Section "Updating the webserver" in this manual for details. @@ -2203,7 +2241,6 @@ for-privoxy-version=3.0.11 - @@ -2220,14 +2257,12 @@ for-privoxy-version=3.0.11 asked for a password): - mkdir dist # delete or choose different name if it already exists cd dist cvs -d:pserver:anonymous@ijbswa.cvs.sourceforge.net:/cvsroot/ijbswa login cvs -z3 -d:pserver:anonymous@ijbswa.cvs.sourceforge.net:/cvsroot/ijbswa export -r v_X_Y_Z current - Do NOT change a single bit, including, but not limited to @@ -2259,7 +2294,6 @@ for-privoxy-version=3.0.11 Please keep these general guidelines in mind when putting together your package. These apply to all platforms! - @@ -2368,7 +2402,6 @@ for-privoxy-version=3.0.11 - @@ -2378,28 +2411,22 @@ for-privoxy-version=3.0.11 version into an empty directory. (See "Building and releasing packages" above). Then run: - cd current autoheader && autoconf && ./configure - Then do: - make tarball-dist - To upload the package to Sourceforge, simply issue - make tarball-upload - Go to the displayed URL and release the file publicly on Sourceforge. For the change log field, use the relevant section of the @@ -2432,28 +2459,22 @@ for-privoxy-version=3.0.11 Then run: - cd current autoheader && autoconf && ./configure - Then do - make dist-dist - To upload the package to Sourceforge, simply issue - make dist-upload rpm_packagerev - where rpm_packagerev is the RPM release number as determined above. @@ -2468,11 +2489,9 @@ for-privoxy-version=3.0.11 version into an empty directory. (See "Building and releasing packages" above). Then get the OS/2 Setup module: - cvs -z3 -d:pserver:anonymous@ijbswa.cvs.sourceforge.net:/cvsroot/ijbswa co os2setup - You will need a mix of development tools. The main compilation takes place with IBM Visual Age C++. @@ -2488,28 +2507,22 @@ for-privoxy-version=3.0.11 Edit the os2build.cmd file to set the final executable filename. For example, - installExeName='privoxyos2_setup_X.Y.Z.exe' - Next, edit the IJB.wis file so the release number matches in the PACKAGEID section: - PACKAGEID="Privoxy Team\Privoxy\Privoxy Package\X\Y\Z" - You're now ready to build. Run: - os2build - You will find the WarpIN-installable executable in the ./files directory. Upload this anonymously to @@ -2523,31 +2536,25 @@ for-privoxy-version=3.0.11 Login to Sourceforge's compilefarm via ssh: - ssh cf.sourceforge.net - Choose the right operating system (not the Debian one). When logged in, make sure that you have freshly exported the right version into an empty directory. (See "Building and releasing packages" above). Then run: - cd current autoheader && autoconf && ./configure - Then run - gmake solaris-dist - which creates a gzip'ed tar archive. Sadly, you cannot use make solaris-upload on the Sourceforge machine (no ncftpput). You now have @@ -2568,22 +2575,18 @@ for-privoxy-version=3.0.11 version into an empty directory. (See "Building and releasing packages" above). Then get the Windows setup module: - cvs -z3 -d:pserver:anonymous@ijbswa.cvs.sourceforge.net:/cvsroot/ijbswa co winsetup - Then you can build the package. This is fully automated, and is controlled by winsetup/GNUmakefile. All you need to do is: - cd winsetup make - Now you can manually rename privoxy_setup.exe to privoxy_setup_X_Y_Z.exe, and upload it to @@ -2600,30 +2603,24 @@ for-privoxy-version=3.0.11 entry to debian/changelog, if it is not already there, for example by running: - debchange -v &p-version;-&p-status;-1 "New upstream version" - Then, run: - dpkg-buildpackage -rfakeroot -us -uc -b - This will create ../privoxy_&p-version;-&p-status;-1_i386.deb which can be uploaded. To upload the package to Sourceforge, simply issue - make debian-upload - Mac OS X @@ -2643,10 +2640,10 @@ for-privoxy-version=3.0.11 The OSXPackageBuilder module generates OS X installer packages supporting all Macs running OS X 10.4 and above. Obtain it from CVS as follows into a folder parallel to the exported privoxy source: + cvs -z3 -d:pserver:anonymous@ijbswa.cvs.sourceforge.net:/cvsroot/ijbswa co OSXPackageBuilder - The module contains complete instructions on its usage in the file OS X Package Builder HOWTO.txt. @@ -2670,19 +2667,17 @@ for-privoxy-version=3.0.11 Check out the module from CVS as follows into a folder parallel to the exported privoxy source: + cvs -z3 -d:pserver:anonymous@ijbswa.cvs.sourceforge.net:/cvsroot/ijbswa co osxsetup - Then run: - cd osxsetup build - This will run autoheader, autoconf and configure as well as make. @@ -2695,11 +2690,9 @@ for-privoxy-version=3.0.11 package" button. If you specify ./Privoxy.pkg as the output package name, you can then create the distributable zip file with the command: - zip -r privoxyosx_setup_x.y.z.zip Privoxy.pkg - You can then upload this file directly to the Files section of the Sourceforge project in the Macintosh (OS X) folder. Each new version @@ -2718,10 +2711,10 @@ for-privoxy-version=3.0.11 Check out the module from CVS as follows into a folder parallel to the exported privoxy source: + cvs -z3 -d:pserver:anonymous@ijbswa.cvs.sourceforge.net:/cvsroot/ijbswa co macsetup - The module contains complete instructions on its usage in its README file. The end result will be the @@ -2745,7 +2738,6 @@ for-privoxy-version=3.0.11 to SourceForge, and go through the release steps. The upload is done via FTP: - @@ -2763,7 +2755,6 @@ for-privoxy-version=3.0.11 - Or use the make targets as described above. @@ -2825,11 +2816,9 @@ for-privoxy-version=3.0.11 If you have changed anything in the stable-branch documentation source SGML files, do: - make dok - That will generate doc/webserver/user-manual, doc/webserver/developer-manual, @@ -2852,11 +2841,9 @@ for-privoxy-version=3.0.11 Next, commit any changes from the above steps to CVS. All set? If these are docs in the stable branch, then do: - make webserver - This will do the upload to the webserver (www.privoxy.org) and ensure all files and directories