X-Git-Url: http://www.privoxy.org/gitweb/?p=privoxy.git;a=blobdiff_plain;f=doc%2Fsource%2Fdeveloper-manual.sgml;h=73b94ea9f60e46121df45f5d13fea62fbe568406;hp=1dc08adf64fe412b687bb3dfc869f50c65bdf214;hb=e7bedf5f2879b7870a5298c1a23d9bd455d33188;hpb=c1a16f7bde8a602c293f56f8bf97af63747268d2 diff --git a/doc/source/developer-manual.sgml b/doc/source/developer-manual.sgml index 1dc08adf..73b94ea9 100644 --- a/doc/source/developer-manual.sgml +++ b/doc/source/developer-manual.sgml @@ -5,7 +5,7 @@ - + @@ -19,7 +19,7 @@ Purpose : developer manual - Copyright (C) 2001-2018 Privoxy Developers https://www.privoxy.org/ + Copyright (C) 2001-2020 Privoxy Developers https://www.privoxy.org/ See LICENSE. ======================================================================== @@ -39,7 +39,7 @@ Copyright - &my-copy; 2001-2018 by + &my-copy; 2001-2020 by Privoxy Developers @@ -110,8 +110,8 @@ Hal. --> Privoxy, as an heir to - Junkbuster, is a Free Software project - and the code is licensed under the GNU General Public License version 2. + Junkbuster, is a Free Software project. As such, Privoxy development is potentially open to anyone who has the time, knowledge, and desire to contribute in any capacity. Our goals are simply to continue the mission, @@ -128,13 +128,13 @@ Hal. The first step is to join the privoxy-devel mailing list. - You can submit your ideas or, even better, patches. Patches are best - submitted to the Sourceforge tracker set up for this purpose, but - can be sent to the list for review too. + You can submit your ideas or, even better, patches. + Patches can also be submitted to the + Sourceforge patch tracker. - You will also need to have a git package installed, which will - entail having ssh installed as well, in order to access the git repository. + You will also need to have a git package installed, + in order to access the git repository. Having the GNU build tools is also going to be important (particularly, autoconf and gmake). @@ -157,11 +157,17 @@ Hal. Access to Git - The project's Git repository is hosted at the - Privoxy site. - The Git repository URL is - ssh://git@git.privoxy.org:23/git/privoxy.git, - the central repository is called privoxy, and the + The project's Git repository is hosted on the + Privoxy webserver. + For Privoxy team members with push privileges the Git repository URL is + ssh://git@git.privoxy.org:23/git/privoxy.git. + + + Contributors without push privileges can + git clone https://www.privoxy.org/git/privoxy.git. + + + The central repository is called privoxy, and the source branch is called master. Subfolders exist within the project for target-dependent build and packaging tools, each including the name of the target operating system in their name (e.g. @@ -182,6 +188,60 @@ Hal. such changes are fully tested ought they be pushed back to the central repository master branch. + + Before pushing stuff, please rebase it on a current master so we get + an uncomplicated commit history. Avoid merges where possible. + + + Here's an example git sesssion that should result in a merge-free history: + + +fk@t520 ~/git/privoxy $git checkout master +Switched to branch 'master' +Your branch is up to date with 'origin/master'. +# Make sure you have the latest changes +fk@t520 ~/git/privoxy $git pull +Already up to date. +# Create a local banch for changes +fk@t520 ~/git/privoxy $git checkout -b local-branch +Switched to a new branch 'local-branch' +# Create some change +fk@t520 ~/git/privoxy $gmake dok dok-tidy +[...] +# Review your change +fk@t520 ~/git/privoxy $git diff +[...] +# Commit your changes if they look goood +fk@t520 ~/git/privoxy $git commit -m "developer-manual: Regenerate" doc/webserver/ +[local-branch 1abb7316] developer-manual: Regenerate + 1 file changed, 2 insertions(+), 2 deletions(-) +# Review your commit +fk@t520 ~/git/privoxy $git show +[...] +# Go to the master branch +fk@t520 ~/git/privoxy $git checkout master +Switched to branch 'master' +Your branch is up to date with 'origin/master'. +# Make sure you are still in sync +fk@t520 ~/git/privoxy $git pull +[...] +Already up to date. +# Apply the commit you made to the local-branch +fk@t520 ~/git/privoxy $git cherry-pick local-branch +[master 046e85e2] developer-manual: Regenerate + Date: Tue Dec 15 05:10:07 2020 +0100 + 1 file changed, 2 insertions(+), 2 deletions(-) +# Make sure the history looks as expected +fk@t520 ~/git/privoxy $git log -p +# Finally push your change to the Privoxy repository +fk@t520 ~/git/privoxy $git push +[...] +# Go back to the local branch +fk@t520 ~/git/privoxy $git checkout local-branch +# Rebase on top of master and continue hacking +fk@t520 ~/git/privoxy $git rebase master +Successfully rebased and updated refs/heads/local-branch. + - Privoxy Custom Entities + Privoxy Custom Entities Privoxy documentation is using a number of customized entities to facilitate @@ -1964,6 +2022,48 @@ Install the rpm. Any error messages? + + Testing with <application>Privoxy-Regression-Test</application> + + If you compiled, packaged or merely installed Privoxy, it is recommended to run + Privoxy-Regression-Test to verify that at least + the tested parts of Privoxy are working as expected. + + + This is actually pretty easy. For details, please see + perldoc privoxy-regression-test.pl. + + + Here is an example of what Privoxy-Regression-Test can do for you: + + +# Run all the tests +fk@t520 ~ $privoxy-regression-test.pl +2020-12-14 12:16:32: Asking Privoxy for the number of action files available ... +2020-12-14 12:16:32: Gathering regression tests from 9 action file(s) delivered by Privoxy 3.0.30. +2020-12-14 12:16:32: Executing regression tests ... +2020-12-14 12:16:41: Ooops. Expected removal but: 'Referer: https://p.p/' is still there. +2020-12-14 12:16:41: Failure for test 785. Header 'Referer: https://p.p/' and tag 'hide-referrer{conditional-block}' +2020-12-14 12:16:41: Ooops. Got: 'Referer: https://p.p/' while expecting: 'Referer: http://p.p/' +2020-12-14 12:16:41: Failure for test 791. Header 'Referer: https://p.p/' and tag 'hide-referrer{conditional-forge}' +2020-12-14 12:16:44: Executed 1087 regression tests. Skipped 115. 1085 successes, 2 failures. +# Repeat one of the failing tests and get a curl command to quickly reproduce the problem +# without causing too much log noise. +fk@t520 ~ $privoxy-regression-test.pl --test-number 785 --verbose --debug 4 +2020-12-14 12:17:55: Asking Privoxy for the number of action files available ... +[...] +2020-12-14 12:17:56: Executing regression tests ... +2020-12-14 12:17:56: Executing: curl --include -H 'Proxy-Connection:' -H 'Connection: close' -s -S --user-agent 'Privoxy-Regression-Test 0.7.2' --max-time '5' --globoff -H 'X-Privoxy-Control: hide-referrer{conditional-block}' -H 'Referer: https://p.p/' http://p.p/show-request 2>&1 +2020-12-14 12:17:56: Ooops. Expected removal but: 'Referer: https://p.p/' is still there. +2020-12-14 12:17:56: Failure for test 785 (0/13/5). Header 'Referer: https://p.p/' and tag 'hide-referrer{conditional-block}' +2020-12-14 12:17:56: Executed 1 regression tests. Skipped 1201. 0 successes, 1 failures. + + + Use the if the --privoxy-address option if the + http_proxy environment variable isn't configured. + + + Fuzzing Privoxy @@ -2046,8 +2146,9 @@ $ afl-fuzz -i input/ -o output/ -f bla.filter -m none privoxy --fuzz filter bla. The following programs are required to follow this process: - ncftpput (ncftp), scp, ssh (ssh), - gmake (GNU's version of make), autoconf, cvs. + ssh, + gmake (GNU's version of make), autoconf, git, + a web browser. @@ -2064,7 +2165,7 @@ $ afl-fuzz -i input/ -o output/ -f bla.filter -m none privoxy --fuzz filter bla. X, the version major, is rarely ever changed. It is increased by one if turning a development branch into stable substantially changes the functionality, user interface or configuration syntax. Majors 1 and 2 were - Junkbuster, and 3 will be the first stable + Junkbuster, and 3 is the first stable Privoxy release. @@ -2139,7 +2240,7 @@ $ afl-fuzz -i input/ -o output/ -f bla.filter -m none privoxy --fuzz filter bla. - Before the Release: Freeze + Before the Release The following must be done by one of the developers prior to each new release. @@ -2194,7 +2295,7 @@ for-privoxy-version=3.0.11 $ git tag # to see the tags - $ git log [last release tag]..HEAD > /tmp/log + $ git log [last release tag]..master > /tmp/log # get the commit log since the last release $ utils/makeChangeLog /tmp/log > /tmp/change.log # reformat the commit log @@ -2233,9 +2334,10 @@ for-privoxy-version=3.0.11 All documentation should be rebuilt: + $ make man $ make dok + $ make dok-man $ make dok-tidy - $ make man $ make config-file Finished docs should be then be committed to Git (for those @@ -2249,6 +2351,11 @@ for-privoxy-version=3.0.11 in this manual for details. + + + Commit all files that were changed in the above steps! + + The User Manual is also used for context @@ -2258,30 +2365,18 @@ for-privoxy-version=3.0.11 (this is in addition to the main User Manual link from the main page since we need to keep manuals for various versions available). The CGI pages will link to something like - http://privoxy.org/$(VERSION)/user-manual/. This + https://www.privoxy.org/$(VERSION)/user-manual/. This will need to be updated for each new release. There is no Makefile target for this at this time!!! It needs to be done manually. - - - Commit all files that were changed in the above steps! - - Tag all files in Git with the version number with - cvs tag v_X_Y_Z. + git tag -s v_X_Y_Z. Don't use vX_Y_Z, ver_X_Y_Z, v_X.Y.Z (won't work) etc. - - - If the release was in a development branch, increase the point version - from even to odd (X.Y.(Z+1)) again in configure.in and - commit your change. - - On the webserver, copy the user manual to a new top-level directory @@ -2312,8 +2407,9 @@ for-privoxy-version=3.0.11 mkdir dist # delete or choose different name if it already exists cd dist - cvs -d:pserver:anonymous@ijbswa.cvs.sourceforge.net:/cvsroot/ijbswa login - cvs -z3 -d:pserver:anonymous@ijbswa.cvs.sourceforge.net:/cvsroot/ijbswa export -r v_X_Y_Z current + git clone https://www.privoxy.org/git/privoxy.git + cd privoxy + git checkout v_X_Y_Z @@ -2504,7 +2600,7 @@ for-privoxy-version=3.0.11 be reset to one if this is the first RPM for dist which is built from version X.Y.Z. Check the - file + file list if unsure. Else, it must be set to the highest already available RPM release number for that version plus one. @@ -2535,55 +2631,6 @@ for-privoxy-version=3.0.11 - OS/2 - - First, make sure that you have freshly exported the right - version into an empty directory. (See "Building and releasing - packages" above). Then get the OS/2 Setup module: - - - cvs -z3 -d:pserver:anonymous@ijbswa.cvs.sourceforge.net:/cvsroot/ijbswa co os2setup - - - You will need a mix of development tools. - The main compilation takes place with IBM Visual Age C++. - Some ancillary work takes place with GNU tools, available from - various sources like hobbes.nmsu.edu. - Specificially, you will need autoheader, - autoconf and sh tools. - The packaging takes place with WarpIN, available from various sources, including - its home page: xworkplace. - - - Change directory to the os2setup directory. - Edit the os2build.cmd file to set the final executable filename. - For example, - - - installExeName='privoxyos2_setup_X.Y.Z.exe' - - - Next, edit the IJB.wis file so the release number matches - in the PACKAGEID section: - - - PACKAGEID="Privoxy Team\Privoxy\Privoxy Package\X\Y\Z" - - - You're now ready to build. Run: - - - os2build - - - You will find the WarpIN-installable executable in the - ./files directory. Upload this anonymously to - uploads.sourceforge.net/incoming, create a release - for it, and you're done. Use the release notes and Change Log from the - source tarball package. - - - Solaris Login to Sourceforge's compilefarm via ssh: @@ -2699,15 +2746,15 @@ for-privoxy-version=3.0.11 packages" above). - There are three modules available in the Git repository for use on Mac + There are three modules available in the CVS repository backups for use on Mac OS X, though technically only two of them generate a release (the other can be used to install from source). - OSXPackageBuilder module + OSXPackageBuilder module (Documentation out of date) The OSXPackageBuilder module generates OS X installer packages - supporting all Macs running OS X 10.4 and above. Obtain it from Git as + supporting all Macs running OS X 10.4 and above. Obtain it from CVS as follows into a folder parallel to the exported privoxy source: @@ -2727,14 +2774,14 @@ for-privoxy-version=3.0.11 - osxsetup module (DEPRECATED) + osxsetup module (DEPRECATED) (Documentation out of date) This module is deprecated since the installer it generates places all Privoxy files in one folder in a non-standard location, and supports only Intel Macs running OS X 10.6 or higher. - Check out the module from Git as follows into a folder parallel to the + Check out the module from CVS as follows into a folder parallel to the exported privoxy source: @@ -2772,13 +2819,13 @@ for-privoxy-version=3.0.11 - macsetup module + macsetup module (Documentation out of date) The macsetup module is ideal if you wish to build and install Privoxy from source on a single machine. - Check out the module from Git as follows into a folder parallel to the + Check out the module from CVS as follows into a folder parallel to the exported privoxy source: @@ -2864,7 +2911,7 @@ for-privoxy-version=3.0.11 announce mailing list, Subject: "Version X.Y.Z available for download". Be sure to include the - + download location, the release notes and the Changelog. Also, post an updated News item on the project page Sourceforge, and update the Home page and docs linked from the Home page (see below). Other news sites @@ -2943,9 +2990,9 @@ for-privoxy-version=3.0.11 make webserver - This will do the upload to the - webserver (www.privoxy.org) and ensure all files and directories - there are group writable. + This will do the upload to the SourceForge webserver (which is manually + syncronized with www.privoxy.org) + and ensure all files and directories there are group writable. Please do NOT use any other means of transferring