X-Git-Url: http://www.privoxy.org/gitweb/?p=privoxy.git;a=blobdiff_plain;f=doc%2Fsource%2Fdeveloper-manual.sgml;h=93f23834ad7b1a1db810b21a55c4326919f8f840;hp=db044832342b3736dd27425141ee97e55f4223ca;hb=979620bb4c7aaaeb96f07b73b823c686d83cf14c;hpb=9eb1467fa22b801e2ddf1da7049767dc12a46af3 diff --git a/doc/source/developer-manual.sgml b/doc/source/developer-manual.sgml index db044832..638a49b9 100644 --- a/doc/source/developer-manual.sgml +++ b/doc/source/developer-manual.sgml @@ -1,41 +1,68 @@ - + + + + + + + + + + + ]>
Privoxy Developer Manual + + + + + Copyright + &my-copy; 2001-2020 by + Privoxy Developers + + + + + @@ -46,137 +73,624 @@ ]]> - The developer manual gives the users information on how to help the developer - team. It provides guidance on coding, testing, documentation and other - issues. - - -&p-intro; + The developer manual provides guidance on coding, testing, packaging, documentation + and other issues of importance to those involved with + Privoxy development. It is mandatory (and helpful!) reading + for anyone who wants to join the team. Note that it's currently out of date + and may not be entirely correct. As always, patches are welcome. + + + + + + + + Please note that this document is constantly evolving. This copy represents + the state at the release of version &p-version;. You can find the latest version of the this manual at http://www.privoxy.org/developer-manual/. - Please see the Contact section of the User Manual on how to contact the - developers. + url="https://www.privoxy.org/developer-manual/">https://www.privoxy.org/developer-manual/. + Please have a look at the + contact section in the user manual + if you are interested in contacting the developers. - - - - - + + Introduction + --> Privoxy, as an heir to - Junkbuster, is an Open Source project - and licensed under the GPL. As such, Privoxy - development is potentially open to anyone who has the time, knowledge, - and desire to contribute in any capacity. Our goals are simply to - continue the mission, to improve Privoxy, and - to make it available to as wide an audience as possible. + Junkbuster, is a Free Software project + and the code is licensed under the GNU General Public License version 2. + As such, Privoxy development is potentially open + to anyone who has the time, knowledge, and desire to contribute + in any capacity. Our goals are simply to continue the mission, + to improve Privoxy, and + to make it available to as wide an audience as possible. One does not have to be a programmer to contribute. Packaging, testing, - and porting, are all important jobs as well. + documenting and porting, are all important jobs as well. - - Quickstart to Privoxy Development + Quickstart to Privoxy Development + + The first step is to join the privoxy-devel mailing list. + You can submit your ideas or, even better, patches. Patches are best + submitted to the Sourceforge tracker set up for this purpose, but + can be sent to the list for review too. + -You'll need an account on Sourceforge to support our development. Mail your ID -to the list and wait until a project manager has added you. - -For the time beeing (read, this section is under construction), please note the -following guidelines for changing stuff in the code. If it is - - - A bugfix / clean-up / cosmetic thing: shoot - - - A new feature that can be turned off: shoot - - - A clear improvement w/o side effects on other parts of the code: shoot - - - A matter of taste: ask the list - - - A major redesign of some part of the code: ask the list - - - - - + You will also need to have a git package installed, + in order to access the git repository. + Having the GNU build tools is also going to be important (particularly, + autoconf and gmake). + + + For the time being (read, this section is under construction), you can + also refer to the extensive comments in the source code. In fact, + reading the code is recommended in any case. + + + + - Documentation Guidelines + The Git Repository - All formal documents are maintained in docbook SGML and located - in the doc/source directory. You will - need docbook and the - docbook stylesheets (or comparable alternatives), and either - jade or openjade - installed in order to build docs from source. Currently there is - user-manual, - FAQ, - and, of course this, the developer-manual in - this format. - - - Other, less formal documents (e.g. README, LICENSE) are - maintained as plain text files in the toplevel source - directory. - - - Packagers are encouraged to include this documentation. For those - without the ability to build the docs locally, text versions of - each are kept in CVS. Or HTML versions can be downloaded from the www.privoxy.org website, which - should be fairly current. - + If you become part of the active development team, you will eventually + need write access to our holy grail, the Git repository. One of the + team members will need to set this up for you. Please read + this chapter completely before accessing via Git. + + + Access to Git + + The project's Git repository is hosted at the + Privoxy website. + For Privoxy team members with push privileges the Git repository URL is + ssh://git@git.privoxy.org:23/git/privoxy.git. + + + Contributors without push privileges can + git clone https://www.privoxy.org/git/privoxy.git. + + + The central repository is called privoxy, and the + source branch is called master. Subfolders exist + within the project for target-dependent build and packaging tools, each + including the name of the target operating system in their name (e.g. + Windows, OSXPackageBuilder, debian). There is a webview of the Git + hierarchy at + + https://www.privoxy.org/gitweb/?p=privoxy.git;a=tree, + which might help with visualizing how these pieces fit together. + + + + + Branches + + Whilst the central repository contains only the master branch, developers + are of course free to create branches in their local repositories as they + develop features, fixes, or update the target-dependent tools. Only once + such changes are fully tested ought they be pushed back to the central + repository master branch. + + + + At one time there were two distinct branches: stable and unstable. The + more drastic changes were to be in the unstable branch. These branches + have now been merged to minimize time and effort of maintaining two + branches. + + + + + Git Commit Guidelines + + The source tree is the heart of every software project. Every effort must + be made to ensure that it is readable, compilable and consistent at all + times. We expect anyone with Git access to strictly + adhere to the following guidelines: + + + + Basic Guidelines, for all branches: + + + + Please don't commit even + a small change without testing it thoroughly first. When we're + close to a public release, ask a fellow developer to review your + changes. + + + Your commit message should give a concise overview of what you + changed (no big details) and why you changed it + Just check previous messages for good examples. + + + Don't use the same message on multiple files, unless it equally applies to + all those files. + + + If your changes span multiple files, and the code won't recompile unless + all changes are committed (e.g. when changing the signature of a function), + then commit all files one after another, without long delays in between. + If necessary, prepare the commit messages in advance. + + + Before changing things on Git, make sure that your changes are in line + with the team's general consensus on what should be done. + + + + Note that near a major public release, we get more cautious. + There is always the possibility to submit a patch to the patch + tracker instead. + + + + + + + + + + +Documentation Guidelines + + All formal documents are maintained in Docbook SGML and located in the + doc/source/* directory. You will need + Docbook, the Docbook + DTD's and the Docbook modular stylesheets (or comparable alternatives), + and either jade or + openjade (recommended) installed in order to + build docs from source. Currently there is user-manual, + FAQ, and, of + course this, the developer-manual in this format. + The README, AUTHORS, + INSTALL, + privoxy.1 (man page), and + config files are also now maintained as Docbook + SGML. These files, when built, in the top-level source directory are + generated files! Also, the Privoxy index.html (and a + variation on this file, privoxy-index.html, + meant for inclusion with doc packages), are maintained as SGML as well. + DO NOT edit these directly. Edit the SGML source, or + contact someone involved in the documentation. + + + config requires some special handling. The reason it + is maintained this way is so that the extensive comments in the file + mirror those in user-manual. But the conversion + process requires going from SGML to HTML to text to special formatting + required for the embedded comments. Some of this does not survive so + well. Especially some of the examples that are longer than 80 characters. + The build process for this file outputs to config.new, + which should be reviewed for errors and mis-formatting. Once satisfied + that it is correct, then it should be hand copied to + config. + + + Other, less formal documents (e.g. LICENSE) are + maintained as plain text files in the top-level source directory. + + + Packagers are encouraged to include this documentation. For those without + the ability to build the docs locally, text versions of each are kept in + Git. HTML versions are also being kept in Git under + doc/webserver/*. + + + Formal documents are built with the Makefile targets of + make dok. + The build process uses the document SGML sources in + doc/source/*/* to update all text files in + doc/text/ and to update all HTML + documents in doc/webserver/. + + + Documentation writers should please make sure documents build + successfully before committing to Git, if possible. + + + How do you update the webserver (i.e. the pages on privoxy.org)? + + + + First, build the docs by running make - dok (or alternately make - redhat-dok). - - - Run make webserver which copies all files from -doc/webserver to the sourceforge webserver -via scp. - - + dok. + + + Run make webserver which copies all + files from doc/webserver to the + sourceforge webserver via scp. + + + + + Finished docs should be occasionally submitted to Git + (doc/webserver/*/*.html) so that those without + the ability to build them locally, have access to them if needed. + This is especially important just prior to a new release! Please + do this after the $VERSION and + other release specific data in configure.in has been + updated (this is done just prior to a new release). + + + + +Quickstart to Docbook and SGML + + If you are not familiar with SGML, it is a markup language similar to HTML. + Actually, not a mark up language per se, but a language used to define + markup languages. In fact, HTML is an SGML application. Both will use + tags to format text and other content. SGML tags can be much + more varied, and flexible, but do much of the same kinds of things. The tags, + or elements, are definable in SGML. There is no set + standards. Since we are using + Docbook, our tags are those that are defined by + Docbook. Much of how the finish document is + rendered is determined by the stylesheets. + The stylesheets determine how each tag gets translated to HTML, or other + formats. + + + + Tags in Docbook SGML need to be always closed. If not, you + will likely generate errors. Example: <title>My + Title</title>. They are also case-insensitive, but we + strongly suggest using all lower case. This keeps compatibility with + [Docbook] XML. + + + + Our documents use sections for the most part. Sections + will be processed into HTML headers (e.g. h1 for + sect1). The Docbook stylesheets + will use these to also generate the Table of Contents for each doc. Our + TOC's are set to a depth of three. Meaning sect1, + sect2, and sect3 will have TOC + entries, but sect4 will not. Each section requires + a <title> element, and at least one + <para>. There is a limit of five section + levels in Docbook, but generally three should be sufficient for our + purposes. + + + + Some common elements that you likely will use: + + + + + <para></para>, paragraph delimiter. Most + text needs to be within paragraph elements (there are some exceptions). + + + <emphasis></emphasis>, the stylesheets + make this italics. + + + <filename></filename>, files and directories. + + + <command></command>, command examples. + + + <literallayout></literallayout>, like + <pre>, more or less. + + + <itemizedlist></itemizedlist>, list with bullets. + + + <listitem></listitem>, member of the above. + + + <screen></screen>, screen output, implies + <literallayout>. + + + <ulink url="example.com"></ulink>, like + HTML <a> tag. + + + <quote></quote>, for, doh, quoting text. + + + + + Look at any of the existing docs for examples of all these and more. + + + + You might also find + + + Writing Documentation Using DocBook - A Crash Course useful. + + + + + + <application>Privoxy</application> Documentation Style + + It will be easier if everyone follows a similar writing style. This + just makes it easier to read what someone else has written if it + is all done in a similar fashion. + + + Here it is: + + + + + All tags should be lower case. + + + + + Tags delimiting a block of text (even small + blocks) should be on their own line. Like: + + + <para> + Some text goes here. + </para> + + + Tags marking individual words, or few words, should be in-line: + + + Just to <emphasis>emphasize</emphasis>, some text goes here. + + + + + Tags should be nested and step indented for block text like: (except + in-line tags) + + + <para> + <itemizedlist> + <para> + <listitem> + Some text goes here in our list example. + </listitem> + </para> + </itemizedlist> + </para> + + + This makes it easier to find the text amongst the tags ;-) + + + + + Use white space to separate logical divisions within a document, + like between sections. Running everything together consistently + makes it harder to read and work on. + + + + + Do not hesitate to make comments. Comments can either use the + <comment> element, or the <!-- --> style comment + familiar from HTML. (Note in Docbook v4.x <comment> is + replaced by <remark>.) + + + + + We have an international audience. Refrain from slang, or English + idiosyncrasies (too many to list :). Humor also does not translate + well sometimes. + + + + + Try to keep overall line lengths in source files to 80 characters or less + for obvious reasons. This is not always possible, with lengthy URLs for + instance. + + + + + Our documents are available in differing formats. Right now, they + are just plain text and/or HTML, but others are always a + future possibility. Be careful with URLs (<ulink>), and avoid + this mistake: + + + My favorite site is <ulink url="http://example.com">here</ulink>. + + + This will render as My favorite site is here, which is + not real helpful in a text doc. Better like this: + + + My favorite site is <ulink url="http://example.com">example.com</ulink>. + + + + + All documents should be spell checked occasionally. + aspell can check SGML with the + -H option. (ispell I think + too.) + + + + + + + + + + + Privoxy Custom Entities + + Privoxy documentation is using + a number of customized entities to facilitate + documentation maintenance. + + + We are using a set of boilerplate files with generic text, + that is used by multiple docs. This way we can write something once, and use + it repeatedly without having to re-write the same content over and over again. + If editing such a file, keep in mind that it should be + generic. That is the purpose; so it can be used in varying + contexts without additional modifications. + + + We are also using what Docbook calls + internal entities. These are like variables in + programming. Well, sort of. For instance, we have the + p-version entity that contains the current + Privoxy version string. You are strongly + encouraged to use these where possible. Some of these obviously + require re-setting with each release (done by the Makefile). A sampling of + custom entities are listed below. See any of the main docs for examples. + + + + + + Re- boilerplate text entities are defined like: + + + <!entity supported SYSTEM "supported.sgml"> + + + In this example, the contents of the file, + supported.sgml is available for inclusion anywhere + in the doc. To make this happen, just reference the now defined + entity: &supported; (starts with an ampersand + and ends with a semi-colon), and the contents will be dumped into + the finished doc at that point. + + + + + + Commonly used internal entities: + + + p-version: the Privoxy + version string, e.g. &p-version;. + + + p-status: the project status, either + alpha, beta, or stable. + + + p-not-stable: use to conditionally include + text in not stable releases (e.g. beta). + + + p-stable: just the opposite. + + + p-text: this doc is only generated as text. + + + + + + There are others in various places that are defined for a specific + purpose. Read the source! + + + + @@ -201,20 +715,20 @@ via scp. Using Comments - + Comment, Comment, Comment Explanation: Comment as much as possible without commenting the obvious. - For example do not comment "aVariable is equal to bVariable". - Instead explain why aVariable should be equal to the bVariable. + For example do not comment "variable_a is equal to variable_b". + Instead explain why variable_a should be equal to the variable_b. Just because a person can read code does not mean they will understand why or what is being done. A reader may spend a lot more time figuring out what is going on when a simple comment or explanation would have prevented the extra research. Please - help your brother IJB'ers out! + help your fellow Privoxy developers out! The comments will also help justify the intent of the code. If the comment describes something different than what the code @@ -223,13 +737,13 @@ via scp. Example: /* if page size greater than 1k ... */ -if ( PageLength() > 1024 ) +if (page_length() > 1024) { ... "block" the page up ... } /* if page size is small, send it in blocks */ -if ( PageLength() > 1024 ) +if (page_length() > 1024) { ... "block" the page up ... } @@ -240,7 +754,7 @@ is actually being done. - + Use blocks for comments @@ -257,33 +771,33 @@ is actually being done. /********************************************************************* * This will stand out clearly in your code! *********************************************************************/ -if ( thisVariable == thatVariable ) +if (this_variable == that_variable) { - DoSomethingVeryImportant(); + do_something_very_important(); } /* unfortunately, this may not */ -if ( thisVariable == thatVariable ) +if (this_variable == that_variable) { - DoSomethingVeryImportant(); + do_something_very_important(); } -if ( thisVariable == thatVariable ) /* this may not either */ +if (this_variable == that_variable) /* this may not either */ { - DoSomethingVeryImportant(); + do_something_very_important(); } Exception: If you are trying to add a small logic comment and do not - wish to "disrubt" the flow of the code, feel free to use a 1 + wish to "disrupt" the flow of the code, feel free to use a 1 line comment which is NOT on the same line as the code. - + - + Keep Comments on their own line @@ -304,14 +818,14 @@ if ( thisVariable == thatVariable ) /* this may not either */ * This will stand out clearly in your code, * But the second example won't. *********************************************************************/ -if ( thisVariable == thatVariable ) +if (this_variable == this_variable) { - DoSomethingVeryImportant(); + do_something_very_important(); } -if ( thisVariable == thatVariable ) /*can you see me?*/ +if (this_variable == this_variable) /*can you see me?*/ { - DoSomethingVeryImportant(); /*not easily*/ + do_something_very_important(); /*not easily*/ } @@ -321,22 +835,22 @@ if ( thisVariable == thatVariable ) /*can you see me?*/ int urls_read = 0; /* # of urls read + rejected */ int urls_rejected = 0; /* # of urls rejected */ -if ( 1 == X ) +if (1 == X) { - DoSomethingVeryImportant(); + do_something_very_important(); } -short DoSomethingVeryImportant( +short do_something_very_important( short firstparam, /* represents something */ short nextparam /* represents something else */ ) { ...code here... -} /* -END- DoSomethingVeryImportant */ +} /* -END- do_something_very_important */ - + Comment each logical step @@ -354,9 +868,9 @@ short DoSomethingVeryImportant( comment. After all, these are usually major logic containers. - + - + Comment All Functions Thoroughly @@ -375,9 +889,9 @@ short DoSomethingVeryImportant( functions should contain the information presented in the addendum section of this document. - + - + Comment at the end of braces if the content is more than one screen length @@ -398,33 +912,33 @@ short DoSomethingVeryImportant( Example: -if ( 1 == X ) +if (1 == X) { - DoSomethingVeryImportant(); + do_something_very_important(); ...some long list of commands... } /* -END- if x is 1 */ or: -if ( 1 == X ) +if (1 == X) { - DoSomethingVeryImportant(); + do_something_very_important(); ...some long list of commands... -} /* -END- if ( 1 == X ) */ +} /* -END- if (1 == X) */ - + Naming Conventions - + Variable Names Explanation: - Use all lowercase, and seperate words via an underscore + Use all lowercase, and separate words via an underscore ('_'). Do not start an identifier with an underscore. (ANSI C reserves these for use by the compiler and system headers.) Do not use identifiers which are reserved in ANSI C++. (E.g. @@ -437,21 +951,19 @@ int ms_iis5_hack = 0; Instead of: - int msiis5hack = 0; int msIis5Hack = 0; - - - + + Function Names Explanation: - Use all lowercase, and seperate words via an underscore + Use all lowercase, and separate words via an underscore ('_'). Do not start an identifier with an underscore. (ANSI C reserves these for use by the compiler and system headers.) Do not use identifiers which are reserved in ANSI C++. (E.g. @@ -460,20 +972,18 @@ int msiis5hack = 0; int msIis5Hack = 0; Example: -int load_some_file( struct client_state *csp ) +int load_some_file(struct client_state *csp) Instead of: - -int loadsomefile( struct client_state *csp ) -int loadSomeFile( struct client_state *csp ) +int loadsomefile(struct client_state *csp) +int loadSomeFile(struct client_state *csp) - - + - + Header file prototypes @@ -485,20 +995,19 @@ int loadSomeFile( struct client_state *csp ) Example: -(.h) extern int load_aclfile( struct client_state *csp ); -(.c) int load_aclfile( struct client_state *csp ) +(.h) extern int load_aclfile(struct client_state *csp); +(.c) int load_aclfile(struct client_state *csp) - Instead of: + Instead of: -(.h) extern int load_aclfile( struct client_state * ); or -(.h) extern int load_aclfile(); -(.c) int load_aclfile( struct client_state *csp ) +(.h) extern int load_aclfile(struct client_state *); or +(.h) extern int load_aclfile(); +(.c) int load_aclfile(struct client_state *csp) - - + - + Enumerations, and #defines @@ -510,7 +1019,7 @@ int loadSomeFile( struct client_state *csp ) Example: -(enumeration) : enum Boolean { FALSE, TRUE }; +(enumeration) : enum Boolean {FALSE, TRUE}; (#define) : #define DEFAULT_SIZE 100; Note: We have a standard naming scheme for #defines @@ -526,7 +1035,7 @@ int loadSomeFile( struct client_state *csp ) #endif /* def FEATURE_FORCE */ - + Constants @@ -546,25 +1055,23 @@ int loadSomeFile( struct client_state *csp ) Instead of: - -#define USE_IMG_LST 1 or +#define USE_IMG_LST 1 or #define _USE_IMAGE_LIST 1 or -#define USE_IMAGE_LIST_ 1 or +#define USE_IMAGE_LIST_ 1 or #define use_image_list 1 or #define UseImageList 1 - - + - + Using Space - + Put braces on a line by themselves. @@ -578,39 +1085,39 @@ int loadSomeFile( struct client_state *csp ) Example: -if ( this == that ) +if (this == that) { ... } Instead of: - if ( this == that ) { ... } + if (this == that) { ... } or - if ( this == that ) { ... } + if (this == that) { ... } Note: In the special case that the if-statement is inside a loop, and it is trivial, i.e. it tests for a - condidtion that is obvious from the purpose of the block, + condition that is obvious from the purpose of the block, one-liners as above may optically preserve the loop structure and make it easier to read. - Status: developer-discrection. + Status: developer-discretion. Example exception: -while ( more lines are read ) +while (more lines are read) { /* Please document what is/is not a comment line here */ - if ( it's a comment ) continue; + if (it's a comment) continue; - do_something( line ); + do_something(line); } - + ALL control statements should have a block @@ -623,19 +1130,19 @@ while ( more lines are read ) Example: -if ( this == that ) +if (this == that) { - DoSomething(); - DoSomethingElse(); + do_something(); + do_something_else(); } Instead of: - if ( this == that ) DoSomething(); DoSomethingElse(); + if (this == that) do_something(); do_something_else(); or - if ( this == that ) DoSomething(); + if (this == that) do_something(); Note: The first example in "Instead of" will execute in a manner other than that which the developer desired (per @@ -643,30 +1150,30 @@ if ( this == that ) "feature". The "explanation" and "exception" from the point above also applies. - + - + Do not belabor/blow-up boolean expressions Example: -structure->flag = ( condition ); +structure->flag = (condition); Instead of: - if ( condition ) { structure->flag = 1; } else { + if (condition) { structure->flag = 1; } else { structure->flag = 0; } - Note: The former is readable and consice. The later + Note: The former is readable and concise. The later is wordy and inefficient. Please assume that any developer new to the project has at least a "good" knowledge of C/C++. (Hope I do not offend by that last comment ... 8-) - + - + Use white space freely because it is free @@ -678,17 +1185,13 @@ structure->flag = ( condition ); Example: -int firstValue = 0; -int someValue = 0; -int anotherValue = 0; -int thisVariable = 0; - -if ( thisVariable == thatVariable ) - -firstValue = oldValue + ( ( someValue - anotherValue ) - whatever ) +int first_value = 0; +int some_value = 0; +int another_value = 0; +int this_variable = 0; - + Don't use white space around structure operators @@ -705,16 +1208,16 @@ firstValue = oldValue + ( ( someValue - anotherValue ) - whatever ) Example: -aStruct->aMember; -aStruct.aMember; -FunctionName(); +a_struct->a_member; +a_struct.a_member; +function_name(); + + Instead of: a_struct -> a_member; a_struct . a_member; + function_name (); - Instead of: aStruct -> aMember; aStruct . aMember; - FunctionName (); - - + Make the last brace of a function stand out @@ -724,35 +1227,35 @@ FunctionName(); int function1( ... ) { ...code... - return( retCode ); + return(ret_code); -} /* -END- function1 */ +} /* -END- function1 */ int function2( ... ) { -} /* -END- function2 */ +} /* -END- function2 */ Instead of: - int function1( ... ) { ...code... return( retCode ); } int + int function1( ... ) { ...code... return(ret_code); } int function2( ... ) { } Note: Use 1 blank line before the closing brace and 2 - lines afterwards. This makes the end of function standout to + lines afterward. This makes the end of function standout to the most casual viewer. Although function comments help - seperate functions, this is still a good coding practice. In + separate functions, this is still a good coding practice. In fact, I follow these rules when using blocks in "for", "while", "do" loops, and long if {} statements too. After all whitespace is free! - Status: developer-discrection on the number of blank + Status: developer-discretion on the number of blank lines. Enforced is the end of function comments. - + - + Use 3 character indentions @@ -773,27 +1276,27 @@ static const char * const url_code_map[256] = int function1( ... ) { - if ( 1 ) + if (1) { - return( ALWAYS_TRUE ); + return ALWAYS_TRUE; } else { - return( HOW_DID_YOU_GET_HERE ); + return HOW_DID_YOU_GET_HERE; } - return( NEVER_GETS_HERE ); + return NEVER_GETS_HERE; } - + Initializing - + Initialize all variables @@ -806,25 +1309,25 @@ int function1( ... ) Example: -short anShort = 0; -float aFloat = 0; +short a_short = 0; +float a_float = 0; struct *ptr = NULL; Note: It is much easier to debug a SIGSEGV if the message says you are trying to access memory address 00000000 - and not 129FA012; or arrayPtr[20] causes a SIGSEV vs. - arrayPtr[0]. + and not 129FA012; or array_ptr[20] causes a SIGSEV vs. + array_ptr[0]. - Status: developer-discrection if and only if the + Status: developer-discretion if and only if the variable is assigned a value "shortly after" declaration. - + Functions - + Name functions that return a boolean as a question. @@ -836,12 +1339,12 @@ struct *ptr = NULL; Example: -ShouldWeBlockThis(); -ContainsAnImage(); -IsWebPageBlank(); +should_we_block_this(); +contains_an_image(); +is_web_page_blank(); - + Always specify a return type for a function. @@ -853,9 +1356,9 @@ IsWebPageBlank(); purpose, and create a void return type if the function does not need to return anything. - + - + Minimize function calls when iterating by using variables @@ -867,7 +1370,7 @@ IsWebPageBlank(); Example: -for ( size_t cnt = 0; cnt < blockListLength(); cnt ++ ) +for (size_t cnt = 0; cnt < block_list_length(); cnt++) { .... } @@ -876,10 +1379,10 @@ for ( size_t cnt = 0; cnt < blockListLength(); cnt ++ ) each and every iteration. This increases the overhead in the program, because the compiler has to look up the function each time, call it, and return a value. Depending on what occurs in - the blockListLength() call, it might even be creating and + the block_list_length() call, it might even be creating and destroying structures with each iteration, even though in each case it is comparing "cnt" to the same value, over and over. - Remember too - even a call to blockListLength() is a function + Remember too - even a call to block_list_length() is a function call, with the same overhead. Instead of using a function call during the iterations, @@ -888,20 +1391,20 @@ for ( size_t cnt = 0; cnt < blockListLength(); cnt ++ ) Example: -size_t len = blockListLength(); +size_t len = block_list_length(); -for ( size_t cnt = 0; cnt < len; cnt ++ ) +for (size_t cnt = 0; cnt < len; cnt++) { .... } - Exceptions: if the value of blockListLength() *may* - change or could *potentially* change, then you must code the + Exceptions: if the value of block_list_length() + *may* change or could *potentially* change, then you must code the function call in the for/while loop. - + - + Pass and Return by Const Reference @@ -910,19 +1413,19 @@ for ( size_t cnt = 0; cnt < len; cnt ++ ) This allows a developer to define a const pointer and call your function. If your function does not have the const keyword, we may not be able to use your function. Consider - strcmp, if it were defined as: extern int strcmp( char *s1, - char *s2 ); + strcmp, if it were defined as: extern int strcmp(char *s1, + char *s2); - I could then not use it to compare argv's in main: int main( - int argc, const char *argv[] ) { strcmp( argv[0], "privoxy" - ); } + I could then not use it to compare argv's in main: int + main(int argc, const char *argv[]) { strcmp(argv[0], "privoxy"); + } Both these pointers are *const*! If the c runtime library maintainers do it, we should too. - + - + Pass and Return by Value @@ -930,15 +1433,15 @@ for ( size_t cnt = 0; cnt < len; cnt ++ ) Most structures cannot fit onto a normal stack entry (i.e. they are not 4 bytes or less). Aka, a function declaration - like: int load_aclfile( struct client_state csp ) + like: int load_aclfile(struct client_state csp) would not work. So, to be consistent, we should declare all - prototypes with "pass by value": int load_aclfile( struct - client_state *csp ) + prototypes with "pass by value": int load_aclfile(struct + client_state *csp) + - - + Names of include files @@ -959,20 +1462,18 @@ for ( size_t cnt = 0; cnt < len; cnt ++ ) Exception: - -/* This is not a local include, but requires a path element. */ +/* This is not a local include, but requires a path element. */ #include <sys/fileName.h> - Note: Please! do not add "-I." to the Makefile without a _very_ good reason. This duplicates the #include - "file.h" behaviour. + "file.h" behavior. + - - + Provide multiple inclusion protection @@ -995,7 +1496,7 @@ for ( size_t cnt = 0; cnt < len; cnt ++ ) #endif /* ndef PROJECT_H_INCLUDED */ - + Use `extern "C"` when appropriate @@ -1019,7 +1520,7 @@ extern "C" #endif /* def __cplusplus */ - + Where Possible, Use Forward Struct Declaration Instead of Includes @@ -1041,17 +1542,17 @@ extern file_list *xyz; Note: If you declare "file_list xyz;" (without the pointer), then including the proper header file is necessary. If you only want to prototype a pointer, however, the header - file is unneccessary. + file is unnecessary. + + Status: Use with discretion. - Status: Use with discrection. - General Coding Practices - + Turn on warnings @@ -1061,9 +1562,9 @@ extern file_list *xyz; should turn on as many as possible. With GCC, the switch is "-Wall". Try and fix as many warnings as possible. - + - + Provide a default case for all switch statements @@ -1077,22 +1578,22 @@ extern file_list *xyz; Example: -switch( hash_string( cmd ) ) +switch (hash_string(cmd)) { - case hash_actions_file : + case hash_actions_file: ... code ... break; - case hash_confdir : + case hash_confdir: ... code ... break; - default : + default: log_error( ... ); - ... anomly code goes here ... + ... anomaly code goes here ... continue; / break; / exit( 1 ); / etc ... -} /* end switch( hash_string( cmd ) ) */ +} /* end switch (hash_string(cmd)) */ Note: If you already have a default condition, you are obviously exempt from this point. Of note, most of the @@ -1100,15 +1601,15 @@ switch( hash_string( cmd ) ) This API call *should* be included in a default statement. Another Note: This is not so much a readability issue - as a robust programming issue. The "anomly code goes here" may + as a robust programming issue. The "anomaly code goes here" may be no more than a print to the STDERR stream (as in - load_config). Or it may really be an ABEND condition. + load_config). Or it may really be an abort condition. Status: Programmer discretion is advised. - + - + Try to avoid falling through cases in a switch statement. @@ -1131,27 +1632,9 @@ switch( hash_string( cmd ) ) the fact of the fall through and reason why you felt it was necessary. - - - - - Use 'long' or 'short' Instead of - 'int' - - Explanation: - On 32-bit platforms, int usually has the range of long. On - 16-bit platforms, int has the range of short. - - Status: open-to-debate. In the case of most FSF - projects (including X/GNU-Emacs), there are typedefs to int4, - int8, int16, (or equivalence ... I forget the exact typedefs - now). Should we add these to IJB now that we have a "configure" - script? - - - + Don't mix size_t and other types @@ -1161,12 +1644,11 @@ switch( hash_string( cmd ) ) assumptions about whether it is signed or unsigned, or about how long it is. Do not compare a size_t against another variable of a different type (or even against a constant) - without casting one of the values. Try to avoid using size_t if - you can. + without casting one of the values. + - - + Declare each variable and struct on its own line. @@ -1194,20 +1676,20 @@ long c = 0; Exceptions: when you want to declare a bunch of loop variables or other trivial variables; feel free to declare them - on 1 line. You should, although, provide a good comment on + on one line. You should, although, provide a good comment on their functions. - Status: developer-discrection. + Status: developer-discretion. + - - + Use malloc/zalloc sparingly Explanation: - Create a local stuct (on the stack) if the variable will + Create a local struct (on the stack) if the variable will live and die within the context of one function call. Only "malloc" a struct (on the heap) if the variable's life @@ -1216,10 +1698,10 @@ long c = 0; Example: If a function creates a struct and stores a pointer to it in a -list, then it should definately be allocated via `malloc'. +list, then it should definitely be allocated via `malloc'. - + The Programmer Who Uses 'malloc' is Responsible for Ensuring 'free' @@ -1232,12 +1714,12 @@ list, then it should definately be allocated via `malloc'. responsible for ensuring that deletion is timely (i.e. not too soon, not too late). This is known as "low-coupling" and is a "good thing (tm)". You may need to offer a - free/unload/destuctor type function to accomodate this. + free/unload/destructor type function to accommodate this. Example: -int load_re_filterfile( struct client_state *csp ) { ... } -static void unload_re_filterfile( void *f ) { ... } +int load_re_filterfile(struct client_state *csp) { ... } +static void unload_re_filterfile(void *f) { ... } Exceptions: @@ -1245,13 +1727,13 @@ static void unload_re_filterfile( void *f ) { ... } functions for C run-time library functions ... such as `strdup'. - Status: developer-discrection. The "main" use of this + Status: developer-discretion. The "main" use of this standard is for allocating and freeing data structures (complex or nested). - + - + Add loaders to the `file_list' structure and in order @@ -1267,39 +1749,39 @@ static void unload_re_filterfile( void *f ) { ... } POPUPs can also be referred to as KILLPOPUPs, it is clear that it should come first. - + - + "Uncertain" new code and/or changes to - exitinst code, use FIXME + existing code, use XXX Explanation: If you have enough confidence in new code or confidence in - your changes, but are not *quite* sure of the reprocussions, + your changes, but are not *quite* sure of the repercussions, add this: - /* FIXME: this code has a logic error on platform XYZ, * - attempthing to fix */ #ifdef PLATFORM ...changed code here... + /* XXX: this code has a logic error on platform XYZ, * + attempting to fix */ #ifdef PLATFORM ...changed code here... #endif or: - /* FIXME: I think the original author really meant this... + /* XXX: I think the original author really meant this... */ ...changed code here... or: - /* FIXME: new code that *may* break something else... */ + /* XXX: new code that *may* break something else... */ ...new code here... Note: If you make it clear that this may or may not be a "good thing (tm)", it will be easier to identify and - include in the project (or conversly exclude from the + include in the project (or conversely exclude from the project). - + @@ -1309,19 +1791,14 @@ static void unload_re_filterfile( void *f ) { ... } Example for file comments: -const char FILENAME_rcs[] = "$Id: developer-manual.sgml,v 1.16 2002/03/31 23:04:40 hal9 Exp $"; /********************************************************************* * - * File : $Source$ + * File : $Source * * Purpose : (Fill me in with a good description!) * - * Copyright : Written by and Copyright (C) 2001 the SourceForge - * Privoxy team. http://www.privoxy.org/ - * - * Based on the Internet Junkbuster originally written - * by and Copyright (C) 1997 Anonymous Coders and - * Junkbusters Corporation. http://www.junkbusters.com + * Copyright : Written by and Copyright (C) 2001-2009 + * the Privoxy team. https://www.privoxy.org/ * * This program is free software; you can redistribute it * and/or modify it under the terms of the GNU General @@ -1337,12 +1814,10 @@ const char FILENAME_rcs[] = "$Id: developer-manual.sgml,v 1.16 2002/03/31 23:04: * * The GNU General Public License should be included with * this file. If not, you can view it at - * http://www.gnu.org/copyleft/gpl.html - * or write to the Free Software Foundation, Inc., 59 - * Temple Place - Suite 330, Boston, MA 02111-1307, USA. - * - * Revisions : - * $Log$ + * http://www.gnu.org/licenses/old-licenses/gpl-2.0.html + * or write to the Free Software Foundation, Inc., + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 , + * USA * *********************************************************************/ @@ -1355,13 +1830,13 @@ const char FILENAME_h_rcs[] = FILENAME_H_VERSION; Note: This declares the rcs variables that should be - added to the "show-proxy-args" page. If this is a brand new + added to the "show-version" page. If this is a brand new creation by you, you are free to change the "Copyright" section to represent the rights you wish to maintain. Note: The formfeed character that is present right after the comment flower box is handy for (X|GNU)Emacs users to - skip the verbige and get to the heart of the code (via + skip the verbiage and get to the heart of the code (via `forward-page' and `backward-page'). Please include it if you can. @@ -1369,19 +1844,14 @@ const char FILENAME_h_rcs[] = FILENAME_H_VERSION; #ifndef _FILENAME_H #define _FILENAME_H -#define FILENAME_H_VERSION "$Id: developer-manual.sgml,v 1.16 2002/03/31 23:04:40 hal9 Exp $" /********************************************************************* * - * File : $Source$ + * File : $Source * * Purpose : (Fill me in with a good description!) * - * Copyright : Written by and Copyright (C) 2001 the SourceForge - * Privoxy team. http://www.privoxy.org/ - * - * Based on the Internet Junkbuster originally written - * by and Copyright (C) 1997 Anonymous Coders and - * Junkbusters Corporation. http://www.junkbusters.com + * Copyright : Written by and Copyright (C) 2001-2009 + * the Privoxy team. https://www.privoxy.org/ * * This program is free software; you can redistribute it * and/or modify it under the terms of the GNU General @@ -1397,12 +1867,10 @@ const char FILENAME_h_rcs[] = FILENAME_H_VERSION; * * The GNU General Public License should be included with * this file. If not, you can view it at - * http://www.gnu.org/copyleft/gpl.html - * or write to the Free Software Foundation, Inc., 59 - * Temple Place - Suite 330, Boston, MA 02111-1307, USA. - * - * Revisions : - * $Log$ + * http://www.gnu.org/licenses/old-licenses/gpl-2.0.html + * or write to the Free Software Foundation, Inc., + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 , + * USA * *********************************************************************/ @@ -1449,10 +1917,10 @@ extern const char FILENAME_h_rcs[]; * Returns : 0 => Ok, everything else is an error. * *********************************************************************/ -int FUNCTION_NAME( void *param1, const char *x ) +int FUNCTION_NAME(void *param1, const char *x) { ... - return( 0 ); + return 0; } @@ -1465,13 +1933,6 @@ int FUNCTION_NAME( void *param1, const char *x ) - - Version Control Guidelines - To be filled. note on cvs comments. Don't only comment what you did, - but also why you did it! - - - Testing Guidelines To be filled. @@ -1481,13 +1942,14 @@ int FUNCTION_NAME( void *param1, const char *x ) Testplan for releases Explain release numbers. major, minor. developer releases. etc. - + Remove any existing rpm with rpm -e Remove any file that was left over. This includes (but is not limited to) + /var/log/privoxy /etc/privoxy @@ -1495,7 +1957,7 @@ Remove any file that was left over. This includes (but is not limited to) /etc/init.d/privoxy /usr/doc/privoxy* - + Install the rpm. Any error messages? @@ -1505,556 +1967,949 @@ Install the rpm. Any error messages? Start browsing. Does Privoxy work? Logfile written? Remove the rpm. Any error messages? All files removed? - + - Test reports - -Please submit test reports only with the test form -at sourceforge. Three simple steps: - - - Select category: the distribution you test on. - Select group: the version of Privoxy that we are about to release. - Fill the Summary and Detailed Description with something - intelligent (keep it short and precise). - - - Do not mail to the mailinglist (we cannot keep track on issues there). - + Fuzzing Privoxy + + To make fuzzing more convenient, Privoxy can be configured + with --enable-fuzz which will result in the --fuzz option + becoming available. + + + Example (tested on ElectroBSD): + + +# Compile Privoxy with instrumentation for afl +$ export CC=afl-clang +$ export CFLAGS="-fsanitize=address -ggdb" +$ export CPPFLAGS=-I/usr/local/include/ +$ export LDFLAGS="-fsanitize=address -L/usr/local/lib" +$ export AFL_USE_ASAN=1 +$ export AFL_HARDEN=1 +$ ./configure --with-debug --enable-extended-host-patterns --enable-accept-filter --enable-no-gifs --enable-compression --enable-strptime-sanity-checks --enable-external-filters --enable-fuzz + +$ ./privoxy --fuzz +Privoxy version 3.0.24 (http://www.privoxy.org/) +Usage: ./privoxy [--config-test] [--chroot] [--help] [--no-daemon] [--pidfile pidfile] [--pre-chroot-nslookup hostname] [--user user[.group]] [--version] [configfile] + ./privoxy --fuzz fuzz-mode ./path/to/fuzzed/input [--stfu] + +Supported fuzz modes and the expected input: + action: Text to parse as action file. + client-request: Client request to parse. Currently incomplete + client-header: Client header to parse. + chunked-transfer-encoding: Chunk-encoded data to dechunk. + deflate: deflate-compressed data to decompress. + filter: Text to parse as filter file. + gif: gif to deanimate. + gzip: gzip-compressed data to decompress. + pcrs-substitute: A pcrs-substitute to compile. Not a whole pcrs job! Example: Bla $1 bla C $3 blah. + server-header: Server header to parse. + server-response: Server response to parse. + +The following fuzz modes read data from stdin if the 'file' is '-' + client-request + client-header + chunked-transfer-encoding + deflate + gif + gzip + pcrs-substitute + server-header + server-response + +Aborting + +$ export ASAN_OPTIONS='abort_on_error=1' +$ mkdir input output +$ echo '$1 bla fasel $2' > input/pcrs +$ afl-fuzz -i input -o output -m none ~/git/privoxy/privoxy --fuzz pcrs-substitute - --stfu + +$ cat >input/pcrs.txt +FILTER: bla fasel +s@(.{1})[432](\d+)@$1$2$hostname@UgisT + +$ afl-fuzz -i input/ -o output/ -f bla.filter -m none privoxy --fuzz filter bla.filter --stfu + - - Releasing a new version + Releasing a New Version - To minimize trouble with distribution contents, webpage - errors and the like, we strongly encourage you - to follow this section if you prepare a new release of - code or new pages on the webserver. + When we release versions of Privoxy, + our work leaves our cozy secret lab and has to work in the cold + RealWorld[tm]. Once it is released, there is no way to call it + back, so it is very important that great care is taken to ensure + that everything runs fine, and not to introduce problems in the + very last minute. - The following programs are required to follow this process: - ncftpput (ncftp), scp (ssh), -gmake (GNU's version of make), autoconf, cvs, ???. + So when releasing a new version, please adhere exactly to the + procedure outlined in this chapter. - - - Before the Release - - The following must be done by one of the - developers prior to each new release: - - - - - + + + The following programs are required to follow this process: + ncftpput (ncftp), scp, ssh (ssh), + gmake (GNU's version of make), autoconf, cvs. + + + + Version numbers + + + First you need to determine which version number the release will have. + Privoxy version numbers consist of three numbers, + separated by dots, like in X.Y.Z (e.g. 3.0.0), where: + + + + + X, the version major, is rarely ever changed. It is increased by one if + turning a development branch into stable substantially changes the functionality, + user interface or configuration syntax. Majors 1 and 2 were + Junkbuster, and 3 will be the first stable + Privoxy release. + + + + + + Y, the version minor, represents the branch within the major version. + At any point in time, there are two branches being maintained: + The stable branch, with an even minor, say, 2N, in which no functionality is + being added and only bug-fixes are made, and 2N+1, the development branch, in + which the further development of Privoxy takes + place. + This enables us to turn the code upside down and inside out, while at the same time + providing and maintaining a stable version. + The minor is reset to zero (and one) when the major is incremented. When a development + branch has matured to the point where it can be turned into stable, the old stable branch + 2N is given up (i.e. no longer maintained), the former development branch 2N+1 becomes the + new stable branch 2N+2, and a new development branch 2N+3 is opened. + + + + + Z, the point or sub version, represents a release of the software within a branch. + It is therefore incremented immediately after each software release. + + The point version is reset to zero when the minor changes. + + + Stable branches work a little differently, since there should be + little to no development happening in such branches. Remember, + only bugfixes, which presumably should have had some testing + before being committed. Stable branches will then have their + version reported as 0.0.0, during that period + between releases when changes are being added. This is to denote + that this code is not for release. Then + as the release nears, the version is bumped according: e.g. + 3.0.1 -> 0.0.0 -> 3.0.2. + + + + + In summary, the main Git trunk is the development branch where new + features are being worked on for the next stable series. This should + almost always be where the most activity takes place. There is always at + least one stable branch from the trunk, e.g now it is + 3.0, which is only used to release stable versions. + Once the initial *.0 release of the stable branch has been done, then as a + rule, only bugfixes that have had prior testing should be committed to + the stable branch. Once there are enough bugfixes to justify a new + release, the version of this branch is again incremented Example: 3.0.0 + -> 3.0.1 -> 3.0.2, etc are all stable releases from within the stable + branch. 3.1.x is currently the main trunk, and where work on 3.2.x is + taking place. If any questions, please post to the devel list + before committing to a stable branch! + + + Developers should remember too that if they commit a bugfix to the stable + branch, this will more than likely require a separate submission to the + main trunk, since these are separate development trees within Git. If you + are working on both, then this would require at least two separate check + outs (i.e main trunk, and the stable release branch, + which is v_3_0_branch at the moment). + + + + + + Before the Release + + The following must be done by one of the + developers prior to each new release. + + + + Make sure that everybody who has worked on the code in the last couple of days has had a chance to yell no! in case - they have pending changes/fixes in their pipelines. - - + they have pending changes/fixes in their pipelines. Announce the + freeze so that nobody will interfere with last minute changes. + + - Increment the version number in configure.in in - CVS. Also, the RPM release number in - configure.in. Do NOT touch version information - after export from CVS. All packages will use the - version and release data from configure.in. - Local files should not be changed, except prior to a CVS commit!!! - This way we are all on the same page! + Update the code status (CODE_STATUS="xxx") in configure.in to one of + "alpha", "beta" or "stable". - + - If the default actionsfile has changed since last release, - bump up its version info in this line: + Rebuild configure and GNUMakefile to make sure the updated values are being used. + + + +$ autoheader && autoconf # rebuild configure +$ ./configure # rebuild GNUmakefile + + + + + make dok-release to update the sgml documentation source files. + + + + + If action file processing has changed and is not backward-compatible, + make sure the "for-privoxy-version=x.y.z" minimum version number in + default.action.master has been updated: - - {+add-header{X-Actions-File-Version: A.B} -filter -no-popups} - +{{settings}} +############################################################################# +#MASTER# COMMENT: The minimum Privoxy version: +for-privoxy-version=3.0.11 + + + + + Create the change log: + + + $ git tag + # to see the tags + $ git log [last release tag]..HEAD > /tmp/log + # get the commit log since the last release + $ utils/makeChangeLog /tmp/log > /tmp/change.log + # reformat the commit log + + + Edit /tmp/change.log to remove trivial + changes and group the changes under general headings like: - - Then change the version info in doc/webserver/actions/index.php, - line: '$required_actions_file_version = "A.B";' + +- Bug fixes: +- Action file improvements: +- Filter file improvements: +- General improvements: +- Documentation improvements: +- Build system improvements: +- Code cleanups: +- Privoxy-Log-Parser: +- Privoxy-Regression-Test: + + + Add the contents of /tmp/change.log to the + start of ChangeLog and re-create + doc/source/changelog.sgml: - + + $ utils/changelog2doc.pl /tmp/change.log >| doc/source/changelog.sgml + + - Tag all files in CVS with the version number with - cvs tag v_X_Y_Z (where X = major, Y - = minor, Z = point). Don't use vX_Y_Z, ver_X_Y_Z, v_X.Y.Z (won't work) - etc. + All developers should look at the ChangeLog and + make sure noteworthy changes are referenced. - + - The first package uploaded should be the official - tarball release. This is built with the - make tarball-dist Makefile - target, and then can be uploaded with - make tarball-upload (see below). + All documentation should be rebuilt: + + $ make man + $ make dok + $ make dok-man + $ make dok-tidy + $ make config-file + + Finished docs should be then be committed to Git (for those + without the ability to build these). Some docs may require + rather obscure processing tools. config, + the man page (and the html version of the man page) + fall in this category. README, the man page, AUTHORS, and config + should all also be committed to Git for other packagers. The + formal docs should be uploaded to the webserver. See the section + "Updating the webserver" + in this manual for details. - + + + + Commit all files that were changed in the above steps! + + + + + The User Manual is also used for context + sensitive help for the CGI editor. This is version sensitive, so that + the user will get appropriate help for his/her release. So with + each release a fresh version should be uploaded to the webserver + (this is in addition to the main User Manual + link from the main page since we need to keep manuals for various + versions available). The CGI pages will link to something like + http://privoxy.org/$(VERSION)/user-manual/. This + will need to be updated for each new release. There is no Makefile + target for this at this time!!! It needs to be done manually. + + + + + Tag all files in Git with the version number with + git tag v_X_Y_Z. + Don't use vX_Y_Z, ver_X_Y_Z, v_X.Y.Z (won't work) etc. + + + + + On the webserver, copy the user manual to a new top-level directory + called X.Y.Z. This ensures that help links from the CGI + pages, which have the version as a prefix, will go into the right version of the manual. + If this is a development branch release, also symlink X.Y.(Z-1) + to X.Y.Z and X.Y.(Z+1) to + . (i.e. dot). + + - - - Update the webserver - - All files must be group-readable and group-writable (or no one else - will be able to change them). To update the webserver, create any - pages locally in the doc/webserver directory (or - create new directories under doc/webserver), then do - - - - make webserver - - - - Note that make dok - (or make redhat-dok) creates - doc/webserver/user-manual, - doc/webserver/developer-manual, - doc/webserver/faq and - doc/webserver/man-page automatically. - + + + Building and Releasing the Packages + + Now the individual packages can be built and released. Note that for + GPL reasons the first package to be released is always the source tarball. + + + + For all types of packages, including the source tarball, + you must make sure that you build from clean sources by exporting + the right version from Git into an empty directory (just press return when + asked for a password): + + + + mkdir dist # delete or choose different name if it already exists + cd dist + git clone https://www.privoxy.org/git/privoxy.git + cd privoxy + git checkout v_X_Y_Z + + + + Do NOT change a single bit, including, but not limited to + version information after export from Git. This is to make sure that + all release packages, and with them, all future bug reports, are based + on exactly the same code. + + + + + Every significant release of Privoxy has included at least one + package that either had incorrect versions of files, missing files, + or incidental leftovers from a previous build process that gave + unknown numbers of users headaches to try to figure out what was + wrong. PLEASE, make sure you are using pristene sources, and are + following the prescribed process! + + + + + Please find additional instructions for the source tarball and the + individual platform dependent binary packages below. And details + on the Sourceforge release process below that. + + + + Note on Privoxy Packaging + + Please keep these general guidelines in mind when putting together + your package. These apply to all platforms! + + + + + Privoxy requires + write access to: all *.action files, all + logfiles, and the trust file. You will + need to determine the best way to do this for your platform. + + + + + Please include up to date documentation. At a bare minimum: + + + + LICENSE (top-level directory) + + + + + README (top-level directory) + + + + + AUTHORS (top-level directory) + + + + + man page (top-level directory, Unix-like + platforms only) + + + + + The User Manual (doc/webserver/user-manual/) + + + + + FAQ (doc/webserver/faq/) + + + + Also suggested: Developer Manual + (doc/webserver/developer-manual) and ChangeLog + (top-level directory). FAQ and the manuals are + HTML docs. There are also text versions in + doc/text/ which could conceivably also be + included. + + + The documentation has been designed such that the manuals are linked + to each other from parallel directories, and should be packaged + that way. privoxy-index.html can also be + included and can serve as a focal point for docs and other links of + interest (and possibly renamed to index.html). + This should be one level up from the manuals. There is a link also + on this page to an HTMLized version of the man page. To avoid 404 for + this, it is in Git as + doc/webserver/man-page/privoxy-man-page.html, + and should be included along with the manuals. There is also a + css stylesheets that can be included for better presentation: + p_doc.css. This should be in the same directory + with privoxy-index.html, (i.e. one level up from + the manual directories). + + + + + user.action and user.filter + are designed for local preferences. Make sure these do not get overwritten! + config should not be overwritten either. This + has especially important configuration data in it. + trust should be left in tact as well. + + + + + Other configuration files (default.action and + default.filter) should be installed as the new + defaults, but all previously installed configuration files should be + preserved as backups. This is just good manners :-) These files are + likely to change between releases and contain important new features + and bug fixes. + + + - Please do NOT use any other means of transferring files to the - webserver. make webserver not only - uploads, but will make sure that the appropriate permissions are - preserved for shared group access. + Please check platform specific notes in this doc, if you haven't + done Privoxy packaging before for other platform + specific issues. Conversely, please add any notes that you know + are important for your platform (or contact one of the doc + maintainers to do this if you can't). - + + + + Packagers should do a clean install of their + package after building it. So any previous installs should be + removed first to ensure the integrity of the newly built package. + Then run the package for a while to make sure there are no + obvious problems, before uploading. + + + + - SuSE or Red Hat + + + Source Tarball - Ensure that you have the latest code version. Hence run: - - - + First, make sure that you have freshly exported the right + version into an empty directory. (See "Building and releasing + packages" above). Then run: + + cd current - cvs -d:pserver:anonymous@cvs.ijbswa.sourceforge.net:/cvsroot/ijbswa login - cvs -z3 -d:pserver:anonymous@cvs.ijbswa.sourceforge.net:/cvsroot/ijbswa export -r v_X_Y_Z current - - - - first. - - - autoheader && autoconf && ./configure - - - - Then do - - - - make suse-dist or make redhat-dist - - - - To upload the package to Sourceforge, simply issue - - - - make suse-upload or make redhat-upload - - - - Go to the displayed URL and release the file publicly on Sourceforge. + + + Then do: - + + make tarball-dist + + + To upload the package to Sourceforge, simply issue + + + make tarball-upload + + + Go to the displayed URL and release the file publicly on Sourceforge. + For the change log field, use the relevant section of the + ChangeLog file. + + - OS/2 + SuSE, Conectiva or Red Hat RPM - Ensure that you have the latest code version. Hence run: - - - + In following text, replace dist + with either rh for Red Hat or suse for SuSE. + + + First, make sure that you have freshly exported the right + version into an empty directory. (See "Building and releasing + packages" above). + + + As the only exception to not changing anything after export from Git, + now examine the file privoxy-dist.spec + and make sure that the version information and the RPM release number are + correct. The RPM release numbers for each version start at one. Hence it must + be reset to one if this is the first RPM for + dist which is built from version + X.Y.Z. Check the + file + list if unsure. Else, it must be set to the highest already available RPM + release number for that version plus one. + + + Then run: + + cd current - cvs -d:pserver:anonymous@cvs.ijbswa.sourceforge.net:/cvsroot/ijbswa login - cvs -z3 -d:pserver:anonymous@cvs.ijbswa.sourceforge.net:/cvsroot/ijbswa export -r v_X_Y_Z current - - - - first. - - - autoheader && autoconf && ./configure - - - - Then do FIXME. - - + + + Then do + + + make dist-dist + + + To upload the package to Sourceforge, simply issue + + + make dist-upload rpm_packagerev + + + where rpm_packagerev is the + RPM release number as determined above. + Go to the displayed URL and release the file publicly on Sourceforge. + Use the release notes and change log from the source tarball package. + + - Solaris + Solaris - Login to Sourceforge's compilefarm via ssh - - - + Login to Sourceforge's compilefarm via ssh: + + ssh cf.sourceforge.net - - - - Choose the right operating system (not the Debian one). If you have - downloaded Privoxy before, - - - + + + Choose the right operating system (not the Debian one). + When logged in, make sure that you have freshly exported the right + version into an empty directory. (See "Building and releasing + packages" above). Then run: + + cd current - cvs -d:pserver:anonymous@cvs.ijbswa.sourceforge.net:/cvsroot/ijbswa login - cvs -z3 -d:pserver:anonymous@cvs.ijbswa.sourceforge.net:/cvsroot/ijbswa export -r v_X_Y_Z current - - - - If not, please checkout - Privoxy via CVS first. Run: - - - autoheader && autoconf && ./configure - - - - Then run - - - + + + Then run + + gmake solaris-dist - - - - which creates a gzip'ed tar archive. Sadly, you cannot use make - solaris-upload on the Sourceforge machine (no ncftpput). You now have - to manually upload the archive to Sourceforge's ftp server and release - the file publicly. - - - - Windows + - Ensure that you have the latest code version. Hence run - - - - cd current - cvs -d:pserver:anonymous@cvs.ijbswa.sourceforge.net:/cvsroot/ijbswa login - cvs -z3 -d:pserver:anonymous@cvs.ijbswa.sourceforge.net:/cvsroot/ijbswa export -r v_X_Y_Z current - - - - Run: - - - - autoheader && autoconf && ./configure - - - - Then do FIXME. - - - - Debian + which creates a gzip'ed tar archive. Sadly, you cannot use make + solaris-upload on the Sourceforge machine (no ncftpput). You now have + to manually upload the archive to Sourceforge's ftp server and release + the file publicly. Use the release notes and Change Log from the + source tarball package. + + + + Windows + - Ensure that you have the latest code version. Hence run: - - - - cd current - cvs -d:pserver:anonymous@cvs.ijbswa.sourceforge.net:/cvsroot/ijbswa login - cvs -z3 -d:pserver:anonymous@cvs.ijbswa.sourceforge.net:/cvsroot/ijbswa export -r v_X_Y_Z current - - - - first. Run: - - - - autoheader && autoconf && ./configure - - - - Then do FIXME. - - + Note that the docbook generated files might need some hand editing, + so the Windows build makefile does not rebuild the docs. + - Mac OSX - Login to Sourceforge's compilefarm via ssh - - - - ssh cf.sourceforge.net - - - - Choose the right operating system. If you have downloaded Privoxy - before, - - - - cd current - cvs -d:pserver:anonymous@cvs.ijbswa.sourceforge.net:/cvsroot/ijbswa login - cvs -z3 -d:pserver:anonymous@cvs.ijbswa.sourceforge.net:/cvsroot/ijbswa export -r v_X_Y_Z current - - - - If not, please checkout - Privoxy via CVS first. Run: - - - - autoheader && autoconf && ./configure - - - - Then run: - - - - make macosx-dist - - - - which creates a gzip'ed tar archive. Sadly, you cannot use make - macosx-upload on the Sourceforge machine (no ncftpput). You now have - to manually upload the archive to Sourceforge's ftp server and release - the file publicly. - - - - FreeBSD + First, make sure that you have freshly exported the right + version into an empty directory. (See "Building and releasing + packages" above). + + - Change the version number of Privoxy in the - configure.in file. Run: - - autoheader && autoconf && ./configure - - Then ... + Then you can build the package. This is fully automated, and is + controlled by windows/GNUmakefile. + All you need to do is: + + cd windows + make + - Login to Sourceforge's compilefarm via ssh: - - - - ssh cf.sourceforge.net - - - - Choose the right operating system. If you have downloaded Privoxy - before, - - - - cd current - cvs -d:pserver:anonymous@cvs.ijbswa.sourceforge.net:/cvsroot/ijbswa login - cvs -z3 -d:pserver:anonymous@cvs.ijbswa.sourceforge.net:/cvsroot/ijbswa export -r v_X_Y_Z current - - - - If not, please checkout - Privoxy via CVS first. Run: - - - - autoheader && autoconf && ./configure - - - - Then run: - - - - gmake freebsd-dist - - - - which creates a gzip'ed tar archive. Sadly, you cannot use make - freebsd-upload on the Sourceforge machine (no ncftpput). You now have - to manually upload the archive to Sourceforge's ftp server and release - the file publicly. - - + Now you can manually rename privoxy_setup.exe to + privoxy_setup_X.Y.Z.exe, and the build + directory to privoxy_X.Y.Z. + Create a .zip file of the newly renamed privoxy_X.Y.Z directory, + GPG sign the installer and zip file, + + + $ gpg --armor --detach --sign privoxy_setup_X.Y.Z.exe + $ gpg --armor --detach --sign privoxy_X.Y.Z.zip + + + and upload the files to SourceForge. + - Tarball - Ensure that you have the latest code version. Hence run: - - - - cd current - cvs -d:pserver:anonymous@cvs.ijbswa.sourceforge.net:/cvsroot/ijbswa login - cvs -z3 -d:pserver:anonymous@cvs.ijbswa.sourceforge.net:/cvsroot/ijbswa export -r v_X_Y_Z current - - - - first. Run: - - - - make clobber - autoheader && autoconf && ./configure - - - - Then do: - - - - make tarball-dist - - - - To upload the package to Sourceforge, simply issue - - - - make tarball-upload - - - - Goto the displayed URL and release the file publicly on Sourceforge. + When releasing the package on SourceForge, use the release notes + and Change Log from the source tarball package. - + - HP-UX 11 + Debian - Ensure that you have the latest code version. Hence run: - - - - cd current - cvs -d:pserver:anonymous@cvs.ijbswa.sourceforge.net:/cvsroot/ijbswa login - cvs -z3 -d:pserver:anonymous@cvs.ijbswa.sourceforge.net:/cvsroot/ijbswa export -r v_X_Y_Z current - - - - first. Run: - - - - autoheader && autoconf && ./configure - - - - Then do FIXME. - - + First, make sure that you have freshly exported the + right version into an empty directory. (See + "Building and releasing packages" above). Then add a log + entry to debian/changelog, if it is not + already there, for example by running: + + + debchange -v &p-version;-&p-status;-1 "New upstream version" + + + Then, run: + + + dpkg-buildpackage -rfakeroot -us -uc -b + + + This will create + ../privoxy_&p-version;-&p-status;-1_i386.deb + which can be uploaded. To upload the package to Sourceforge, simply + issue + + + make debian-upload + + - Amiga OS + Mac OS X - Ensure that you have the latest code version. Hence run: - - - - cd current - cvs -d:pserver:anonymous@cvs.ijbswa.sourceforge.net:/cvsroot/ijbswa login - cvs -z3 -d:pserver:anonymous@cvs.ijbswa.sourceforge.net:/cvsroot/ijbswa export -r v_X_Y_Z current - - - - first. Run: - - - - autoheader && autoconf && ./configure - - - - Then do FIXME. - - + First, make sure that you have freshly exported the right + version into an empty directory. (See "Building and releasing + packages" above). + + + There are three modules available in the Git repository for use on Mac + OS X, though technically only two of them generate a release (the other + can be used to install from source). + + + OSXPackageBuilder module + + The OSXPackageBuilder module generates OS X installer packages + supporting all Macs running OS X 10.4 and above. Obtain it from Git as + follows into a folder parallel to the exported privoxy source: + + + cvs -z3 -d:pserver:anonymous@ijbswa.cvs.sourceforge.net:/cvsroot/ijbswa co OSXPackageBuilder + + + The module contains complete instructions on its usage in the file + OS X Package Builder HOWTO.txt. + + + Once the package(s) have been generated, you can then upload them + directly to the Files section of the Sourceforge project in the + Macintosh (OS X) folder. Each new version release of Privoxy should + have a new subfolder created in which to store its files. Please + ensure that the folder contains a readme file that makes it clear + which package is for whichversion of OS X. + + + + osxsetup module (DEPRECATED) + + This module is deprecated since the installer it generates + places all Privoxy files in one folder in a non-standard location, and + supports only Intel Macs running OS X 10.6 or higher. + + + Check out the module from Git as follows into a folder parallel to the + exported privoxy source: + + + cvs -z3 -d:pserver:anonymous@ijbswa.cvs.sourceforge.net:/cvsroot/ijbswa co osxsetup + + + Then run: + + + cd osxsetup + build + + + This will run autoheader, autoconf + and configure as well as make. + Finally, it will copy over the necessary files to the ./osxsetup/files + directory for further processing by PackageMaker. + + + Bring up PackageMaker with the PrivoxyPackage.pmsp definition file, + modify the package name to match the release, and hit the "Create + package" button. If you specify ./Privoxy.pkg as the output package + name, you can then create the distributable zip file with the command: + + + zip -r privoxyosx_setup_x.y.z.zip Privoxy.pkg + + + You can then upload this file directly to the Files section of the + Sourceforge project in the Macintosh (OS X) folder. Each new version + release of Privoxy should have a new subfolder created in which to + store its files. + Please ensure that the folder contains a readme file that makes it + clear which version(s) of OS X the package supports. + + + + macsetup module + + The macsetup module is ideal if you wish to build and install Privoxy + from source on a single machine. + + + Check out the module from Git as follows into a folder parallel to the + exported privoxy source: + + + cvs -z3 -d:pserver:anonymous@ijbswa.cvs.sourceforge.net:/cvsroot/ijbswa co macsetup + + + The module contains complete instructions on its usage in its + README file. The end result will be the + exported version of Privoxy installed on the build machine. + + + - AIX + FreeBSD - Login to Sourceforge's compilefarm via ssh: - - - - ssh cf.sourceforge.net - - - - Choose the right operating system. If you have downloaded Privoxy - before: - - - - cd current - cvs -d:pserver:anonymous@cvs.ijbswa.sourceforge.net:/cvsroot/ijbswa login - cvs -z3 -d:pserver:anonymous@cvs.ijbswa.sourceforge.net:/cvsroot/ijbswa export -r v_X_Y_Z current - - - - If not, please checkout - Privoxy via CVS first. Run: - - - - autoheader && autoconf && ./configure - - - - Then run: - - - - make aix-dist - - - - which creates a gzip'ed tar archive. Sadly, you cannot use make - aix-upload on the Sourceforge machine (no ncftpput). You now have - to manually upload the archive to Sourceforge's ftp server and release - the file publicly. - - + Update the www/privoxy port and submit a diff upstream. + For details see the FreeBSD Porter's Handbook. + + + - - - - Contact the developers + + Uploading and Releasing Your Package - Please see the contact page in the user-manual for details. + After the package is ready, it is time to upload it + to SourceForge, and go through the release steps. The upload + is done via FTP: - - - - Copyright and History + + + + Upload to: ftp://upload.sourceforge.net/incoming + + + + + user: anonymous + + + + + password: ijbswa-developers@lists.sourceforge.net + + + - Please see the user-manual for - information on Copyright and History. - - - - - See also + Or use the make targets as described above. + + + Once this done go to + + https://sourceforge.net/project/admin/editpackages.php?group_id=11118, + making sure you are logged in. Find your target platform in the + second column, and click Add Release. You will + then need to create a new release for your package, using the format + of $VERSION ($CODE_STATUS), e.g. &p-version; + (beta). + - Please see the user-manual for others - references. + Now just follow the prompts. Be sure to add any appropriate Release + notes. You should see your freshly uploaded packages in + Step 2. Add Files To This Release. Check the + appropriate box(es). Remember at each step to hit the + Refresh/Submit buttons! You should now see your + file(s) listed in Step 3. Fill out the forms with the appropriate + information for your platform, being sure to hit Update + for each file. If anyone is monitoring your platform, check the + email box at the very bottom to notify them of + the new package. This should do it! + + If you have made errors, or need to make changes, you can go through + essentially the same steps, but select Edit Release, + instead of Add Release. + + + + + After the Release + + When all (or: most of the) packages have been uploaded and made available, + send an email to the + announce mailing + list, Subject: "Version X.Y.Z available for download". Be sure to + include the + + download location, the release notes and the Changelog. Also, post an + updated News item on the project page Sourceforge, and update the Home + page and docs linked from the Home page (see below). Other news sites + and release oriented sites, such as Freshmeat, should also be notified. + + + Then update the source code for the next version to be released: + + + + + Increment the version number and change the code status to "UNRELEASED" + in configure.in + + + + + Rebuild configure (autoheader && autoconf) + and GNUMakefile (./configure) + + + + + make dok-release to update the sgml documentation source files. + + + + + Commit all your changes. + + + + + + + + + + Update the Webserver + + The webserver should be updated at least with each stable release. When + updating, please follow these steps to make sure that no broken links, + inconsistent contents or permission problems will occur (as it has many + times in the past!): + + + If you have changed anything in the stable-branch documentation source + SGML files, do: + + + make dok + + + That will generate doc/webserver/user-manual, + doc/webserver/developer-manual, + doc/webserver/faq, + doc/webserver/index.html automatically. + + + If you changed the manual page sources, generate + doc/webserver/man-page/privoxy-man-page.html + by running make man. (This is + a separate target due to dependencies on some obscure perl scripts + [now in Git, but not well tested]. See comments in GNUmakefile.) + + + If you want to add new files to the webserver, create them locally in + the doc/webserver/* directory (or + create new directories under doc/webserver). + + + Next, commit any changes from the above steps to Git. All set? + If these are docs in the stable branch, then do: + + + make webserver + + + This will do the upload to the + webserver (www.privoxy.org) and ensure all files and directories + there are group writable. + + + Please do NOT use any other means of transferring + files to the webserver to avoid permission problems. Also, please do not + upload docs from development branches or versions. The publicly posted + docs should be in sync with the last official release. +