Main Page   Alphabetical List   Data Structures   File List   Data Fields   Globals  

parsers.c File Reference

Declares functions to parse/crunch headers and pages. More...


Defines

#define ijb_isupper(__X)   isupper((int)(unsigned char)(__X))
#define ijb_tolower(__X)   tolower((int)(unsigned char)(__X))

Functions

int flush_socket (jb_socket fd, struct client_state *csp)
jb_err add_to_iob (struct client_state *csp, char *buf, int n)
char * get_header (struct client_state *csp)
char * get_header_value (const struct list *header_list, const char *header_name)
char * sed (const struct parsers pats[], const add_header_func_ptr more_headers[], struct client_state *csp)
jb_err crumble (struct client_state *csp, char **header)
jb_err server_content_type (struct client_state *csp, char **header)
jb_err server_transfer_coding (struct client_state *csp, char **header)
jb_err server_content_encoding (struct client_state *csp, char **header)
jb_err server_content_length (struct client_state *csp, char **header)
jb_err server_content_md5 (struct client_state *csp, char **header)
jb_err client_accept_encoding (struct client_state *csp, char **header)
jb_err client_te (struct client_state *csp, char **header)
jb_err client_referrer (struct client_state *csp, char **header)
jb_err client_uagent (struct client_state *csp, char **header)
jb_err client_ua (struct client_state *csp, char **header)
jb_err client_from (struct client_state *csp, char **header)
jb_err client_send_cookie (struct client_state *csp, char **header)
jb_err client_x_forwarded (struct client_state *csp, char **header)
jb_err client_host_adder (struct client_state *csp)
jb_err client_cookie_adder (struct client_state *csp)
jb_err client_accept_encoding_adder (struct client_state *csp)
jb_err client_xtra_adder (struct client_state *csp)
jb_err client_x_forwarded_adder (struct client_state *csp)
jb_err connection_close_adder (struct client_state *csp)
jb_err server_http (struct client_state *csp, char **header)
jb_err server_set_cookie (struct client_state *csp, char **header)

Variables

const char parsers_rcs [] = "$Id: parsers.c,v 2.0 2002/06/04 14:34:21 jongfoster Exp $"
const char parsers_h_rcs [] = PARSERS_H_VERSION
const struct parsers client_patterns []
const struct parsers server_patterns []
const add_header_func_ptr add_client_headers []
const add_header_func_ptr add_server_headers []


Detailed Description

Declares functions to parse/crunch headers and pages.

Functions declared include: `add_to_iob', `client_cookie_adder', `client_from', `client_referrer', `client_send_cookie', `client_ua', `client_uagent', `client_x_forwarded', `client_x_forwarded_adder', `client_xtra_adder', `content_type', `crumble', `destroy_list', `enlist', `flush_socket', ``get_header', `sed', and `server_set_cookie'.

Log:
parsers.c,v
Revision 2.0 2002/06/04 14:34:21 jongfoster Moving source files to src/

Revision 1.56 2002/05/12 15:34:22 jongfoster Fixing typo in a comment

Revision 1.55 2002/05/08 16:01:07 oes Optimized add_to_iob:

Revision 1.54 2002/04/02 15:03:16 oes Tiny code cosmetics

Revision 1.53 2002/03/26 22:29:55 swa we have a new homepage!

Revision 1.52 2002/03/24 13:25:43 swa name change related issues

Revision 1.51 2002/03/13 00:27:05 jongfoster Killing warnings

Revision 1.50 2002/03/12 01:45:35 oes More verbose logging

Revision 1.49 2002/03/09 20:03:52 jongfoster

THIS WAS A MAJOR BUG - it caused frequent, unpredictable crashes, and also frequently caused JB to jump to 100% CPU and stay there. (Because it thought it had just read ((unsigned)-1) == 4Gb of data...)

Revision 1.48 2002/03/07 03:46:53 oes Fixed compiler warnings etc

Revision 1.47 2002/02/20 23:15:13 jongfoster Parsing functions now handle out-of-memory gracefully by returning an error code.

Revision 1.46 2002/01/17 21:03:47 jongfoster Moving all our URL and URL pattern parsing code to urlmatch.c.

Revision 1.45 2002/01/09 14:33:03 oes Added support for localtime_r.

Revision 1.44 2001/12/14 01:22:54 steudten Remove 'user:pass@' from 'proto://user:pass@host' for the new added header 'Host: ..'. (See Req ID 491818)

Revision 1.43 2001/11/23 00:26:38 jongfoster Fixing two really stupid errors in my previous commit

Revision 1.42 2001/11/22 21:59:30 jongfoster Adding code to handle +no-cookies-keep

Revision 1.41 2001/11/05 23:43:05 steudten Add time+date to log files.

Revision 1.40 2001/10/26 20:13:09 jongfoster ctype.h is needed in Windows, too.

Revision 1.39 2001/10/26 17:40:04 oes Introduced get_header_value() Removed http->user_agent, csp->referrer and csp->accept_types Removed client_accept()

Revision 1.38 2001/10/25 03:40:48 david__schmidt Change in porting tactics: OS/2's EMX porting layer doesn't allow multiple threads to call select() simultaneously. So, it's time to do a real, live, native OS/2 port. See defines for __EMX__ (the porting layer) vs. __OS2__ (native). Both versions will work, but using __OS2__ offers multi-threading.

Revision 1.37 2001/10/23 21:36:02 jongfoster Documenting sed()'s error behaviou (doc change only)

Revision 1.36 2001/10/13 12:51:51 joergs Removed client_host, (was only required for the old 2.0.2-11 http://noijb. force-load), instead crumble Host: and add it (again) in client_host_adder (in case we get a HTTP/1.0 request without Host: header and forward it to a HTTP/1.1 server/proxy).

Revision 1.35 2001/10/09 22:39:21 jongfoster assert.h is also required under Win32, so moving out of ifndef _WIN32 block.

Revision 1.34 2001/10/07 18:50:55 oes Added server_content_encoding, renamed server_transfer_encoding

Revision 1.33 2001/10/07 18:04:49 oes Changed server_http11 to server_http and its pattern to "HTTP". Additional functionality: it now saves the HTTP status into csp->http->status and sets CT_TABOO for Status 206 (partial range)

Revision 1.32 2001/10/07 15:43:28 oes Removed FEATURE_DENY_GZIP and replaced it with client_accept_encoding, client_te and client_accept_encoding_adder, triggered by the new +no-compression action. For HTTP/1.1 the Accept-Encoding header is changed to allow only identity and chunked, and the TE header is crunched. For HTTP/1.0, Accept-Encoding is crunched.

parse_http_request no longer does anything than parsing. The rewriting of http->cmd and version mangling are gone. It now also recognizes the put and delete methods and saves the url in http->url. Removed unused variable.

renamed content_type and content_length to have the server_ prefix

server_content_type now only works if csp->content_type != CT_TABOO

added server_transfer_encoding, which

added server_content_md5 which crunches any Content-MD5 headers if the body was modified.

made server_http11 conditional on +downgrade action

Replaced 6 boolean members of csp with one bitmap (csp->flags)

Revision 1.31 2001/10/05 14:25:02 oes Crumble Keep-Alive from Server

Revision 1.30 2001/09/29 12:56:03 joergs IJB now changes HTTP/1.1 to HTTP/1.0 in requests and answers.

Revision 1.29 2001/09/24 21:09:24 jongfoster Fixing 2 memory leaks that Guy spotted, where the paramater to enlist() was not being free()d.

Revision 1.28 2001/09/22 16:32:28 jongfoster Removing unused includes.

Revision 1.27 2001/09/20 15:45:25 steudten

add casting from size_t to int for printf() remove local variable shadow s2

Revision 1.26 2001/09/16 17:05:14 jongfoster Removing unused include showarg.h

Revision 1.25 2001/09/16 13:21:27 jongfoster Changes to use new list functions.

Revision 1.24 2001/09/13 23:05:50 jongfoster Changing the string paramater to the header parsers a "const".

Revision 1.23 2001/09/12 18:08:19 steudten

In parse_http_request() header rewriting miss the host value, so from http://www.mydomain.com the result was just " / " not http://www.mydomain.com/ in case we forward.

Revision 1.22 2001/09/10 10:58:53 oes Silenced compiler warnings

Revision 1.21 2001/07/31 14:46:00 oes

Revision 1.20 2001/07/30 22:08:36 jongfoster Tidying up defines: Revision 1.19 2001/07/25 17:21:54 oes client_uagent now saves copy of User-Agent: header value

Revision 1.18 2001/07/13 14:02:46 oes

Revision 1.17 2001/06/29 21:45:41 oes Indentation, CRLF->LF, Tab-> Space

Revision 1.16 2001/06/29 13:32:42 oes

Revision 1.15 2001/06/03 19:12:38 oes deleted const struct interceptors

Revision 1.14 2001/06/01 18:49:17 jongfoster Replaced "list_share" with "list" - the tiny memory gain was not worth the extra complexity.

Revision 1.13 2001/05/31 21:30:33 jongfoster Removed list code - it's now in list.[ch] Renamed "permission" to "action", and changed many features to use the actions file rather than the global config.

Revision 1.12 2001/05/31 17:33:13 oes

CRLF -> LF

Revision 1.11 2001/05/29 20:11:19 joergs '/ * inside comment' warning removed.

Revision 1.10 2001/05/29 09:50:24 jongfoster Unified blocklist/imagelist/permissionslist. File format is still under discussion, but the internal changes are (mostly) done.

Also modified interceptor behaviour:

Revision 1.9 2001/05/28 17:26:33 jongfoster Fixing segfault if last header was crunched. Fixing Windows build (snprintf() is _snprintf() under Win32, but we can use the cross-platform sprintf() instead.)

Revision 1.8 2001/05/27 22:17:04 oes

Revision 1.7 2001/05/27 13:19:06 oes Patched Joergs solution for the content-length in.

Revision 1.6 2001/05/26 13:39:32 jongfoster Only crunches Content-Length header if applying RE filtering. Without this fix, Microsoft Windows Update wouldn't work.

Revision 1.5 2001/05/26 00:28:36 jongfoster Automatic reloading of config file. Removed obsolete SIGHUP support (Unix) and Reload menu option (Win32). Most of the global variables have been moved to a new struct configuration_spec, accessed through csp->config->globalname Most of the globals remaining are used by the Win32 GUI.

Revision 1.4 2001/05/22 18:46:04 oes

Revision 1.3 2001/05/20 01:21:20 jongfoster Version 2.9.4 checkin. Revision 1.2 2001/05/17 23:02:36 oes Revision 1.1.1.1 2001/05/15 13:59:01 oes Initial import of version 2.9.3 source tree


Function Documentation

jb_err add_to_iob struct client_state   csp,
char *    buf,
int    n
 

Add content to the buffered page, expanding the buffer if necessary.

Parameters:
csp  Current client state (buffers, headers, etc...)
buf  holds the content to be added to the page
n  number of bytes to be added
Returns:
JB_ERR_OK on success, JB_ERR_MEMORY if out-of-memory or buffer limit reached.

jb_err client_accept_encoding struct client_state   csp,
char **    header
 

Rewrite the client's Accept-Encoding header so that if doesn't allow compression, if the action applies.

Note For HTTP/1.0 the absence of the header is enough.

Parameters:
csp  Current client state (buffers, headers, etc...)
header  On input, pointer to header to modify. On output, pointer to the modified header, or NULL to remove the header. This function frees the original string if necessary.
Returns:
JB_ERR_OK on success, or JB_ERR_MEMORY on out-of-memory error.

jb_err client_accept_encoding_adder struct client_state   csp
 

Add an Accept-Encoding header to the client's request that disables compression if the action applies, and the header is not already there.

Called from `sed'.

Note For HTTP/1.0, the absence of the header is enough.

Parameters:
csp  Current client state (buffers, headers, etc...)
Returns:
JB_ERR_OK on success, or JB_ERR_MEMORY on out-of-memory error.

jb_err client_cookie_adder struct client_state   csp
 

Used in the add_client_headers list.

Called from `sed'.

Parameters:
csp  Current client state (buffers, headers, etc...)
Returns:
JB_ERR_OK on success, or JB_ERR_MEMORY on out-of-memory error.

jb_err client_from struct client_state   csp,
char **    header
 

Handle the "from" config setting properly.

Called from `sed'.

Parameters:
csp  Current client state (buffers, headers, etc...)
header  On input, pointer to header to modify. On output, pointer to the modified header, or NULL to remove the header. This function frees the original string if necessary.
Returns:
JB_ERR_OK on success, or JB_ERR_MEMORY on out-of-memory error.

jb_err client_host_adder struct client_state   csp
 

(re)adds the host header.

Called from `sed'.

Parameters:
csp  Current client state (buffers, headers, etc...)
Returns:
JB_ERR_OK on success, or JB_ERR_MEMORY on out-of-memory error.

jb_err client_referrer struct client_state   csp,
char **    header
 

Handle the "referer" config setting properly.

Called from `sed'.

Parameters:
csp  Current client state (buffers, headers, etc...)
header  On input, pointer to header to modify. On output, pointer to the modified header, or NULL to remove the header. This function frees the original string if necessary.
Returns:
JB_ERR_OK on success, or JB_ERR_MEMORY on out-of-memory error.

jb_err client_send_cookie struct client_state   csp,
char **    header
 

Handle the "cookie" header properly.

Called from `sed'. If cookie is accepted, add it to the cookie_list, else we crunch it. Mmmmmmmmmmm ... cookie ......

Parameters:
csp  Current client state (buffers, headers, etc...)
header  On input, pointer to header to modify. On output, pointer to the modified header, or NULL to remove the header. This function frees the original string if necessary.
Returns:
JB_ERR_OK on success, or JB_ERR_MEMORY on out-of-memory error.

jb_err client_te struct client_state   csp,
char **    header
 

Rewrite the client's TE header so that if doesn't allow compression, if the action applies.

Parameters:
csp  Current client state (buffers, headers, etc...)
header  On input, pointer to header to modify. On output, pointer to the modified header, or NULL to remove the header. This function frees the original string if necessary.
Returns:
JB_ERR_OK on success, or JB_ERR_MEMORY on out-of-memory error.

jb_err client_ua struct client_state   csp,
char **    header
 

Handle "ua-" headers properly.

Called from `sed'.

Parameters:
csp  Current client state (buffers, headers, etc...)
header  On input, pointer to header to modify. On output, pointer to the modified header, or NULL to remove the header. This function frees the original string if necessary.
Returns:
JB_ERR_OK on success, or JB_ERR_MEMORY on out-of-memory error.

jb_err client_uagent struct client_state   csp,
char **    header
 

Handle the "user-agent" config setting properly and remember its original value to enable browser bug workarounds.

Called from `sed'.

Parameters:
csp  Current client state (buffers, headers, etc...)
header  On input, pointer to header to modify. On output, pointer to the modified header, or NULL to remove the header. This function frees the original string if necessary.
Returns:
JB_ERR_OK on success, or JB_ERR_MEMORY on out-of-memory error.

jb_err client_x_forwarded struct client_state   csp,
char **    header
 

Handle the "x-forwarded-for" config setting properly, also used in the add_client_headers list.

Called from `sed'.

Parameters:
csp  Current client state (buffers, headers, etc...)
header  On input, pointer to header to modify. On output, pointer to the modified header, or NULL to remove the header. This function frees the original string if necessary.
Returns:
JB_ERR_OK on success, or JB_ERR_MEMORY on out-of-memory error.

jb_err client_x_forwarded_adder struct client_state   csp
 

Used in the add_client_headers list.

Called from `sed'.

Parameters:
csp  Current client state (buffers, headers, etc...)
Returns:
JB_ERR_OK on success, or JB_ERR_MEMORY on out-of-memory error.

jb_err client_xtra_adder struct client_state   csp
 

Used in the add_client_headers list.

Called from `sed'.

Parameters:
csp  Current client state (buffers, headers, etc...)
Returns:
JB_ERR_OK on success, or JB_ERR_MEMORY on out-of-memory error.

jb_err connection_close_adder struct client_state   csp
 

Adds a "Connection: close" header to csp->headers as a temporary fix for the needed but missing HTTP/1.1 support.

Called from `sed'.

FIXME This whole function shouldn't be neccessary!

Parameters:
csp  Current client state (buffers, headers, etc...)
Returns:
JB_ERR_OK on success, or JB_ERR_MEMORY on out-of-memory error.

jb_err crumble struct client_state   csp,
char **    header
 

This is called if a header matches a pattern to "crunch".

Parameters:
csp  Current client state (buffers, headers, etc...)
header  On input, pointer to header to modify. On output, pointer to the modified header, or NULL to remove the header. This function frees the original string if necessary.
Returns:
JB_ERR_OK on success, or JB_ERR_MEMORY on out-of-memory error.

int flush_socket jb_socket    fd,
struct client_state   csp
 

Write any pending "buffered" content.

Parameters:
fd  file descriptor of the socket to read
csp  Current client state (buffers, headers, etc...)
Returns:
On success, the number of bytes written are returned (zero indicates nothing was written). On error, -1 is returned, and errno is set appropriately. If count is zero and the file descriptor refers to a regular file, 0 will be returned without causing any other effect. For a special file, the results are not portable.

char* get_header struct client_state   csp
 

This (odd) routine will parse the csp->iob.

Parameters:
csp  Current client state (buffers, headers, etc...)
Returns:
Any one of the following: 1) a pointer to a dynamically allocated string that contains a header line 2) NULL indicating that the end of the header was reached 3) "" indicating that the end of the iob was reached before finding a complete header line.

char* get_header_value const struct list   header_list,
const char *    header_name
 

Get the value of a given header from a chained list of header lines or return NULL if no such header is present in the list.

Parameters:
header_list  pointer to list
header_name  string with name of header to look for. Trailing colon required, capitalization doesn't matter.
Returns:
NULL if not found, else value of header

char* sed const struct parsers    pats[],
const add_header_func_ptr    more_headers[],
struct client_state   csp
 

add, delete or modify lines in the HTTP header streams.

On entry, it receives a linked list of headers space that was allocated dynamically (both the list nodes and the header contents). As a side effect it frees the space used by the original header lines.

Parameters:
pats  list of patterns to match against headers
more_headers  list of functions to add more headers (client or server)
csp  Current client state (buffers, headers, etc...)
Returns:
Single pointer to a fully formed header, or NULL on out-of-memory error.

jb_err server_content_encoding struct client_state   csp,
char **    header
 

Prohibit filtering (CT_TABOO) if content encoding compresses.

Parameters:
csp  Current client state (buffers, headers, etc...)
header  On input, pointer to header to modify. On output, pointer to the modified header, or NULL to remove the header. This function frees the original string if necessary.
Returns:
JB_ERR_OK on success, or JB_ERR_MEMORY on out-of-memory error.

jb_err server_content_length struct client_state   csp,
char **    header
 

Adjust Content-Length header if we modified the body.

Parameters:
csp  Current client state (buffers, headers, etc...)
header  On input, pointer to header to modify. On output, pointer to the modified header, or NULL to remove the header. This function frees the original string if necessary.
Returns:
JB_ERR_OK on success, or JB_ERR_MEMORY on out-of-memory error.

jb_err server_content_md5 struct client_state   csp,
char **    header
 

Crumble any Content-MD5 headers if the document was modified.

FIXME: Should we re-compute instead?

Parameters:
csp  Current client state (buffers, headers, etc...)
header  On input, pointer to header to modify. On output, pointer to the modified header, or NULL to remove the header. This function frees the original string if necessary.
Returns:
JB_ERR_OK on success, or JB_ERR_MEMORY on out-of-memory error.

jb_err server_content_type struct client_state   csp,
char **    header
 

Set the content-type for filterable types (text/.*, javascript and image/gif) unless filtering has been forbidden (CT_TABOO) while parsing earlier headers.

Parameters:
csp  Current client state (buffers, headers, etc...)
header  On input, pointer to header to modify. On output, pointer to the modified header, or NULL to remove the header. This function frees the original string if necessary.
Returns:
JB_ERR_OK on success, or JB_ERR_MEMORY on out-of-memory error.

jb_err server_http struct client_state   csp,
char **    header
 

Save the HTTP Status into csp->http->status Set CT_TABOO to prevent filtering if the answer is a partial range (HTTP status 206) Rewrite HTTP/1.1 answers to HTTP/1.0 if +downgrade action applies.

  • * - * -
Parameters:
csp  Current client state (buffers, headers, etc...)
header  On input, pointer to header to modify. On output, pointer to the modified header, or NULL to remove the header. This function frees the original string if necessary.
Returns:
JB_ERR_OK on success, or JB_ERR_MEMORY on out-of-memory error.

jb_err server_set_cookie struct client_state   csp,
char **    header
 

Handle the server "cookie" header properly.

Log cookie to the jar file. Then "crunch" it, or accept it. Called from `sed'.

Parameters:
csp  Current client state (buffers, headers, etc...)
header  On input, pointer to header to modify. On output, pointer to the modified header, or NULL to remove the header. This function frees the original string if necessary.
Returns:
JB_ERR_OK on success, or JB_ERR_MEMORY on out-of-memory error.

jb_err server_transfer_coding struct client_state   csp,
char **    header
 

Prohibit filtering (CT_TABOO) if transfer coding compresses Raise the CSP_FLAG_CHUNKED flag if coding is "chunked" Change from "chunked" to "identity" if body was chunked but has been de-chunked for filtering.

  • * - * -
Parameters:
csp  Current client state (buffers, headers, etc...)
header  On input, pointer to header to modify. On output, pointer to the modified header, or NULL to remove the header. This function frees the original string if necessary.
Returns:
JB_ERR_OK on success, or JB_ERR_MEMORY on out-of-memory error.


Variable Documentation

const add_header_func_ptr add_client_headers[]
 

Initial value:

 {
   client_host_adder,
   client_cookie_adder,
   client_x_forwarded_adder,
   client_xtra_adder,
   client_accept_encoding_adder,
   connection_close_adder,
   NULL
}

const add_header_func_ptr add_server_headers[]
 

Initial value:

 {
   connection_close_adder,
   NULL
}

const struct parsers client_patterns[]
 

Initial value:

 {
   { "referer:",                 8,    client_referrer },
   { "user-agent:",              11,   client_uagent },
   { "ua-",                      3,    client_ua },
   { "from:",                    5,    client_from },
   { "cookie:",                  7,    client_send_cookie },
   { "x-forwarded-for:",         16,   client_x_forwarded },
   { "Accept-Encoding:",         16,   client_accept_encoding },
   { "TE:",                      3,    client_te },
   { "Host:",                     5,   crumble },

   { "Keep-Alive:",              11,   crumble },
   { "connection:",              11,   crumble },
   { "proxy-connection:",        17,   crumble },
   { NULL,                       0,    NULL }
}

const char parsers_h_rcs[] = PARSERS_H_VERSION
 

Version information about parsers.h.

const char parsers_rcs[] = "$Id: parsers.c,v 2.0 2002/06/04 14:34:21 jongfoster Exp $"
 

Version information about parsers.c.

const struct parsers server_patterns[]
 

Initial value:

 {
   { "HTTP",                4, server_http },
   { "set-cookie:",        11, server_set_cookie },
   { "connection:",        11, crumble },
   { "Content-Type:",      13, server_content_type },
   { "Content-Length:",    15, server_content_length },
   { "Content-MD5:",       12, server_content_md5 },
   { "Content-Encoding:",  17, server_content_encoding },
   { "Transfer-Encoding:", 18, server_transfer_coding },
   { "Keep-Alive:",        11, crumble },
   { NULL, 0, NULL }
}


Generated on Tue Jun 4 18:54:49 2002 for Privoxy 3.1.1 by doxygen1.2.15