If the the response is chunk-encoded, ignore the Content-Length

... header sent by the server.

Allows to load https://redmine.lighttpd.net/ with filtering enabled.

Previously requests would fail with complaints like:

   2021-03-20 14:02:08.924 619000011880 Connect: Done reading from server. Expected content length: 7235. Actual content length: 7243. Bytes most recently read: 8130.
   2021-03-20 14:02:08.924 619000011880 Re-Filter: Need to de-chunk first
   2021-03-20 14:02:08.924 619000011880 Error: Not enough room for trailing CRLF.
   2021-03-20 14:02:08.925 619000011880 Connect: Received 7243 bytes while expecting 7235.
   2021-03-20 14:02:08.925 619000011880 Connect: Marking the server socket 8 tainted.

Privoxy would then forward a partialy de-chunked response with
trailing garbage without removing the Transfer-Encoding header.

diff --git a/parsers.c b/parsers.c
index 3197c4f..dcb1230 100644 (file)
--- a/parsers.c
+++ b/parsers.c
@@ -1301,6 +1301,17 @@ jb_err sed(struct client_state *csp, int filter_server_headers)
       v++;
    }
 
+   if (filter_server_headers &&
+      (csp->flags & CSP_FLAG_SERVER_CONTENT_LENGTH_SET) &&
+      (csp->flags & CSP_FLAG_CHUNKED))
+   {
+      /* RFC 2616 4.4 3 */
+      log_error(LOG_LEVEL_HEADER, "Ignoring the Content-Length header "
+         "sent by the server as the response is chunk-encoded.");
+      csp->flags &= ~CSP_FLAG_CONTENT_LENGTH_SET;
+      csp->expected_content_length = 0;
+   }
+
    /* place additional headers on the csp->headers list */
    while ((err == JB_ERR_OK) && (*f))
    {