[THREESCALE-10164] Add support to set large_client_header_buffers directive #1446

tkan145 · 2024-02-08T07:00:20Z

What

Fixes: https://issues.redhat.com/browse/THREESCALE-10164

Verification Steps

Creat a apicast-config.json file with the following content

cat <<EOF >apicast-config.json
{
    "services": [
        {
            "id": "1",
            "backend_version": "1",
            "proxy": {
                "hosts": [
                    "one"
                ],
                "api_backend": "https://echo-api.3scale.net:443",
                "backend": {
                    "endpoint": "http://127.0.0.1:8081",
                    "host": "backend"
                },
                "policy_chain": [
                    {
                        "name": "apicast.policy.apicast"
                    }
                ],
                "proxy_rules": [
                    {
                        "http_method": "GET",
                        "pattern": "/",
                        "metric_system_name": "hits",
                        "delta": 1,
                        "parameters": [],
                        "querystring_parameters": {}
                    }
                ]
            }
        }
    ]
} 
EOF

Checkout this branch and start dev environment

make development
make dependencies

Run apicast locally

THREESCALE_DEPLOYMENT_ENV=staging APICAST_LOG_LEVEL=debug APICAST_WORKER=1 APICAST_CONFIGURATION_LOADER=lazy APICAST_CONFIGURATION_CACHE=0 THREESCALE_CONFIG_FILE=apicast-config.json ./bin/apicast

Capture apicast IP

APICAST_IP=$(docker inspect apicast_build_0-development-1 | yq e -P '.[0].NetworkSettings.Networks.apicast_build_0_default.IPAddress' -)

Generate big header

LARGE_HEADER=$(for i in {1..1024}; do echo -n 'ABCDEFGH'; done)

Send request with big header

curl -i -k -H "Host: one" -H "Accept: application/json" -H "Large-Header: ${LARGE_HEADER}" "http://${APICAST_IP}:8080/?user_key="

The response should be HTTP/1.1 400 Bad Request

HTTP/1.1 400 Bad Request                                          
Server: openresty                                                 
Date: Thu, 08 Feb 2024 06:49:26 GMT                               
Content-Type: text/html                                           
Content-Length: 230                                               
Connection: close                                                 
                                                                  
<html>                                                            
<head><title>400 Request Header Or Cookie Too Large</title></head>
<body>                                                            
<center><h1>400 Bad Request</h1></center>                         
<center>Request Header Or Cookie Too Large</center>               
<hr><center>openresty</center>                                    
</body>                                                           
</html>

Stop the gateway

CTRL-C

Start gateway again with APICAST_LARGE_CLIENT_HEADER_BUFFERS=4 12k

THREESCALE_DEPLOYMENT_ENV=staging APICAST_LOG_LEVEL=debug APICAST_WORKER=1 APICAST_CONFIGURATION_LOADER=lazy APICAST_CONFIGURATION_CACHE=0 THREESCALE_CONFIG_FILE=apicast-config.json APICAST_LARGE_CLIENT_HEADER_BUFFERS="4 12k" ./bin/apicast

Send another request

curl -i -k -H "Host: one" -H "Accept: application/json" -H "Large-Header: ${LARGE_HEADER}" "http://${APICAST_IP}:8080/?user_key="

The response should be HTTP/1.1 200 OK

eguzki

Looking good.

tkan145 · 2024-02-09T06:22:27Z

I kindly requested review from @3scale/documentation team.

For some reason the test keeps failing in this PR 😞 , hopefully after I address the feedback from the docs team and make a new commit the test will work

dfennessy · 2024-02-09T09:32:20Z

CHANGELOG.md

@@ -45,6 +45,8 @@ and this project adheres to [Semantic Versioning](http://semver.org/).

 - Dev environment: Camel proxy [PR #1441](https://github.com/3scale/APIcast/pull/1441)

+- Added `APICAST_CLIENT_REQUEST_HEADER_BUFFERS` variable that allows to configure nginx `client_request_header_buffers` directive [PR #1446](https://github.com/3scale/APIcast/pull/1446), [THREESCALE-10164](https://issues.redhat.com/browse/THREESCALE-10164)


Suggested change

- Added `APICAST_CLIENT_REQUEST_HEADER_BUFFERS` variable that allows to configure nginx `client_request_header_buffers` directive [PR #1446](https://github.com/3scale/APIcast/pull/1446), [THREESCALE-10164](https://issues.redhat.com/browse/THREESCALE-10164)

- Added `APICAST_CLIENT_REQUEST_HEADER_BUFFERS` variable to allow configuration of the NGINX `client_request_header_buffers` directive: [PR #1446](https://github.com/3scale/APIcast/pull/1446), [THREESCALE-10164](https://issues.redhat.com/browse/THREESCALE-10164)

dfennessy

I've added a suggestion.

eguzki

Verification steps working like a charm

Some nitpicking and ready to merge.

eguzki · 2024-02-13T09:42:02Z

t/large-client-header-buffers.t

+
+__DATA__
+
+=== TEST 1: large header (the header exceed exceed the size of one buffer)


exceed exceed 👀

eguzki · 2024-02-13T09:46:17Z

t/large-client-header-buffers.t

+  }
+--- upstream
+
+  client_header_buffer_size 10;


this is needed? why?

This is not needed. I was trying to force nginx to use a large client header buffer, but I realized it's easier to just send a large header.

eguzki

LGTM but there are new conflicts (in CHANGELOG)

tkan145 · 2024-02-19T04:59:07Z

Conflict resolved

eguzki · 2024-02-19T12:23:59Z

@dfennessy your approval is needed as you requested changes

dfennessy

/lgtm

tkan145 changed the title ~~Add support to set large_client_header_buffers directive~~ [THREESCALE-10164] Add support to set large_client_header_buffers directive Feb 8, 2024

tkan145 force-pushed the THREESCALE-10164-large_client_header_buffers branch from 59ae70b to 9e6c973 Compare February 8, 2024 07:35

eguzki reviewed Feb 8, 2024

View reviewed changes

tkan145 marked this pull request as ready for review February 9, 2024 06:12

tkan145 requested review from a team as code owners February 9, 2024 06:12

dfennessy reviewed Feb 9, 2024

View reviewed changes

dfennessy requested changes Feb 9, 2024

View reviewed changes

tkan145 force-pushed the THREESCALE-10164-large_client_header_buffers branch from 9e6c973 to 5f752bf Compare February 11, 2024 23:01

eguzki requested changes Feb 13, 2024

View reviewed changes

eguzki mentioned this pull request Feb 13, 2024

[WIP] THREESCALE-10164 Including the env var LARGE_CLIENT_HEADER_BUFFERS #1429

Closed

eguzki reviewed Feb 14, 2024

View reviewed changes

tkan145 added 2 commits February 19, 2024 12:30

Add support to set large_client_header_buffers directive

88c421c

t/large-client-header-buffers.t fix typo and remove unsued code

30cec6c

tkan145 force-pushed the THREESCALE-10164-large_client_header_buffers branch from 3cda591 to 30cec6c Compare February 19, 2024 02:30

eguzki approved these changes Feb 19, 2024

View reviewed changes

dfennessy approved these changes Feb 19, 2024

View reviewed changes

tkan145 merged commit 5b97638 into 3scale:master Feb 20, 2024
12 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[THREESCALE-10164] Add support to set large_client_header_buffers directive #1446

[THREESCALE-10164] Add support to set large_client_header_buffers directive #1446

tkan145 commented Feb 8, 2024 •

edited by eguzki

Loading

eguzki left a comment

tkan145 commented Feb 9, 2024

dfennessy Feb 9, 2024

dfennessy left a comment

eguzki left a comment

eguzki Feb 13, 2024

tkan145 Feb 13, 2024

tkan145 Feb 19, 2024

eguzki Feb 13, 2024

tkan145 Feb 13, 2024

tkan145 Feb 19, 2024

eguzki left a comment

tkan145 commented Feb 19, 2024

eguzki commented Feb 19, 2024

dfennessy left a comment

		@@ -45,6 +45,8 @@ and this project adheres to [Semantic Versioning](http://semver.org/).

		- Dev environment: Camel proxy [PR #1441](https://github.com/3scale/APIcast/pull/1441)

		- Added `APICAST_CLIENT_REQUEST_HEADER_BUFFERS` variable that allows to configure nginx `client_request_header_buffers` directive [PR #1446](https://github.com/3scale/APIcast/pull/1446), [THREESCALE-10164](https://issues.redhat.com/browse/THREESCALE-10164)


		__DATA__

		=== TEST 1: large header (the header exceed exceed the size of one buffer)

[THREESCALE-10164] Add support to set large_client_header_buffers directive #1446

[THREESCALE-10164] Add support to set large_client_header_buffers directive #1446

Conversation

tkan145 commented Feb 8, 2024 • edited by eguzki Loading

What

Verification Steps

eguzki left a comment

Choose a reason for hiding this comment

tkan145 commented Feb 9, 2024

dfennessy Feb 9, 2024

Choose a reason for hiding this comment

dfennessy left a comment

Choose a reason for hiding this comment

eguzki left a comment

Choose a reason for hiding this comment

eguzki Feb 13, 2024

Choose a reason for hiding this comment

tkan145 Feb 13, 2024

Choose a reason for hiding this comment

tkan145 Feb 19, 2024

Choose a reason for hiding this comment

eguzki Feb 13, 2024

Choose a reason for hiding this comment

tkan145 Feb 13, 2024

Choose a reason for hiding this comment

tkan145 Feb 19, 2024

Choose a reason for hiding this comment

eguzki left a comment

Choose a reason for hiding this comment

tkan145 commented Feb 19, 2024

eguzki commented Feb 19, 2024

dfennessy left a comment

Choose a reason for hiding this comment

tkan145 commented Feb 8, 2024 •

edited by eguzki

Loading