test(*): implement new HTTP mocking #10885

StarlightIbuki · 2023-05-17T09:47:44Z

Summary

We have 3 types of implementation for HTTP mocking, each of which has some issues.

This PR introduces a new implementation to unify them with a dedicated Nginx instance.

It allows users to use the same logic as start_kong fixture. And it collects requests/responses/errors for later assertions on them.

The collected information is then fetched from a debugging port, and a set of assertion APIs handles the fetching process.

It's also possible to extend this solution to support clients-to-server debugging API, e.g, to send a piece of text as the response body for the next request.

Checklist

The Pull Request has tests
There's an entry in the CHANGELOG (Should we?)

Full changelog

New HTTP mocking implementations, including:
- Logic to create and manage the Nginx server instance;
- API to create an HTTP client to the server;
- APIs to make assertions on requests, responses, and errors;
Tests of the new implementation;
Deprecating start_kong fixture, http_server, mock_http, and https_server;
An example of using the new implementation.

Issue reference

Fix KAG-1148

Todo after this merged

Migrate current uses of start_kong fixture, http_server, mock_http, and https_server.

ADD-SP

Could you please move the tests from spec/helpers/http_mock_spec.lua to spec/02-integration/01-helpers?

ADD-SP

Please use the new mocking server replace some old code that used the old mocking servers.

spec/helpers/http_mock/asserts.lua

spec/helpers/http_mock/template.lua

spec/helpers/http_mock.lua

spec/helpers/http_mock/nginx_instance.lua

spec/helpers/http_mock/template.lua

ADD-SP

spec/02-integration/05-proxy/03-upstream_headers_spec.lua is red

ADD-SP

LGTM

hanshuebner

I like this proposal but I have a couple of comments and suggestions. Generally, do we have an understanding of the time that it takes to start up nginx compared to the thread and coroutine based mock servers?

spec/02-integration/01-helpers/03-http_mock_spec.lua

spec/helpers/http_mock.lua

hanshuebner · 2023-05-22T14:13:32Z

spec/helpers/http_mock/asserts.lua

+local function register_assert(name, impl)
+  eventually_MT["has_" .. name] = function(self, ...)
+    return eventually_has(impl, self.__mock, ...)
+  end
+
+  eventually_MT["all_" .. name] = function(self, ...)
+    return eventually_all(impl, self.__mock, ...)
+  end
+
+  local function reverse_impl(session, ...)
+    local ok, err = pcall(impl, session, ...)
+    if ok then
+      error("we don't expect that: " .. (name or err), 2)
+    end
+    return true
+  end
+
+  eventually_MT["has_no_" .. name] = function(self, ...)
+    return eventually_all(reverse_impl, self.__mock, ...)
+  end
+
+  eventually_MT["not_all_" .. name] = function(self, ...)
+    return eventually_has(reverse_impl, self.__mock, ...)
+  end
+end
+
+for name, impl in pairs(build_in_checks) do
+  register_assert(name, impl)
+end


This looks cool and would probably be something to add to helpers.lua or live in a separate module.

We have this:

kong/spec/helpers/wait.lua

Line 453 in 458fd9f

local function eventually(state, arguments)

Actually, the design is inspired by assert.eventually.

This logic is coupled with self.__mock and get_all_records(). Maybe we will find a way to decouple them.

spec/helpers/http_mock/asserts.lua

spec/helpers/http_mock/debug_port.lua

spec/helpers/http_mock/template.lua

StarlightIbuki · 2023-05-23T02:54:29Z

I like this proposal but I have a couple of comments and suggestions. Generally, do we have an understanding of the time that it takes to start up nginx compared to the thread and coroutine based mock servers?

Starting a vanilla Nginx is really quick. I tried the test 03-http_mock_spec.lua on my own laptop and it takes less than 4s to run 15 tests (if we set a very small timeout for those waiting tests).

spec/helpers.lua

spec/fixtures/https_server.lua

spec/helpers.lua

spec/helpers/http_mock/nginx_instance.lua

hanshuebner

just some minor nits, please

spec/helpers/http_mock.lua

spec/helpers/http_mock/asserts.lua

Fix KAG-1148 apply suggestions Co-authored-by: Chrono <chrono_cpp@me.com> Co-authored-by: Hans Hübner <hans.huebner@gmail.com>

This commit backports HTTP mocking to release/3.2.x, including the following PRs from master branch: #10885, #11009, #11182 and #11331.

pull-request-size bot added the size/L label May 17, 2023

github-actions bot assigned StarlightIbuki May 17, 2023

StarlightIbuki force-pushed the test/new-http-mock branch from c9e149c to fe7868e Compare May 18, 2023 06:52

pull-request-size bot added size/XL and removed size/L labels May 18, 2023

ADD-SP requested changes May 19, 2023

View reviewed changes

spec/helpers/http_mock/asserts.lua Outdated Show resolved Hide resolved

spec/helpers/http_mock/template.lua Outdated Show resolved Hide resolved

StarlightIbuki force-pushed the test/new-http-mock branch 2 times, most recently from 97e1b0b to fbd28cb Compare May 22, 2023 06:46

pull-request-size bot added size/XXL and removed size/XL labels May 22, 2023

StarlightIbuki force-pushed the test/new-http-mock branch from fbd28cb to cbfaaa0 Compare May 22, 2023 06:50

StarlightIbuki marked this pull request as ready for review May 22, 2023 06:50

StarlightIbuki requested a review from ADD-SP May 22, 2023 06:50

StarlightIbuki force-pushed the test/new-http-mock branch 2 times, most recently from 315ec03 to b6a439e Compare May 22, 2023 07:24

ADD-SP requested changes May 22, 2023

View reviewed changes

spec/helpers/http_mock.lua Show resolved Hide resolved

spec/helpers/http_mock/nginx_instance.lua Outdated Show resolved Hide resolved

spec/helpers/http_mock/nginx_instance.lua Show resolved Hide resolved

spec/helpers/http_mock/template.lua Outdated Show resolved Hide resolved

StarlightIbuki force-pushed the test/new-http-mock branch from b6a439e to 5d77f9f Compare May 22, 2023 07:48

StarlightIbuki requested a review from ADD-SP May 22, 2023 07:48

ADD-SP requested changes May 22, 2023

View reviewed changes

StarlightIbuki force-pushed the test/new-http-mock branch from 5d77f9f to 1b4db6f Compare May 22, 2023 08:44

StarlightIbuki requested a review from ADD-SP May 22, 2023 08:44

ADD-SP approved these changes May 22, 2023

View reviewed changes

hanshuebner suggested changes May 22, 2023

View reviewed changes

chronolaw reviewed May 23, 2023

View reviewed changes

spec/helpers.lua Outdated Show resolved Hide resolved

spec/helpers.lua Outdated Show resolved Hide resolved

spec/fixtures/https_server.lua Outdated Show resolved Hide resolved

spec/helpers.lua Outdated Show resolved Hide resolved

StarlightIbuki force-pushed the test/new-http-mock branch from 80edf0e to f1b91ef Compare May 23, 2023 04:33

StarlightIbuki requested a review from hanshuebner May 23, 2023 04:36

chronolaw reviewed May 23, 2023

View reviewed changes

spec/helpers/http_mock/nginx_instance.lua Show resolved Hide resolved

StarlightIbuki force-pushed the test/new-http-mock branch from f1b91ef to 89282cf Compare May 23, 2023 06:55

hanshuebner suggested changes May 23, 2023

View reviewed changes

spec/helpers/http_mock.lua Outdated Show resolved Hide resolved

spec/helpers/http_mock.lua Show resolved Hide resolved

spec/helpers/http_mock/asserts.lua Show resolved Hide resolved

StarlightIbuki and others added 2 commits May 23, 2023 17:28

test(*): implement new HTTP mocking

fbb3a4d

Fix KAG-1148 apply suggestions Co-authored-by: Chrono <chrono_cpp@me.com> Co-authored-by: Hans Hübner <hans.huebner@gmail.com>

chore(test): use new http_mock for 05-proxy

fc15f47

StarlightIbuki force-pushed the test/new-http-mock branch from 0ad116e to fc15f47 Compare May 23, 2023 09:28

StarlightIbuki requested a review from hanshuebner May 23, 2023 09:28

hanshuebner approved these changes May 23, 2023

View reviewed changes

hanshuebner merged commit 068b318 into master May 23, 2023

hanshuebner deleted the test/new-http-mock branch May 23, 2023 10:09

StarlightIbuki mentioned this pull request May 24, 2023

chore(test): eliminate duplicate mig http_server #10867

Closed

1 task

chobits mentioned this pull request Nov 29, 2023

[backport -> release/3.2.x] chore(tests): deprecate uses of mockbin.com #12031

Merged

chobits added a commit that referenced this pull request Nov 29, 2023

test(*): implement new HTTP mocking

463f010

This commit backports HTTP mocking to release/3.2.x, including the following PRs from master branch: #10885, #11009, #11182 and #11331.

windmgc pushed a commit that referenced this pull request Nov 29, 2023

test(*): implement new HTTP mocking

df3ac9c

This commit backports HTTP mocking to release/3.2.x, including the following PRs from master branch: #10885, #11009, #11182 and #11331.

chobits added a commit that referenced this pull request Nov 30, 2023

test(*): implement new HTTP mocking

b48577f

This commit backports HTTP mocking to release/3.2.x, including the following PRs from master branch: #10885, #11009, #11182 and #11331.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test(*): implement new HTTP mocking #10885

test(*): implement new HTTP mocking #10885

StarlightIbuki commented May 17, 2023 •

edited

Loading

ADD-SP left a comment •

edited

Loading

ADD-SP left a comment

ADD-SP left a comment

ADD-SP left a comment

hanshuebner left a comment

hanshuebner May 22, 2023

StarlightIbuki May 23, 2023

StarlightIbuki commented May 23, 2023 •

edited

Loading

hanshuebner left a comment

test(*): implement new HTTP mocking #10885

test(*): implement new HTTP mocking #10885

Conversation

StarlightIbuki commented May 17, 2023 • edited Loading

Summary

Checklist

Full changelog

Issue reference

Todo after this merged

ADD-SP left a comment • edited Loading

Choose a reason for hiding this comment

ADD-SP left a comment

Choose a reason for hiding this comment

ADD-SP left a comment

Choose a reason for hiding this comment

ADD-SP left a comment

Choose a reason for hiding this comment

hanshuebner left a comment

Choose a reason for hiding this comment

hanshuebner May 22, 2023

Choose a reason for hiding this comment

StarlightIbuki May 23, 2023

Choose a reason for hiding this comment

StarlightIbuki commented May 23, 2023 • edited Loading

hanshuebner left a comment

Choose a reason for hiding this comment

StarlightIbuki commented May 17, 2023 •

edited

Loading

ADD-SP left a comment •

edited

Loading

StarlightIbuki commented May 23, 2023 •

edited

Loading