-
-
Notifications
You must be signed in to change notification settings - Fork 590
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[TlsInterception] v2.4.x
vs v2.3.1
differences
#979
Conversation
Following works Gen cert
Run proxy$ python -m proxy --port 9000 --plugins proxy.plugin.CacheResponsesPlugin --ca-key-file ca-key.pem --ca-cert-file ca-cert.pem --ca-signing-key ca-signing-key.pem --enable-web-server --threadless
2022-01-13 16:34:21,954 - pid:23607 [I] plugins.load:85 - Loaded plugin proxy.http.proxy.HttpProxyPlugin
2022-01-13 16:34:21,955 - pid:23607 [I] plugins.load:85 - Loaded plugin proxy.http.server.HttpWebServerPlugin
2022-01-13 16:34:21,955 - pid:23607 [I] plugins.load:85 - Loaded plugin proxy.plugin.CacheResponsesPlugin
2022-01-13 16:34:21,955 - pid:23607 [I] listener.setup:97 - Listening on 127.0.0.1:9000
2022-01-13 16:34:22,011 - pid:23607 [I] pool.setup:106 - Started 16 acceptors in threadless (local) mode
2022-01-13 16:34:26,906 - pid:23614 [I] server.access_log:400 - 127.0.0.1:60346 - CONNECT httpbin.org:443 - 486 bytes - 1216.18ms
2022-01-13 16:34:26,906 - pid:23614 [I] disk.close:60 - Cached response at /var/folders/ry/tkb2qyj13qv1mdwf6z7r13l00000gn/T/httpbin.org-5-0-29.txt Curl$ curl -v -x localhost:9000 --cacert ca-cert.pem https://httpbin.org/get ─╯
* Trying ::1:9000...
* connect to ::1 port 9000 failed: Connection refused
* Trying 127.0.0.1:9000...
* Connected to localhost (127.0.0.1) port 9000 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to httpbin.org:443
> CONNECT httpbin.org:443 HTTP/1.1
> Host: httpbin.org:443
> User-Agent: curl/7.77.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
<
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: ca-cert.pem
* CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: CN=httpbin.org
* start date: Jan 13 11:03:03 2022 GMT
* expire date: Jan 13 11:03:03 2024 GMT
* subjectAltName: host "httpbin.org" matched cert's "httpbin.org"
* issuer: CN=example.com
* SSL certificate verify ok.
> GET /get HTTP/1.1
> Host: httpbin.org
> User-Agent: curl/7.77.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Thu, 13 Jan 2022 11:04:26 GMT
< Content-Type: application/json
< Content-Length: 256
< Connection: keep-alive
< Server: gunicorn/19.9.0
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Credentials: true
<
{
"args": {},
"headers": {
"Accept": "*/*",
"Host": "httpbin.org",
"User-Agent": "curl/7.77.0",
"X-Amzn-Trace-Id": "Root=1-61e0073a-6fdb4457650af7fd2ac13448"
},
"origin": "106.202.255.39",
"url": "https://httpbin.org/get"
}
* Connection #0 to host localhost left intact |
Codecov Report
@@ Coverage Diff @@
## develop #979 +/- ##
===========================================
- Coverage 85.90% 85.72% -0.19%
===========================================
Files 145 145
Lines 6252 6257 +5
Branches 636 639 +3
===========================================
- Hits 5371 5364 -7
- Misses 758 769 +11
- Partials 123 124 +1
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
v2.4.x
vs v2.3.1
differences fixesv2.4.x
vs v2.3.1
differences
* Remove menubar (#930) Remove `menubar` * Use `128 KB` as default value for `DEFAULT_BUFFER_SIZE` (#926) * Add `TlsInterception` acceptance test * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Generate CA cert if not available * Fix lint and tests * Fix args * Remove acceptance tests for now Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * pip prod(deps): bump wheel from 0.37.0 to 0.37.1 (#934) Bumps [wheel](https://github.com/pypa/wheel) from 0.37.0 to 0.37.1. - [Release notes](https://github.com/pypa/wheel/releases) - [Changelog](https://github.com/pypa/wheel/blob/main/docs/news.rst) - [Commits](pypa/wheel@0.37.0...0.37.1) --- updated-dependencies: - dependency-name: wheel dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * npm: bump jasmine from 3.10.0 to 4.0.0 in /dashboard (#933) Bumps [jasmine](https://github.com/jasmine/jasmine-npm) from 3.10.0 to 4.0.0. - [Release notes](https://github.com/jasmine/jasmine-npm/releases) - [Commits](jasmine/jasmine-npm@v3.10.0...v4.0.0) --- updated-dependencies: - dependency-name: jasmine dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * npm: bump ws from 8.3.0 to 8.4.0 in /dashboard (#936) Bumps [ws](https://github.com/websockets/ws) from 8.3.0 to 8.4.0. - [Release notes](https://github.com/websockets/ws/releases) - [Commits](websockets/ws@8.3.0...8.4.0) --- updated-dependencies: - dependency-name: ws dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * pip prod(deps): bump types-paramiko from 2.8.4 to 2.8.6 (#937) Bumps [types-paramiko](https://github.com/python/typeshed) from 2.8.4 to 2.8.6. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-paramiko dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Abhinav Singh <126065+abhinavsingh@users.noreply.github.com> * `DescriptorsHandlerMixin` and `Descriptors`, `SelectableEvents` types (#938) * Add `Descriptors` type * Add a `DescriptorsHandlerMixin` class used throughout the http framework * Remove dependency upon `HasFileno` ie `typing_extension` too * Define `SelectableEvents` type * Fix doc * Blank line * Remove dep on `typing-extensions` * Discover base plugin class * await on now async handlers * Fix doc spell * Add `--port-file` flag (#942) * Add `--port-file` flag * Use `--port-file` flag for integration tests using `get_available_port` * Use temp dir * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix `base_klass` variable related lint issues * Fix main tests * Fix integration * Use timeout when terminating proc * Skip integration on win instead of xmark Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Expose `UpstreamConnectionPool` to web & proxy plugins (#946) * Expose conn pool to plugins * Fix reusable state handling * Separate `release` and `retain` methods * Fix conn pool tests * Fix tests * npm: bump chrome-devtools-frontend in /dashboard (#949) Bumps [chrome-devtools-frontend](https://github.com/ChromeDevTools/devtools-frontend) from 1.0.952865 to 1.0.956881. - [Release notes](https://github.com/ChromeDevTools/devtools-frontend/releases) - [Changelog](https://github.com/ChromeDevTools/devtools-frontend/blob/main/docs/release_management.md) - [Commits](https://github.com/ChromeDevTools/devtools-frontend/commits) --- updated-dependencies: - dependency-name: chrome-devtools-frontend dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * pip prod(deps): bump sphinxcontrib-towncrier from 0.2.0a0 to 0.2.1a0 (#941) Bumps [sphinxcontrib-towncrier](https://github.com/sphinx-contrib/sphinxcontrib-towncrier) from 0.2.0a0 to 0.2.1a0. - [Release notes](https://github.com/sphinx-contrib/sphinxcontrib-towncrier/releases) - [Commits](sphinx-contrib/sphinxcontrib-towncrier@v0.2.0a0...v0.2.1a0) --- updated-dependencies: - dependency-name: sphinxcontrib-towncrier dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Abhinav Singh <126065+abhinavsingh@users.noreply.github.com> * `isort` everything except lib (for now) (#952) * isort the tests folder * Carry over changes from #672 * Disable pre-commit * Revert flake8 config change * isort examples too * pip prod(deps): bump httpx from 0.20.0 to 0.21.3 (#951) Bumps [httpx](https://github.com/encode/httpx) from 0.20.0 to 0.21.3. - [Release notes](https://github.com/encode/httpx/releases) - [Changelog](https://github.com/encode/httpx/blob/master/CHANGELOG.md) - [Commits](encode/httpx@0.20.0...0.21.3) --- updated-dependencies: - dependency-name: httpx dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Abhinav Singh <126065+abhinavsingh@users.noreply.github.com> * Decouple transport framework from dashboard plugin (#953) * Decouple transport framework from dashboard plugin * Move `InspectTrafficPlugin` within `http.inspector` module * Avoid exporting plugins within `__init__.py` files * Use `/transport/` prefix to avoid #945 conflict issue * Add todo * Build containers in parallel and pre-check (#954) * Build containers in parallel and pre-check * Fix loop in workflow * Test container needs build * Invoke `WebSocketTransportBasePlugin` connected and disconnected callbacks (#956) * [HttpProtocolHandler] Handle invalid request parsing exceptions (#957) * Handle invalid request parsing exception when raised, log the bytes for later inspection * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * npm: bump jasmine-ts from 0.3.0 to 0.4.0 in /dashboard (#958) Bumps [jasmine-ts](https://github.com/svi3c/jasmine-ts) from 0.3.0 to 0.4.0. - [Release notes](https://github.com/svi3c/jasmine-ts/releases) - [Changelog](https://github.com/svi3c/jasmine-ts/blob/master/CHANGELOG.md) - [Commits](https://github.com/svi3c/jasmine-ts/commits/v0.4.0) --- updated-dependencies: - dependency-name: jasmine-ts dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Ignore `utf-8` decoding errors within event core (#961) * [WebServer] Refactor routing to allow same path for websocket and web requests (#962) * Switch to WS * Refactor * Ignore utf-8 decode error during logging (#966) * On-demand `TlsInterception` capability, driven by plugins (#965) * Add `TlsInterceptionPropertyMixin` * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add `do_intercept` hook * call super init * No super from mixin as it is followed by abc? * type ignore * spell Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [WebServer] Fix routing (#968) * Fix web server routing * Fix mypy * pip prod(deps): bump furo from 2021.11.23 to 2022.1.2 (#959) Bumps [furo](https://github.com/pradyunsg/furo) from 2021.11.23 to 2022.1.2. - [Release notes](https://github.com/pradyunsg/furo/releases) - [Changelog](https://github.com/pradyunsg/furo/blob/main/docs/changelog.md) - [Commits](pradyunsg/furo@2021.11.23...2022.01.02) --- updated-dependencies: - dependency-name: furo dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Abhinav Singh <126065+abhinavsingh@users.noreply.github.com> * pip prod(deps): bump paramiko from 2.9.1 to 2.9.2 (#970) Bumps [paramiko](https://github.com/paramiko/paramiko) from 2.9.1 to 2.9.2. - [Release notes](https://github.com/paramiko/paramiko/releases) - [Changelog](https://github.com/paramiko/paramiko/blob/main/NEWS) - [Commits](paramiko/paramiko@2.9.1...2.9.2) --- updated-dependencies: - dependency-name: paramiko dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Abhinav Singh <126065+abhinavsingh@users.noreply.github.com> * Test submodule and refactor (#971) * Refactor tests into submodules * isort tests * Add malicious request headers test * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Handle `OSError` on shutdown & `TimeoutError` on recv (#974) * Expect `TimeoutError` during `recv` * Expect `OSError` during socket shutdown, can happen if other end has already closed the socket * pip prod(deps): bump mypy from 0.920 to 0.931 (#976) Bumps [mypy](https://github.com/python/mypy) from 0.920 to 0.931. - [Release notes](https://github.com/python/mypy/releases) - [Commits](python/mypy@v0.920...v0.931) --- updated-dependencies: - dependency-name: mypy dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * npm: bump @types/jquery from 3.5.4 to 3.5.13 in /dashboard (#975) Bumps [@types/jquery](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/jquery) from 3.5.4 to 3.5.13. - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/jquery) --- updated-dependencies: - dependency-name: "@types/jquery" dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Abhinav Singh <126065+abhinavsingh@users.noreply.github.com> * Refactor into separate `Work` module (#977) * work module * Fix imports * String based typing for multiprocessing.synchronize * Fix `test_accepts_client_from_server_socket` * Move staticmethod outside of threadless pool class * Fix doc build * Fix test mock * mp grouped together * pylint happy * import only for type checking * doc build * wrong import order * Type fixes (#979) * Type fixes * Type fix * Py class role * unused any import * [TlsInterception] GHA integration tests (#981) * Add TLS interception integration tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixture to gen certificate once for the `test_integration` module * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Use https endpoints in tls interception tests * Fix modify post data integration test * Only start 3 acceptor & 3 workers during integration run * disable chunk response Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * npm: bump typescript from 3.9.7 to 4.5.4 in /dashboard (#982) Bumps [typescript](https://github.com/Microsoft/TypeScript) from 3.9.7 to 4.5.4. - [Release notes](https://github.com/Microsoft/TypeScript/releases) - [Commits](microsoft/TypeScript@v3.9.7...v4.5.4) --- updated-dependencies: - dependency-name: typescript dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * pip prod(deps): bump types-paramiko from 2.8.6 to 2.8.9 (#983) Bumps [types-paramiko](https://github.com/python/typeshed) from 2.8.6 to 2.8.9. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-paramiko dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Pass separate `--ca-cert-dir` flag for parallel TLS interception tests (#984) * Pass separate `--ca-cert-dir` flag for parallel TLS interception tests * Temp disable `test_modify_post_response_integration` * mock ca cert dir * Is threaded an issue with TLS interception? * Disable modify chunk response for python < 3.10 * Disable modify chunk response for python < 3.10 * [TlsInterception] Fix broken `ChunkedResponsePlugin` for `Python < 3.10` (#986) * Add TLS interception integration tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix CI * unused * Well 3.9 just worked locally * Dispatching empty byte to client results in OSError for Python < 3.10 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Uncomment old integration tests Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Add `# pragma: no cover` for unnecessary code (#987) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
OSError
when sending empty chunk to clients underPython < 3.10
#978