Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add compress api for coro client #597

Merged
merged 2 commits into from
Jun 12, 2024

Conversation

helintongh
Copy link
Collaborator

@helintongh helintongh commented Jun 10, 2024

  1. add multiple encoding type support
  2. add encoding for coro client
  3. coro server add accept-encoding check

Copy link

Code Coverage Report
for detail, goto summary download Artifacts

Filename                                      Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
cinatra_log_wrapper.hpp                               4                 1    75.00%           9                 1    88.89%           0                 0         -
cookie.hpp                                           15                 2    86.67%          84                 2    97.62%          28                 3    89.29%
coro_http_client.hpp                                 83                 5    93.98%        1280               198    84.53%         410               114    72.20%
coro_http_connection.hpp                             36                 2    94.44%         667               208    68.82%         202                71    64.85%
coro_http_request.hpp                                29                 5    82.76%         201                24    88.06%          74                10    86.49%
coro_http_response.hpp                               31                 6    80.65%         225                68    69.78%          90                36    60.00%
coro_http_router.hpp                                 15                 0   100.00%         181                32    82.32%          20                 8    60.00%
coro_http_server.hpp                                 50                 5    90.00%         947               122    87.12%         178                47    73.60%
coro_radix_tree.hpp                                  17                 0   100.00%         264                38    85.61%         136                30    77.94%
define.h                                              2                 0   100.00%          28                 2    92.86%          20                 1    95.00%
http_parser.hpp                                      29                 5    82.76%         201                32    84.08%          56                12    78.57%
metric_conf.hpp                                       8                 4    50.00%          96                80    16.67%          32                28    12.50%
mime_types.hpp                                        1                 0   100.00%           7                 2    71.43%           2                 1    50.00%
multipart.hpp                                         4                 0   100.00%          89                15    83.15%          22                 5    77.27%
picohttpparser.h                                     14                 6    57.14%         467               278    40.47%         212               103    51.42%
response_cv.hpp                                       2                 0   100.00%          55                32    41.82%          46                16    65.22%
session.hpp                                          11                 0   100.00%          54                 2    96.30%           4                 1    75.00%
session_manager.hpp                                  10                 1    90.00%          67                 5    92.54%          10                 1    90.00%
sha1.hpp                                             12                 0   100.00%         181                 8    95.58%          12                 2    83.33%
string_resize.hpp                                     2                 0   100.00%          18                 0   100.00%           2                 0   100.00%
time_util.hpp                                        20                 7    65.00%         172                70    59.30%          28                 2    92.86%
uri.hpp                                              17                 5    70.59%         200                62    69.00%         166                52    68.67%
url_encode_decode.hpp                                 4                 0   100.00%          85                20    76.47%          48                16    66.67%
utils.hpp                                            13                 0   100.00%         201                22    89.05%         116                43    62.93%
websocket.hpp                                        15                 4    73.33%         170                43    74.71%          70                18    74.29%
ylt/coro_io/channel.hpp                              15                 0   100.00%         106                 7    93.40%          30                 5    83.33%
ylt/coro_io/client_pool.hpp                          21                 3    85.71%         324               125    61.42%          62                38    38.71%
ylt/coro_io/coro_file.hpp                            22                 1    95.45%         213                38    82.16%          56                19    66.07%
ylt/coro_io/coro_io.hpp                              52                 6    88.46%         329                38    88.45%           8                 2    75.00%
ylt/coro_io/detail/client_queue.hpp                  10                 5    50.00%          47                23    51.06%          10                 4    60.00%
ylt/coro_io/io_context_pool.hpp                      38                 6    84.21%         216                29    86.57%          28                 8    71.43%
ylt/metric/counter.hpp                               18                18     0.00%         192               192     0.00%           2                 2     0.00%
ylt/metric/detail/ckms_quantiles.hpp                  9                 9     0.00%         127               127     0.00%           0                 0         -
ylt/metric/detail/time_window_quantiles.hpp           4                 4     0.00%          25                25     0.00%           0                 0         -
ylt/metric/gauge.hpp                                  5                 5     0.00%          31                31     0.00%           0                 0         -
ylt/metric/histogram.hpp                              5                 5     0.00%          55                55     0.00%           0                 0         -
ylt/metric/metric.hpp                                25                25     0.00%         128               128     0.00%           8                 8     0.00%
ylt/metric/summary.hpp                               10                10     0.00%          98                98     0.00%           0                 0         -
ylt/util/concurrentqueue.h                           79                28    64.56%        1568              1049    33.10%         290               198    31.72%
ylt/util/expected.hpp                                 6                 4    33.33%           6                 4    33.33%           0                 0         -
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                               763               187    75.49%        9414              3335    64.57%        2478               904    63.52%

Comment on lines 1441 to 1447
if (parser_.get_header_value("Content-Encoding") == "gzip")
encoding_type_ = content_encoding::gzip;
else if (parser_.get_header_value("Content-Encoding") == "deflate")
encoding_type_ = content_encoding::deflate;
}
Copy link
Owner

@qicosmos qicosmos Jun 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里有点问题,"Content-Encoding"可能有多个字段,有可能一个字段有多个encoding方式,比如:

"Content-Encoding":"gzip"
"Content-Encoding":"deflate"

或者

"Content-Encoding": "gzip, deflate, br"

另外reply的时候如果发现set_status_and_content时已经设置了gzip或者deflate,就按设置的方式去reply,如果没有设置,但是头里面又能找到gzip或者deflate,那么就默认选择gzip;如果只有一种encoding就按头里面的来。

如果头里面没有Content-Encoding,但是用户设置response的时候又指定了gzip或deflate,则忽略它,不去压缩。

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

设置response的时候是解析客户端的Accept-Encoding,如果包含当前用户设置的按照用户设置的reponse进行响应。
如果不包含或无此字段则按照原始的响应

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已修改

1. add multiple encoding type support
2. add encoding for coro client
3. coro server add accept-encoding check
@helintongh helintongh force-pushed the feat_client_add_compress_support branch from c46cbf5 to f101377 Compare June 11, 2024 11:06
@helintongh helintongh requested a review from qicosmos June 11, 2024 11:08
Copy link

Code Coverage Report
for detail, goto summary download Artifacts

Filename                                      Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
cinatra_log_wrapper.hpp                               4                 1    75.00%           9                 1    88.89%           0                 0         -
cookie.hpp                                           15                 2    86.67%          84                 2    97.62%          28                 3    89.29%
coro_http_client.hpp                                 83                 5    93.98%        1280               198    84.53%         410               114    72.20%
coro_http_connection.hpp                             36                 2    94.44%         667               208    68.82%         202                71    64.85%
coro_http_request.hpp                                29                 5    82.76%         201                24    88.06%          74                10    86.49%
coro_http_response.hpp                               31                 6    80.65%         225                68    69.78%          90                36    60.00%
coro_http_router.hpp                                 15                 0   100.00%         181                32    82.32%          20                 8    60.00%
coro_http_server.hpp                                 50                 5    90.00%         947               122    87.12%         178                47    73.60%
coro_radix_tree.hpp                                  17                 0   100.00%         264                38    85.61%         136                30    77.94%
define.h                                              2                 0   100.00%          28                 2    92.86%          20                 1    95.00%
http_parser.hpp                                      29                 5    82.76%         201                32    84.08%          56                12    78.57%
metric_conf.hpp                                       8                 4    50.00%          96                80    16.67%          32                28    12.50%
mime_types.hpp                                        1                 0   100.00%           7                 2    71.43%           2                 1    50.00%
multipart.hpp                                         4                 0   100.00%          89                15    83.15%          22                 5    77.27%
picohttpparser.h                                     14                 6    57.14%         467               278    40.47%         212               103    51.42%
response_cv.hpp                                       2                 0   100.00%          55                32    41.82%          46                16    65.22%
session.hpp                                          11                 0   100.00%          54                 2    96.30%           4                 1    75.00%
session_manager.hpp                                  10                 1    90.00%          67                 5    92.54%          10                 1    90.00%
sha1.hpp                                             12                 0   100.00%         181                 8    95.58%          12                 2    83.33%
string_resize.hpp                                     2                 0   100.00%          18                 0   100.00%           2                 0   100.00%
time_util.hpp                                        20                 7    65.00%         172                70    59.30%          28                 2    92.86%
uri.hpp                                              17                 5    70.59%         200                62    69.00%         166                52    68.67%
url_encode_decode.hpp                                 4                 0   100.00%          85                20    76.47%          48                16    66.67%
utils.hpp                                            13                 0   100.00%         201                22    89.05%         116                43    62.93%
websocket.hpp                                        15                 4    73.33%         170                43    74.71%          70                18    74.29%
ylt/coro_io/channel.hpp                              15                 0   100.00%         106                 7    93.40%          30                 5    83.33%
ylt/coro_io/client_pool.hpp                          21                 3    85.71%         324               125    61.42%          62                38    38.71%
ylt/coro_io/coro_file.hpp                            22                 1    95.45%         213                38    82.16%          56                19    66.07%
ylt/coro_io/coro_io.hpp                              52                 6    88.46%         329                38    88.45%           8                 2    75.00%
ylt/coro_io/detail/client_queue.hpp                  10                 5    50.00%          47                23    51.06%          10                 4    60.00%
ylt/coro_io/io_context_pool.hpp                      38                 6    84.21%         216                29    86.57%          28                 8    71.43%
ylt/metric/counter.hpp                               18                18     0.00%         192               192     0.00%           2                 2     0.00%
ylt/metric/detail/ckms_quantiles.hpp                  9                 9     0.00%         127               127     0.00%           0                 0         -
ylt/metric/detail/time_window_quantiles.hpp           4                 4     0.00%          25                25     0.00%           0                 0         -
ylt/metric/gauge.hpp                                  5                 5     0.00%          31                31     0.00%           0                 0         -
ylt/metric/histogram.hpp                              5                 5     0.00%          55                55     0.00%           0                 0         -
ylt/metric/metric.hpp                                25                25     0.00%         128               128     0.00%           8                 8     0.00%
ylt/metric/summary.hpp                               10                10     0.00%          98                98     0.00%           0                 0         -
ylt/util/concurrentqueue.h                           79                28    64.56%        1568              1049    33.10%         290               198    31.72%
ylt/util/expected.hpp                                 6                 4    33.33%           6                 4    33.33%           0                 0         -
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                               763               187    75.49%        9414              3335    64.57%        2478               904    63.52%

Copy link

Code Coverage Report
for detail, goto summary download Artifacts

Filename                                      Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
cinatra_log_wrapper.hpp                               4                 1    75.00%           9                 1    88.89%           0                 0         -
cookie.hpp                                           15                 2    86.67%          84                 2    97.62%          28                 3    89.29%
coro_http_client.hpp                                 83                 5    93.98%        1280               198    84.53%         410               114    72.20%
coro_http_connection.hpp                             36                 2    94.44%         667               208    68.82%         202                71    64.85%
coro_http_request.hpp                                29                 5    82.76%         201                24    88.06%          74                10    86.49%
coro_http_response.hpp                               31                 6    80.65%         225                68    69.78%          90                36    60.00%
coro_http_router.hpp                                 15                 0   100.00%         181                32    82.32%          20                 8    60.00%
coro_http_server.hpp                                 50                 5    90.00%         947               122    87.12%         178                47    73.60%
coro_radix_tree.hpp                                  17                 0   100.00%         264                38    85.61%         136                30    77.94%
define.h                                              2                 0   100.00%          28                 2    92.86%          20                 1    95.00%
http_parser.hpp                                      29                 5    82.76%         201                32    84.08%          56                12    78.57%
metric_conf.hpp                                       8                 4    50.00%          96                80    16.67%          32                28    12.50%
mime_types.hpp                                        1                 0   100.00%           7                 2    71.43%           2                 1    50.00%
multipart.hpp                                         4                 0   100.00%          89                15    83.15%          22                 5    77.27%
picohttpparser.h                                     14                 6    57.14%         467               278    40.47%         212               103    51.42%
response_cv.hpp                                       2                 0   100.00%          55                32    41.82%          46                16    65.22%
session.hpp                                          11                 0   100.00%          54                 2    96.30%           4                 1    75.00%
session_manager.hpp                                  10                 1    90.00%          67                 5    92.54%          10                 1    90.00%
sha1.hpp                                             12                 0   100.00%         181                 8    95.58%          12                 2    83.33%
string_resize.hpp                                     2                 0   100.00%          18                 0   100.00%           2                 0   100.00%
time_util.hpp                                        20                 7    65.00%         172                70    59.30%          28                 2    92.86%
uri.hpp                                              17                 5    70.59%         200                62    69.00%         166                52    68.67%
url_encode_decode.hpp                                 4                 0   100.00%          85                20    76.47%          48                16    66.67%
utils.hpp                                            13                 0   100.00%         201                22    89.05%         116                43    62.93%
websocket.hpp                                        15                 4    73.33%         170                43    74.71%          70                18    74.29%
ylt/coro_io/channel.hpp                              15                 0   100.00%         106                 7    93.40%          30                 5    83.33%
ylt/coro_io/client_pool.hpp                          21                 3    85.71%         324               125    61.42%          62                38    38.71%
ylt/coro_io/coro_file.hpp                            22                 1    95.45%         213                38    82.16%          56                19    66.07%
ylt/coro_io/coro_io.hpp                              52                 6    88.46%         329                38    88.45%           8                 2    75.00%
ylt/coro_io/detail/client_queue.hpp                  10                 5    50.00%          47                23    51.06%          10                 4    60.00%
ylt/coro_io/io_context_pool.hpp                      38                 6    84.21%         216                29    86.57%          28                 8    71.43%
ylt/metric/counter.hpp                               18                18     0.00%         192               192     0.00%           2                 2     0.00%
ylt/metric/detail/ckms_quantiles.hpp                  9                 9     0.00%         127               127     0.00%           0                 0         -
ylt/metric/detail/time_window_quantiles.hpp           4                 4     0.00%          25                25     0.00%           0                 0         -
ylt/metric/gauge.hpp                                  5                 5     0.00%          31                31     0.00%           0                 0         -
ylt/metric/histogram.hpp                              5                 5     0.00%          55                55     0.00%           0                 0         -
ylt/metric/metric.hpp                                25                25     0.00%         128               128     0.00%           8                 8     0.00%
ylt/metric/summary.hpp                               10                10     0.00%          98                98     0.00%           0                 0         -
ylt/util/concurrentqueue.h                           79                28    64.56%        1568              1049    33.10%         290               198    31.72%
ylt/util/expected.hpp                                 6                 4    33.33%           6                 4    33.33%           0                 0         -
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                               763               187    75.49%        9414              3335    64.57%        2478               904    63.52%

@@ -145,6 +145,25 @@ class coro_http_request {

bool is_req_ranges() { return parser_.is_req_ranges(); }

std::string get_accept_encoding() {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

返回类型改成std::string_view

@@ -52,26 +52,63 @@ class coro_http_response {
}
void set_status_and_content(
status_type status, std::string content = "",
content_encoding encoding = content_encoding::none) {
set_status_and_content_view(status, std::move(content), encoding, false);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

std::string client_encoding_type的类型改成std::string_view

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已经修改

Copy link

Code Coverage Report
for detail, goto summary download Artifacts

Filename                                      Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
cinatra_log_wrapper.hpp                               4                 1    75.00%           9                 1    88.89%           0                 0         -
cookie.hpp                                           15                 2    86.67%          84                 2    97.62%          28                 3    89.29%
coro_http_client.hpp                                 83                 5    93.98%        1280               198    84.53%         410               114    72.20%
coro_http_connection.hpp                             36                 2    94.44%         667               208    68.82%         202                71    64.85%
coro_http_request.hpp                                29                 5    82.76%         201                24    88.06%          74                10    86.49%
coro_http_response.hpp                               31                 6    80.65%         225                68    69.78%          90                36    60.00%
coro_http_router.hpp                                 15                 0   100.00%         181                32    82.32%          20                 8    60.00%
coro_http_server.hpp                                 50                 5    90.00%         947               122    87.12%         178                47    73.60%
coro_radix_tree.hpp                                  17                 0   100.00%         264                38    85.61%         136                30    77.94%
define.h                                              2                 0   100.00%          28                 2    92.86%          20                 1    95.00%
http_parser.hpp                                      29                 5    82.76%         201                32    84.08%          56                12    78.57%
metric_conf.hpp                                       8                 4    50.00%          96                80    16.67%          32                28    12.50%
mime_types.hpp                                        1                 0   100.00%           7                 2    71.43%           2                 1    50.00%
multipart.hpp                                         4                 0   100.00%          89                15    83.15%          22                 5    77.27%
picohttpparser.h                                     14                 6    57.14%         467               278    40.47%         212               103    51.42%
response_cv.hpp                                       2                 0   100.00%          55                32    41.82%          46                16    65.22%
session.hpp                                          11                 0   100.00%          54                 2    96.30%           4                 1    75.00%
session_manager.hpp                                  10                 1    90.00%          67                 5    92.54%          10                 1    90.00%
sha1.hpp                                             12                 0   100.00%         181                 8    95.58%          12                 2    83.33%
string_resize.hpp                                     2                 0   100.00%          18                 0   100.00%           2                 0   100.00%
time_util.hpp                                        20                 7    65.00%         172                70    59.30%          28                 2    92.86%
uri.hpp                                              17                 5    70.59%         200                62    69.00%         166                52    68.67%
url_encode_decode.hpp                                 4                 0   100.00%          85                20    76.47%          48                16    66.67%
utils.hpp                                            13                 0   100.00%         201                22    89.05%         116                43    62.93%
websocket.hpp                                        15                 4    73.33%         170                43    74.71%          70                18    74.29%
ylt/coro_io/channel.hpp                              15                 0   100.00%         106                 7    93.40%          30                 5    83.33%
ylt/coro_io/client_pool.hpp                          21                 3    85.71%         324               125    61.42%          62                38    38.71%
ylt/coro_io/coro_file.hpp                            22                 1    95.45%         213                38    82.16%          56                19    66.07%
ylt/coro_io/coro_io.hpp                              52                 6    88.46%         329                38    88.45%           8                 2    75.00%
ylt/coro_io/detail/client_queue.hpp                  10                 5    50.00%          47                23    51.06%          10                 4    60.00%
ylt/coro_io/io_context_pool.hpp                      38                 6    84.21%         216                29    86.57%          28                 8    71.43%
ylt/metric/counter.hpp                               18                18     0.00%         192               192     0.00%           2                 2     0.00%
ylt/metric/detail/ckms_quantiles.hpp                  9                 9     0.00%         127               127     0.00%           0                 0         -
ylt/metric/detail/time_window_quantiles.hpp           4                 4     0.00%          25                25     0.00%           0                 0         -
ylt/metric/gauge.hpp                                  5                 5     0.00%          31                31     0.00%           0                 0         -
ylt/metric/histogram.hpp                              5                 5     0.00%          55                55     0.00%           0                 0         -
ylt/metric/metric.hpp                                25                25     0.00%         128               128     0.00%           8                 8     0.00%
ylt/metric/summary.hpp                               10                10     0.00%          98                98     0.00%           0                 0         -
ylt/util/concurrentqueue.h                           79                28    64.56%        1568              1049    33.10%         290               198    31.72%
ylt/util/expected.hpp                                 6                 4    33.33%           6                 4    33.33%           0                 0         -
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                               763               187    75.49%        9414              3335    64.57%        2478               904    63.52%

Copy link
Owner

@qicosmos qicosmos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@qicosmos qicosmos merged commit 2b0b5d8 into qicosmos:master Jun 12, 2024
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants