bugfix:ngx.encode_args() not escaped "|" to "%7c". see: https://en.wi… #542

goecho · 2015-08-05T07:48:49Z

…kipedia.org/wiki/Percent-encoding

agentzh · 2015-08-05T09:13:20Z

src/ngx_http_lua_util.c

-    static uint32_t  *map[] =
-        { uri, args, html, refresh, memcached, memcached };
-
+	static uint32_t  *map[] =


Please fix the indentation.

agentzh · 2015-08-05T09:26:11Z

@goecho From what I can see. You're escaping not only | but also quite some others like @, *, (, ), !, and etc. So your commit log message is inaccurate.

It's worth mentioning that even Google Chrome's JavaScript API function encodeURIComponent() does not escape all your new candidates:

> encodeURIComponent(",*()'$!@`|")
"%2C*()'%24!%40%60%7C"

We needn't be harsher than Google Chrome I think :)

agentzh · 2015-08-05T09:29:32Z

src/ngx_http_lua_util.c

@@ -2424,7 +2443,7 @@ ngx_http_lua_process_args_option(ngx_http_request_t *r, lua_State *L,

            if (total_escape) {
                p = (u_char *) ngx_http_lua_escape_uri(p, key, key_len,
-                                                       NGX_ESCAPE_URI);
+                                                       NGX_ESCAPE_URI_COMPONENT);


This line exceeds 80 columns (so do the following changes from you). Please fix them. Thanks.

agentzh · 2015-08-05T09:30:08Z

@goecho Will you add a simple test case to the existing test case to cover these changes? Thanks!

…ble with What Google Chrome's JavaScript API function encodeURIComponent() does. see:https://tools.ietf.org/html/rfc2396.

goecho · 2015-08-10T11:49:12Z

@agentzh I had added a simple test case to the existing test case to cover these changes.

100% compatible with What Google Chrome's JavaScript API function encodeURIComponent() does.

see:https://tools.ietf.org/html/rfc2396.

>encodeURIComponent(",$@|`")
"%2C%24%40%7C%60"
>encodeURIComponent("-_.!~*'()")
"-_.!~*'()"

agentzh · 2015-08-11T04:02:12Z

@goecho Cool, thanks. I'll look into this when I have a chance.

Percent-encode query args when re-attaching them to the `upstream_uri`. Since `ngx.encode_args` does not perform percent-encoding on various reserved characters, this implements a custom `utils.encode_args` function which uses LuaSocket's `url.encode` function. It tries to mimic the `ngx.encode_uri` behaviour 100%. Ideally, `ngx.encode_args` would proceed to the percent-encoding itself (see openresty/lua-nginx-module#542). This also makes some perf and style changes. Fix #749

Tieske · 2016-08-18T10:05:36Z

What's the status on this? We still got workarounds in place waiting for this to gets fixed....

toruneko · 2017-07-04T02:27:23Z

2 years ago...
What's the status on this?

toruneko · 2017-07-04T02:38:16Z

It seems that the same code with function ngx_escape_uri from file ngx_string.c.

agentzh · 2017-07-04T06:48:00Z

OK, I've merged a slightly modified version of this patch. Sorry for the long delay on my side.

snpcp · 2021-12-13T08:21:18Z

Hello, I using the ngx.escape_uri and ngx.encode_args function doesn't look right.

This is RFC 3986 : https://datatracker.ietf.org/doc/html/rfc3986#appendix-A
[[
reserved = gen-delims / sub-delims
gen-delims = ":" / "/" / "?" / "#" / "[" / "]" / "@"
sub-delims = "!" / "$" / "&" / "'" / "(" / ")"
/ "*" / "+" / "," / ";" / "="
]]

This is my code:
print(ngx.encode_args({a="1,2!3^4_5;6"}));
print(ngx.escape_uri("1,2!3^4_5"));

This is output:
a=1%2C2!3%5E4_5%3B6
a%3D1%2C2!3%5E4_5%3B6

This is Openresty Version:
resty 0.23
nginx version: openresty/1.15.8.2
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-36) (GCC)
built with OpenSSL 1.0.2k-fips 26 Jan 2017

The expected output is '1,2!3%5E_5;6' by RFC 3986, however that's not true.

zhuizhuhaomeng · 2021-12-14T02:36:56Z

@snpcp according to the rfc you given, the "," should be escaped.
but the expected output is not escaped. They are contradictory

snpcp · 2021-12-14T06:05:11Z

In the fact, Many software doesn't strictly follow this standard(about gen-delims and sub-delims), And the Openresty look like the same strategy as encodeURIComponent of JavaScript.
That being the case, I accepted. thanks for your response to me.

bugfix:ngx.encode_args() not escaped "|" to "%7c". see: https://en.wi…

1bf4441

…kipedia.org/wiki/Percent-encoding

agentzh reviewed Aug 5, 2015
View reviewed changes

goecho added 2 commits August 10, 2015 19:26

bugfix:ngx.encode_args() not escaped "|",",","$","@","`";100% compati…

5e3992e

…ble with What Google Chrome's JavaScript API function encodeURIComponent() does. see:https://tools.ietf.org/html/rfc2396.

add a simple test case on ngx.encode_args()

b226e97

thibaultcha mentioned this pull request Dec 1, 2015

Incorrectly decoding query parameters Kong/kong#749

Closed

thibaultcha mentioned this pull request Dec 1, 2015

fix(resolver) percent-encode querystring before proxying Kong/kong#752

Merged

Merge remote-tracking branch 'openresty-lua-nginx-module/master'

fb3ef85

agentzh force-pushed the master branch 2 times, most recently from e7ac10c to cfd4f90 Compare January 31, 2016 19:04

agentzh force-pushed the master branch from ab60f70 to 397f366 Compare October 31, 2016 23:56

toruneko mentioned this pull request Jul 4, 2017

Why not escape "|" on URI #1103

Closed

agentzh closed this Jul 4, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bugfix:ngx.encode_args() not escaped "|" to "%7c". see: https://en.wi… #542

bugfix:ngx.encode_args() not escaped "|" to "%7c". see: https://en.wi… #542

goecho commented Aug 5, 2015

agentzh Aug 5, 2015

agentzh commented Aug 5, 2015

agentzh Aug 5, 2015

agentzh commented Aug 5, 2015

goecho commented Aug 10, 2015

agentzh commented Aug 11, 2015

Tieske commented Aug 18, 2016

toruneko commented Jul 4, 2017

toruneko commented Jul 4, 2017

agentzh commented Jul 4, 2017

snpcp commented Dec 13, 2021 •

edited

Loading

zhuizhuhaomeng commented Dec 14, 2021

snpcp commented Dec 14, 2021

bugfix:ngx.encode_args() not escaped "|" to "%7c". see: https://en.wi… #542

bugfix:ngx.encode_args() not escaped "|" to "%7c". see: https://en.wi… #542

Conversation

goecho commented Aug 5, 2015

agentzh Aug 5, 2015

Choose a reason for hiding this comment

agentzh commented Aug 5, 2015

agentzh Aug 5, 2015

Choose a reason for hiding this comment

agentzh commented Aug 5, 2015

goecho commented Aug 10, 2015

agentzh commented Aug 11, 2015

Tieske commented Aug 18, 2016

toruneko commented Jul 4, 2017

toruneko commented Jul 4, 2017

agentzh commented Jul 4, 2017

snpcp commented Dec 13, 2021 • edited Loading

This is RFC 3986 : https://datatracker.ietf.org/doc/html/rfc3986#appendix-A [[ reserved = gen-delims / sub-delims gen-delims = ":" / "/" / "?" / "#" / "[" / "]" / "@" sub-delims = "!" / "$" / "&" / "'" / "(" / ")" / "*" / "+" / "," / ";" / "=" ]]

This is my code: print(ngx.encode_args({a="1,2!3^4_5;6"})); print(ngx.escape_uri("1,2!3^4_5"));

This is output: a=1%2C2!3%5E4_5%3B6 a%3D1%2C2!3%5E4_5%3B6

This is Openresty Version: resty 0.23 nginx version: openresty/1.15.8.2 built by gcc 4.8.5 20150623 (Red Hat 4.8.5-36) (GCC) built with OpenSSL 1.0.2k-fips 26 Jan 2017

zhuizhuhaomeng commented Dec 14, 2021

snpcp commented Dec 14, 2021

snpcp commented Dec 13, 2021 •

edited

Loading

This is RFC 3986 : https://datatracker.ietf.org/doc/html/rfc3986#appendix-A
[[
reserved = gen-delims / sub-delims
gen-delims = ":" / "/" / "?" / "#" / "[" / "]" / "@"
sub-delims = "!" / "$" / "&" / "'" / "(" / ")"
/ "*" / "+" / "," / ";" / "="
]]

This is my code:
print(ngx.encode_args({a="1,2!3^4_5;6"}));
print(ngx.escape_uri("1,2!3^4_5"));

This is output:
a=1%2C2!3%5E4_5%3B6
a%3D1%2C2!3%5E4_5%3B6

This is Openresty Version:
resty 0.23
nginx version: openresty/1.15.8.2
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-36) (GCC)
built with OpenSSL 1.0.2k-fips 26 Jan 2017