Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Invalid character in header content errors, probably due to x-error #201

Closed
trieloff opened this issue Apr 7, 2021 · 8 comments
Closed

Invalid character in header content errors, probably due to x-error #201

trieloff opened this issue Apr 7, 2021 · 8 comments
Assignees
@trieloff

Comments

@trieloff
Copy link
Contributor

trieloff commented Apr 7, 2021

We are seeing an increase of net new errors since April 5th, across many actions. The common thread is "Invalid character in header content", e.g.

  • from word2md: "message":"Invalid character in header content [\"<!DOCTYPE html><html xml:lang=\"en\" xmlns=\"http://www.w3.org/1999/xhtml\" xmlns:Web=\"http://schemas.live.com/Web/\"><head><meta http-equiv=\"Content-Type\" content=\"text/html; charset=UTF-8\"/><title>Sharepoint Online</title><meta http-equiv=\"X-UA-Compatible\" content=\"IE=edge,chrome=1\"/><meta name=\"viewport\" content=\"width=device-width, initial-scale=1\"/></head><body><style>\n body,html {\n font: 18px/normal \"Segoe UI\",Arial,Helvetica,Sans-Serif;\n line-height: 23px;\n color: #333333;\n background: #F3F3F3 !important;\n }\n a {\n color: #3366BB;\n text-decoration: none;\n white-space: nowrap;\n }\n #sw_content {\n width: 760px;\n height: auto;\n margin: 0 auto;\n padding: 0;\n margin-bottom: 150px;\n background: #F3F3F3;\n }\n .panda { margin-top: 60px;}\n .panda img {\n width: 350px;\n height: 223px;\n }\n\n .title {\n font-size: 38px;\n line-height: 48px;\n color: #777777;\n text-align: center;\n margin-top: 30px;\n font-family: \"Segoe UI\",Arial,Helvetica,Sans-Serif;\n }\n .sc_errD {\n font-size: 13px;\n line-height: 16px;\n color: #777777;\n text-align: left;\n margin-top: 20px;\n white-space: nowrap\n }\n .sc_error {\n margin-top: 10px;\n padding: 0 20px;\n }\n .sc_error p{ margin: 18px 0px; }\n .bing_logo { margin-top: 25px; }\n .panda, .bing_logo, .sc_error{ text-align: center;}\n #b_header { display: none; }\n a:hover{text-decoration: underline;}\n\n @media(max-width: 736px){\n body{font-family: HelveticaNeue, Roboto, Arial, sans-serif;}\n .panda { margin-top: 60px;}\n .panda img {\n width: 225px;\n height: 143px;\n }\n .title {\n margin-top: 20px;\n font-family: HelveticaNeue, Roboto, Arial, sans-serif;;\n font-size: 23px;\n line-height: 28px;\n color: #333333;\n }\n .sc_errD, .sc_error p {\n font-size: 13px;\n line-height: 16px;\n }\n .sc_error { margin-top: 5px; }\n .sc_error p{ margin: 0px;}\n .bing_logo { margin-top: 20px; }\n #sw_content{ width: auto;}\n }\n</style><div id=\"sw_content\"><div class=\"panda\" data-tag=\"bing_panda\"><img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAK4AAACuCAYAAACvDDbuAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAD8ZJREFUeNrsnd9uFEcWxs+AjWBD2DHsxmRBME4gCpY2GRtEVlokZnKVmwQj5X49b+B5ApsngDyBh3skz+aKXCx2JC4ARdhJtFEgJAyBxNib4IEAdmyCt05Tbdrt6arqv+6Z/n7SyAbP3+5vTp3z1alqIgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAoAPIhX2C8fHxgvgxLG4nxa0hbucrlcoUDi1IrXCFaIf4h7jlXX+qCfFWcHhBXGwJIVqOshMtRMsMi7+fxeEFqYq4QpTjMj3QwYG3hsMMNlW4Qq95GWVLPh42IMQ7g0MNNiVVkEXYpE/RMpPisUUcapB4xJXCm/TIZ03giFsWkbeJQw489DVKLxypeiQRVxZh0yFEyxRligGAW19jUl9DfjSyxeBJxyN6jyVZ1AHA2mI93JaR1sZ4RO5SFGFnDZ0DP7BN9qUYDs7h1GVWsCpt1QJHXPnEkzGI1uasTD9A9kTL6cBthbbOByrOZJLMeUYh5s/QlMUabLJsCLYgU86SqoAXehjwHXEdzkHcoqXdu3fnP/zww8nV1dU8Tmtn83Rxaaxn9+5p0tuo5/08rzNVmAjpHJiKlj744APas2ePlZJAvB0r2JK4sWBHy+Vyft/+/bqH1Pw8f84Rym/H/WEOHTpEJ06ccP93PZfLncap7hjB5qVTMOL+2/T0dbp540arh9VFmuBLA3bEjT09KBaL60T7zTff0MzMDC0vLw+JqIuGnM4QLRdf061EywwMDNLx9/4ROk1gbDsstiJp27ZtdPz4cSva2ty6dYuuXbtm/f748WMW9IgQ75ci8tZw+ttSsBz4OPgM6e7b19dHPT15unTpEq0sL1uFuuls2YaIK6dia3GIlvNZp2hVxacQbwkyaDvRjtDLmS8j8vkeEsU55Xt6KKjunBMQVXoxNRtJQwwXYe+//z7t3LlzLbJypGUR802kCNatv79/7TF//PHHxPUvvigPHjsGmyz9gi3KKBso2HR3b7P0cf/+/fNBHu/2cfOySAtV6R84cMDKZzniMizQCxcuWD9ZyB9//PGGxwjR0nc3b9Li06cNToeEeNGQk97ia4TWT9UGZeZPO7YPBHngupkzmTKUyceccSvngL9JtmjtaLv8Ip9Z++lEiJX++/XX1k9ZKE5CIqkUbUmmBaMRPeX5oA9s2dYop2R9N8RwlHXms2IYsCIs39hB4H9zasAR2eZhs0l3Gg0r4rqoiaiLdWvpibLjfvJYQ3pExG1GJlwp3hGZwxgVYSxapyC5avzxxx+tv3300Udrua6T/83P0727d1VPfUaIdwzS2VTRDksdRD1RVBeiDezfe7Y1yg4ubcXHgmTnwClaO9raqQGnCm4uX75MFz/7rFWkdTIqirVhyGdTBFsQt0lqvYp7U9MEZcR1RN5pL6fBnr51FmF2BGYHgScZWNic89rwfTga28LesX07vfnGG7R161avt2Dl3XAaEhXtWIR5bMtzKqJtT5gn6DK4T1kWS0V3EcYTC7ZoOS1gQTK2d+v2bzny8n0ePHjwsjBbWqK79+5R4eBBr9e3ehpE5GWnoQFZxYc4xvnX/7ZvkiKyRBXUwj6BdumOdBoqTqfBnr51OgdOMdrR1An//dNPP113v7UC7dEj+nl2VvU2rNXFfGAhr1gZn/35p+L8/dm4X+d87MKV4p2RkdcSLAvXzd69ey0h882d73LawKJtZYWtFWq//EIPFhZUbwPr1uKNtmtTtg9+/ZV+EiPoc3X9EZSGSBNCp32+9lVYWFgYzufz43auevHiRSuCum0wJ2yD8c0Uznd3vvKKcpiBTRa5aIephf25XdQf+w4cpO7u7ihfriqEG3rplq8tmHp6ejg3OWcP/fawzxG1Fewc+BGt9XW8c8fKexUMw2mIVLT21O0GlsR5+P7mDWo2F6J8yXoUT+J777BcLsc9DTV2C+wcl90FJ6urq1Zq4CVoFWyPsXg1Ntm4OOBDkF34Yow0+2V8/8MP9JkYWW/fjqRdm73bSArsQHuHyVULkyJdKLJT4BTus2fP6OHCQ/r2xrf03a3vAr8xtsneOnxYaakQbLIoROvpILDb46w7uCXRo5/WlIoQbi2K9x9ot0YRdS3RiIjbdIp2ZXmFFh4sWNHy8KHDtH/f/sBvzLbJ4DTExlmVaFmw7mKZo+7kpf/QyspykNdrRiXawMJ1ildGPlpaXLJEu/p8de0+R44coV27dgV+c3zg5ubnVXcp0AuPF+L1F23HSLH9wOMnTzyDxrw4H+zFB8h761F+hi1hHizEy8N05dHDR8Q3N91d3fTe8fdox44dgV/j/tyciU2GpT/mouXaYFQ10nGNoQyd4nyweOfVQcXNJ6kRrhRvXURbT3uKxXt08GgoS4UnJwychjHI0shBGFcVxnfv3tUVxjItXLbShps3bxiZRVF4t5EKl+l9vZdzF8/8Zderu+jowNHAz88Hkqtb1QQGoSHHpBhTbkHA6YEmQGxg+vp1unb1SqLRNjLhSvFy1J3y+jsXce/8/Z1Q4jW0ybAXb2uUm73wqMZT70Hgom1metqyQZPIbyMVroT7Kz2HBHYZwjoNuvxLFmsQ7/poO65zEHjKPSh7du+hN/veoKe/PW41TRyZdxubcEXUbUrxena1c9Tt7e0N/BqqileSl5EXTsML0Y6oHAQD21EJu0aDg4PW78+fP6enj5/Qs5UV513+Hcfnijrisnj521VW3YfFG9YmQ0OOkWhLKseFawauHYLCBTefSy7AbThdWHzylJaXfrcMiCi921iFK8Vr2WRKp2EgnNPAUUKTk5XkEJllB2EiZM2ghM8hF96t+F1EchF9P4nr822J64ml03DG6+/s7bLHG1a8BjbZSEYdBOWSGwOLUUn/kf4NPSouan/5656xuD5jLu6DODc7p7wm2r2f7tFXX38V+Pm50eetQ4dUS3+sonHw2LF6kuKR27by7WTSV9kUwlVeHYlnI3liJyhcYGscIqt/W9Y87RVxHVR1TgN/e4Ni52lpssmEaDkBn5ZRr7AJDkJJVR+EES3XJhrRWgV6nKJNRLjyA3Cx5mmJFA4WQttkBkt/kuxpcH5RGwmKdljnIGiOkxI7vdNQlgU6tbVwHeLV2mQxN+QkKd7PHb/fSUi02ulcg5FJ6SBYU/ddypqkIgtz6gjhOpwG5QYQ/G0OI17DhpwknAZnPj2VgGg5HVFuWxVGtGuB5VXluTknC3LqKOFK8fJJVNpkli8Yr9MwJBcGxoZcXNpMIlWIqwfBCfdW976mnDSqi3NbTVJLiQpXipe/lZ7fzLANOXZ00ZyokQQacupSxA1FERdF2qKczjVYPa11EFi4mnw+8cWriQtXirdCisaLKBpyDNrzxuXMUpx5ri5NGBXinQgqYOeS8lYY7FehdRB4MYDGQajE7SCkRrj2iEoam6xQKIRyGgwaciZitMmmDNKEYSm8Sb/ilSPGiOrzh+lB4HTNmiBSF2OnkyrGUiNcE6eh/+3+UDbZZjbkyBThE0WaMOTISwvkw+9VLSm3R5ww07mGoq3ImoUyJVwpXj65yo2ko1i3pmnZK1JMG0lrrpx5yjHcGl9l02RJOYtW03Sv5MjbR3QOQi1JByF1wpXi5RNWVTkNYXsaDJqki0EbcuQ14oIwFIdoeYThkSYonJ5pRrkpWaNQpoXrcBpiFW8cDTmyH2Hab34q0wQi/9cz9r2k3Nd56O210jMFDdJ48ZkSrhSvciNpHrrCOg0GJvxZU5vMcZX5PPnfYv6UX9GGWVJu6iCkoQeh7YQrxat0GtgED9OQ40O8Jk5DyTFkn/L5Vs74FG3oJeW6Yszqj07JdK4JOUoZc7Nz2q2BuA2S2yGD8mcRXRQbSdvRRbuRtEwVJqQj0CP3Eo4U+SXyzGvtL2OYmbET/zyhK8aqckRMDVvSJlw5FFVI05DDC/SCEtVG0jJq8nW6pij6K9IkMp1r0INQS5toUylch9OgLAJ4gV4Ym8xwI2mt08BRVtzKFE8zTWxLyhmDVdczaXAQ2iJVcKUNwyrxPPrtEV29dpVW1q8q9UVaN5KW9tywykEIU4zxiKXpreURry8txVhbRFxH5GWX4ZzKaTBobFaSxo2kk1xSrhBtOa2iTb1wpXitjaRV4u2AhhynaPl1El1S7lGMpXrf4dQL1z6QpGnI0bTeKeEIZiCGibjXrW32knLJmc2ezu0Y4TrWrXkOXe2+kXRalpSLYz3WDppol4hrJN4230h6ghTeNb+vsA3hvChV5SCQYtodwg0nXu0OOe24kXRWlpRnVrhSvHXSNOS000bSWVpSnmnhSvEm0pAT90bSWVtSnnnhSvEqN5Lmhpw0bySdxSXlEO5LYt9I2sDon5Qi9OsgZG5JOYS73mnQNuSE2UiaewFisMkyuaQcwt3oNLTNRtJZXlIO4bYWb+wbSWvWcmk3knYtKW/KArPmMzVRFmNpXlIO4bYWLwsg1o2kDRtySgoHgaNtg17YeX2y6+yOj2IwrGg3dUk5hOst3jHS2GS89Dpmp6HkUYzx0puqECsL9py4Nd1fik5fUh4lXdR5VGXOWfQqXBYXFwNf2d3uzuI+Xo9d0N91/4cU6ekQaYiSdllSjoirdxqUG0lH0ZCjKKBKfnNnReHXVDkm1udtoyXlEK6ZeDdrI+m8n0kJ8TwND8GekV/AvMpB6LQehEwL18RpYGJsyPETdd3C5Tx0oFKpjJHC623HJeXIcc3FW5+bnWPxtrSp7IacAOvWuNGHr5hYEEP9uyLyFbq2bi2q8lwDuNqvuvZbOBXiS1eVDUkE4baneGtCvCfJowPL3kj6yrUrfp62KAR2uoV7UCCfOy/KiMu72kyZRu52XVIeJTnKCEK8PLPlOWMV4HprXmKLBLm/2EQrV0R3jTEh2oFOP59bKDtEvZH0v2J+vyfd/8FLyg2KsXIWTmZmhGu6kXSLhhyvqDoU81secjsI7b6kHMINLt4GadattbDJ2Jbqo40zcnnHdqFRpwkFmfvW5etXe1/rLYtikoWZo9bXz6h2qoOQ6RzXle8Ok2LlwcqzFZr6fMp2GtZyWSmoUUehV0v6Or3y/buv1XumXVbnIuKGdBpI0ZDj2ki6sZYkVyoNKVQ7Ag+l4OPUsibazEZcR+TSXtn96LGjOc2QrryWWcwRN/arlEO46RUvX+W8qIjOadxDeFXm6QPttjoXqUJ0cLF2jhRWWVrfd1ZFy3RlXbVymK3KSJaXQ/BJ+bOQ0rddyZKDAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAm/F+AAQA1GwLUNH600gAAAABJRU5ErkJggg==\"/></div><div class=\"title\">Something went wrong</div><div class=\"sc_error\"><p>This is a temporary issue, so try again in a few minutes.</p></div>\n<div class=\"sc_errD\" data-tag=\"refcode\"><p>Technical details:</><p>Ref A: 75D8FEABF1DD42C790B69603BB369A71 Ref B: BL2EDGE1417 Ref C: 2021-04-07T01:28:16Z</p></div><div class=\"bing_logo\"></div></div><!--AppServer--></body></html></body></html>\"]"
  • from dispatch: "message":"Invalid character in header content [\"[1] Error invoking pages_4.23.6/html@(adobe/theblog/master/ko/authors/어도비코리아.md): 502 \"]"
  • from content-proxy: "message":"Invalid character in header content [\"Invalid character in header content [\"Unable to fetch adobe/theblog/master/ko/authors/어도비코리아 (500) from word2md: unexpected response: no headers. is: { statusCode: 500 }\"]\"]"

I suggest we crop and sanitize the x-error header values before setting them.
@trieloff
Copy link
Contributor Author

trieloff commented Apr 7, 2021

Most likely thrown in

https://github.com/adobe/helix-fetch/blob/8545155125e604534ffa38ea1c779ffeeb94046f/src/fetch/headers.js#L49-L52

@trieloff trieloff self-assigned this Apr 7, 2021
@trieloff
Copy link
Contributor Author

trieloff commented Apr 7, 2021

It looks like the offending code is not in the adapter, but in the actual action code. I'd like to provide a small helper function that the actions can use and could think of two places to put it:

  1. into the context object provided by helix-deploy
  2. in helix-shared

(1) would have the advantage of being dependency free.
(2) would have the advantage of being the more common way of doing things

@tripodsan
Copy link
Contributor

tripodsan commented Apr 7, 2021 •
edited
Loading

good idea: ``` context.sanitizeHeader(e) ```

@tripodsan
Copy link
Contributor

It looks like the offending code is not in the adapter, but in the actual action code. I'd like to provide a small helper function that the actions can use and could think of two places to put it:

  1. into the context object provided by helix-deploy
  2. in helix-shared

what about:

  1. in helix-fetch ?

@tripodsan
Copy link
Contributor

probably in https://github.com/adobe/helix-shared/blob/main/src/utils.js would be a good location.

@tripodsan
Copy link
Contributor

(1) would have the advantage of being dependency free.

ask @koraa about hidden dependency injection :-)

@stefan-guggisberg
Copy link
Contributor

what about:
3. in helix-fetch ?

-1, helix-fetch behaviour is spec compliant and in line with browser fetch and node-fetch.

@trieloff
Copy link
Contributor Author

trieloff commented Apr 7, 2021 

value.replace(/[^\t\u0020-\u007E\u0080-\u00FF]/g, '').substr(0,1024)

I will add it as a private function in the wrapper and to helix-shared.

trieloff added a commit that referenced this issue Apr 7, 2021
@trieloff
fix(template): clean up x-error header value prior to sending response
2e03414 
see #201 (not a complete fix)
@trieloff trieloff mentioned this issue Apr 7, 2021
Merged
trieloff added a commit to adobe/helix-shared that referenced this issue Apr 7, 2021
@trieloff
feat(utils): add headercleanup helper function
4d28f25 
see adobe/helix-deploy#201
@trieloff trieloff mentioned this issue Apr 7, 2021
Merged
github-actions bot pushed a commit that referenced this issue Apr 7, 2021
@semantic-release-bot
chore(release): 3.18.1 [skip ci]
9c7bfb9 
## [3.18.1](v3.18.0...v3.18.1) (2021-04-07)

### Bug Fixes

* **template:** clean up x-error header value prior to sending response ([2e03414](2e03414)), closes [#201](#201)
adobe-bot pushed a commit to adobe/helix-shared that referenced this issue Apr 7, 2021
@semantic-release-bot
chore(release): 7.22.0 [skip ci]
d72a4de 
# [7.22.0](v7.21.14...v7.22.0) (2021-04-07)

### Features

* **utils:** add headercleanup helper function ([4d28f25](4d28f25)), closes [adobe/helix-deploy#201](adobe/helix-deploy#201)
trieloff added a commit to adobe/helix-dispatch that referenced this issue Apr 7, 2021
@trieloff
fix: clean up x-error values
d2e3c04 
see adobe/helix-deploy#201
@trieloff trieloff mentioned this issue Apr 7, 2021
Merged
trieloff added a commit to adobe/helix-content-proxy that referenced this issue Apr 7, 2021
@trieloff
fix(utils): ensure x-error header value is clean
876728f 
see adobe/helix-deploy#201
@trieloff trieloff mentioned this issue Apr 7, 2021
Merged
tripodsan pushed a commit to adobe/helix-content-proxy that referenced this issue Apr 8, 2021
@trieloff
fix(utils): ensure x-error header value is clean (#363)
30b6cd9 
see adobe/helix-deploy#201
trieloff pushed a commit to adobe/helix-dispatch that referenced this issue Apr 8, 2021
@semantic-release-bot
chore(release): 4.8.1 [skip ci]
467a7cc 
## [4.8.1](v4.8.0...v4.8.1) (2021-04-08)

### Bug Fixes

* clean up x-error values ([d2e3c04](d2e3c04)), closes [adobe/helix-deploy#201](adobe/helix-deploy#201)
trieloff pushed a commit to adobe/helix-content-proxy that referenced this issue Apr 8, 2021
@semantic-release-bot
chore(release): 2.12.4 [skip ci]
cabf1af 
## [2.12.4](v2.12.3...v2.12.4) (2021-04-08)

### Bug Fixes

* **utils:** ensure x-error header value is clean ([#363](#363)) ([30b6cd9](30b6cd9)), closes [adobe/helix-deploy#201](adobe/helix-deploy#201)
@trieloff trieloff closed this as completed Apr 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@trieloff trieloff

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@trieloff @tripodsan @stefan-guggisberg