Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for Blob #2229

Merged
merged 1 commit into from
Jul 2, 2024
Merged

Add support for Blob #2229

merged 1 commit into from
Jul 2, 2024
+775 −93

Conversation

lpinca
Copy link
Member

@lpinca lpinca commented Jun 11, 2024

Closes #2206

@lpinca lpinca force-pushed the support/blob branch 7 times, most recently from 223c6fc to a83321b Compare June 13, 2024 17:53
@lpinca
Copy link
Member Author

lpinca commented Jun 13, 2024

@kettanaito I'm still not very happy with this because, other than compatibility with the WHAWG spec, it is pretty much useless. Anyway, here it is. I also didn't run any benchmarks to see the impact on performance.

kettanaito
kettanaito reviewed Jun 14, 2024
View reviewed changes
lib/websocket.js
@@ -1030,6 +1031,11 @@ function initAsClient(websocket, address, protocols, options) {
*/
function emitErrorAndClose(websocket, err) {
websocket._readyState = WebSocket.CLOSING;
//
// The following assignment is practically useless and is done only for
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which summarizes the entire change pretty flawlessly. I am still grateful for this consistency! Thank you!

@lpinca lpinca merged commit 59b9629 into master Jul 2, 2024
101 checks passed
@lpinca lpinca deleted the support/blob branch July 2, 2024 15:50
@titanism
Copy link

titanism commented Jul 4, 2024

Hi there @lpinca, I can't confirm exactly what's wrong here, but I did a minor version bump and this entirely broke our implementation of ws. It's sending everything over the wire as a BLOB now instead of as a Buffer it seems. Seems to be a major breaking change (e.g. we are using msgpackr under the hood to pack/unpack messages sent via websockets, and now they are throwing an error Source must be a Uint8Array or Buffer but was a Blob).

@titanism titanism mentioned this pull request Jul 4, 2024
Closed
1 task
@lpinca
Copy link
Member Author

lpinca commented Jul 4, 2024 •
edited

@titanism are you are setting websocket.binaryType to 'blob'? Let's continue the discussion in #2239.

This was referenced Jul 24, 2024
Open
Open
Open
@Exkaleburx Exkaleburx mentioned this pull request Jul 24, 2024
Open
@FaroukAmr FaroukAmr mentioned this pull request Jul 25, 2024
Open
@taeb3 taeb3 mentioned this pull request Jul 25, 2024
Open
@RosaleeKnight RosaleeKnight mentioned this pull request Jul 25, 2024
Merged
@heyvaldemar heyvaldemar mentioned this pull request Jul 25, 2024
Merged
@j-mendez j-mendez mentioned this pull request Jul 25, 2024
Open
@epacke epacke mentioned this pull request Jul 25, 2024
Merged
@rater193 rater193 mentioned this pull request Jul 25, 2024
Open
@KDONJS KDONJS mentioned this pull request Jul 25, 2024
Merged
This was referenced Jul 25, 2024
Merged
Merged
@Alex-Werner Alex-Werner mentioned this pull request Jul 25, 2024
Open
This was referenced Jul 26, 2024
Open
Open
@OKEAMAH OKEAMAH mentioned this pull request Aug 1, 2024
Merged
@kisuka kisuka mentioned this pull request Aug 4, 2024
Open
This was referenced Aug 7, 2024
Merged
Merged
@neznaamey neznaamey mentioned this pull request Aug 12, 2024
Open
nikwen
nikwen reviewed Aug 15, 2024
View reviewed changes
Copy link

@nikwen nikwen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was just looking at the commit to make sure that upgrading ws won't break my app. Looks great overall. Just had a small suggestion. :)

lib/websocket.js
Comment on lines +1198 to +1199
websocket._errorEmitted = true;
websocket.emit('error', err);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems worthwhile to pull out into a small function:

function emitError(err) {
  websocket._errorEmitted = true;
  websocket.emit('error', err);
}

Seems like these lines should always be called together. Pulling them out into a function will make it less likely that someone will forget to do so in the future.

@tim2zg tim2zg mentioned this pull request Aug 16, 2024
Merged
@OKEAMAH OKEAMAH mentioned this pull request Aug 18, 2024
Open
OKEAMAH added a commit to OKEAMAH/ethers.js that referenced this pull request Aug 19, 2024
@OKEAMAH
[Snyk] Upgrade ws from 8.17.1 to 8.18.0 (#8)
8710c31 
![snyk-top-banner](https://github.com/andygongea/OWASP-Benchmark/assets/818805/c518c423-16fe-447e-b67f-ad5a49b5d123)


<h3>Snyk has created this PR to upgrade ws from 8.17.1 to 8.18.0.</h3>

:information_source: Keep your dependencies up-to-date. This makes it
easier to fix existing vulnerabilities and to more quickly identify and
fix newly disclosed vulnerabilities when they affect your project.

<hr/>


- The recommended version is **1 version** ahead of your current
version.

- The recommended version was released on **a month ago**.



<details>
<summary><b>Release notes</b></summary>
<br/>
  <details>
    <summary>Package name: <b>ws</b></summary>
    <ul>
      <li>
<b>8.18.0</b> - <a
href="https://github.com/websockets/ws/releases/tag/8.18.0">2024-07-03</a></br><h1>Features</h1>
<ul>
<li>Added support for <code>Blob</code> (<a class="issue-link
js-issue-link" data-error-text="Failed to load title"
data-id="2347258138" data-permission-text="Title is private"
data-url="websockets/ws#2229"
data-hovercard-type="pull_request"
data-hovercard-url="/websockets/ws/pull/2229/hovercard"
href="https://github.com/websockets/ws/pull/2229">#2229</a>).</li>
</ul>
      </li>
      <li>
<b>8.17.1</b> - <a
href="https://github.com/websockets/ws/releases/tag/8.17.1">2024-06-16</a></br><h1>Bug
fixes</h1>
<ul>
<li>Fixed a DoS vulnerability (<a class="issue-link js-issue-link"
data-error-text="Failed to load title" data-id="2355202628"
data-permission-text="Title is private"
data-url="websockets/ws#2231"
data-hovercard-type="pull_request"
data-hovercard-url="/websockets/ws/pull/2231/hovercard"
href="https://github.com/websockets/ws/pull/2231">#2231</a>).</li>
</ul>
<p>A request with a number of headers exceeding the<a
href="https://nodejs.org/api/http.html#servermaxheaderscount"
rel="nofollow"><code>server.maxHeadersCount</code></a><br>
threshold could be used to crash a ws server.</p>
<div class="highlight highlight-source-js notranslate position-relative
overflow-auto" data-snippet-clipboard-copy-content="const http =
require('http');
const WebSocket = require('ws');

const wss = new WebSocket.Server({ port: 0 }, function () {
const chars =
&quot;!#$%&amp;'*+-.0123456789abcdefghijklmnopqrstuvwxyz^_`|~&quot;.split('');
  const headers = {};
  let count = 0;

  for (let i = 0; i &lt; chars.length; i++) {
    if (count === 2000) break;

    for (let j = 0; j &lt; chars.length; j++) {
      const key = chars[i] + chars[j];
      headers[key] = 'x';

      if (++count === 2000) break;
    }
  }

  headers.Connection = 'Upgrade';
  headers.Upgrade = 'websocket';
  headers['Sec-WebSocket-Key'] = 'dGhlIHNhbXBsZSBub25jZQ==';
  headers['Sec-WebSocket-Version'] = '13';

  const request = http.request({
    headers: headers,
    host: '127.0.0.1',
    port: wss.address().port
  });

  request.end();
});"><pre><span class="pl-k">const</span> <span
class="pl-s1">http</span> <span class="pl-c1">=</span> <span
class="pl-en">require</span><span class="pl-kos">(</span><span
class="pl-s">'http'</span><span class="pl-kos">)</span><span
class="pl-kos">;</span>
<span class="pl-k">const</span> <span class="pl-v">WebSocket</span>
<span class="pl-c1">=</span> <span class="pl-en">require</span><span
class="pl-kos">(</span><span class="pl-s">'ws'</span><span
class="pl-kos">)</span><span class="pl-kos">;</span>

<span class="pl-k">const</span> <span class="pl-s1">wss</span> <span
class="pl-c1">=</span> <span class="pl-k">new</span> <span
class="pl-v">WebSocket</span><span class="pl-kos">.</span><span
class="pl-c1">Server</span><span class="pl-kos">(</span><span
class="pl-kos">{</span> <span class="pl-c1">port</span>: <span
class="pl-c1">0</span> <span class="pl-kos">}</span><span
class="pl-kos">,</span> <span class="pl-k">function</span> <span
class="pl-kos">(</span><span class="pl-kos">)</span> <span
class="pl-kos">{</span>
<span class="pl-k">const</span> <span class="pl-s1">chars</span> <span
class="pl-c1">=</span> <span
class="pl-s">"!#$%&amp;'*+-.0123456789abcdefghijklmnopqrstuvwxyz^_`|~"</span><span
class="pl-kos">.</span><span class="pl-en">split</span><span
class="pl-kos">(</span><span class="pl-s">''</span><span
class="pl-kos">)</span><span class="pl-kos">;</span>
<span class="pl-k">const</span> <span class="pl-s1">headers</span> <span
class="pl-c1">=</span> <span class="pl-kos">{</span><span
class="pl-kos">}</span><span class="pl-kos">;</span>
<span class="pl-k">let</span> <span class="pl-s1">count</span> <span
class="pl-c1">=</span> <span class="pl-c1">0</span><span
class="pl-kos">;</span>

<span class="pl-k">for</span> <span class="pl-kos">(</span><span
class="pl-k">let</span> <span class="pl-s1">i</span> <span
class="pl-c1">=</span> <span class="pl-c1">0</span><span
class="pl-kos">;</span> <span class="pl-s1">i</span> <span
class="pl-c1">&lt;</span> <span class="pl-s1">chars</span><span
class="pl-kos">.</span><span class="pl-c1">length</span><span
class="pl-kos">;</span> <span class="pl-s1">i</span><span
class="pl-c1">++</span><span class="pl-kos">)</span> <span
class="pl-kos">{</span>
<span class="pl-k">if</span> <span class="pl-kos">(</span><span
class="pl-s1">count</span> <span class="pl-c1">===</span> <span
class="pl-c1">2000</span><span class="pl-kos">)</span> <span
class="pl-k">break</span><span class="pl-kos">;</span>

<span class="pl-k">for</span> <span class="pl-kos">(</span><span
class="pl-k">let</span> <span class="pl-s1">j</span> <span
class="pl-c1">=</span> <span class="pl-c1">0</span><span
class="pl-kos">;</span> <span class="pl-s1">j</span> <span
class="pl-c1">&lt;</span> <span class="pl-s1">chars</span><span
class="pl-kos">.</span><span class="pl-c1">length</span><span
class="pl-kos">;</span> <span class="pl-s1">j</span><span
class="pl-c1">++</span><span class="pl-kos">)</span> <span
class="pl-kos">{</span>
<span class="pl-k">const</span> <span class="pl-s1">key</span> <span
class="pl-c1">=</span> <span class="pl-s1">chars</span><span
class="pl-kos">[</span><span class="pl-s1">i</span><span
class="pl-kos">]</span> <span class="pl-c1">+</span> <span
class="pl-s1">chars</span><span class="pl-kos">[</span><span
class="pl-s1">j</span><span class="pl-kos">]</span><span
class="pl-kos">;</span>
<span class="pl-s1">headers</span><span class="pl-kos">[</span><span
class="pl-s1">key</span><span class="pl-kos">]</span> <span
class="pl-c1">=</span> <span class="pl-s">'x'</span><span
class="pl-kos">;</span>

<span class="pl-k">if</span> <span class="pl-kos">(</span><span
class="pl-c1">++</span><span class="pl-s1">count</span> <span
class="pl-c1">===</span> <span class="pl-c1">2000</span><span
class="pl-kos">)</span> <span class="pl-k">break</span><span
class="pl-kos">;</span>
    <span class="pl-kos">}</span>
  <span class="pl-kos">}</span>

<span class="pl-s1">headers</span><span class="pl-kos">.</span><span
class="pl-c1">Connection</span> <span class="pl-c1">=</span> <span
class="pl-s">'Upgrade'</span><span class="pl-kos">;</span>
<span class="pl-s1">headers</span><span class="pl-kos">.</span><span
class="pl-c1">Upgrade</span> <span class="pl-c1">=</span> <span
class="pl-s">'websocket'</span><span class="pl-kos">;</span>
<span class="pl-s1">headers</span><span class="pl-kos">[</span><span
class="pl-s">'Sec-WebSocket-Key'</span><span class="pl-kos">]</span>
<span class="pl-c1">=</span> <span
class="pl-s">'dGhlIHNhbXBsZSBub25jZQ=='</span><span
class="pl-kos">;</span>
<span class="pl-s1">headers</span><span class="pl-kos">[</span><span
class="pl-s">'Sec-WebSocket-Version'</span><span class="pl-kos">]</span>
<span class="pl-c1">=</span> <span class="pl-s">'13'</span><span
class="pl-kos">;</span>

<span class="pl-k">const</span> <span class="pl-s1">request</span> <span
class="pl-c1">=</span> <span class="pl-s1">http</span><span
class="pl-kos">.</span><span class="pl-en">request</span><span
class="pl-kos">(</span><span class="pl-kos">{</span>
<span class="pl-c1">headers</span>: <span
class="pl-s1">headers</span><span class="pl-kos">,</span>
<span class="pl-c1">host</span>: <span
class="pl-s">'127.0.0.1'</span><span class="pl-kos">,</span>
<span class="pl-c1">port</span>: <span class="pl-s1">wss</span><span
class="pl-kos">.</span><span class="pl-en">address</span><span
class="pl-kos">(</span><span class="pl-kos">)</span><span
class="pl-kos">.</span><span class="pl-c1">port</span>
<span class="pl-kos">}</span><span class="pl-kos">)</span><span
class="pl-kos">;</span>

<span class="pl-s1">request</span><span class="pl-kos">.</span><span
class="pl-en">end</span><span class="pl-kos">(</span><span
class="pl-kos">)</span><span class="pl-kos">;</span>
<span class="pl-kos">}</span><span class="pl-kos">)</span><span
class="pl-kos">;</span></pre></div>
<p>The vulnerability was reported by <a
href="https://github.com/rrlapointe">Ryan LaPointe</a> in <a
class="issue-link js-issue-link" data-error-text="Failed to load title"
data-id="2354846108" data-permission-text="Title is private"
data-url="websockets/ws#2230"
data-hovercard-type="issue"
data-hovercard-url="/websockets/ws/issues/2230/hovercard"
href="https://github.com/websockets/ws/issues/2230">#2230</a>.</p>
<p>In vulnerable versions of ws, the issue can be mitigated in the
following ways:</p>
<ol>
<li>Reduce the maximum allowed length of the request headers using
the<br>
<a href="https://nodejs.org/api/cli.html#--max-http-header-sizesize"
rel="nofollow"><code>--max-http-header-size=size</code></a> and/or the
<a
href="https://nodejs.org/api/http.html#httpcreateserveroptions-requestlistener"
rel="nofollow"><code>maxHeaderSize</code></a> options so<br>
that no more headers than the <code>server.maxHeadersCount</code> limit
can be sent.</li>
<li>Set <code>server.maxHeadersCount</code> to <code>0</code> so that no
limit is applied.</li>
</ol>
      </li>
    </ul>
from <a href="https://github.com/websockets/ws/releases">ws GitHub
release notes</a>
  </details>
</details>

---

> [!IMPORTANT]
>
> - Check the changes in this PR to ensure they won't cause issues with
your project.
> - This PR was automatically created by Snyk using the credentials of a
real user.

---

**Note:** _You are seeing this because you or someone else with access
to this repository has authorized Snyk to open upgrade PRs._

**For more information:** <img
src="https://api.segment.io/v1/pixel/track?data=eyJ3cml0ZUtleSI6InJyWmxZcEdHY2RyTHZsb0lYd0dUcVg4WkFRTnNCOUEwIiwiYW5vbnltb3VzSWQiOiI5OWYyNjVlZi00ZjIwLTQ2MTItOWI4NS05OGZhMTU1Y2IwN2IiLCJldmVudCI6IlBSIHZpZXdlZCIsInByb3BlcnRpZXMiOnsicHJJZCI6Ijk5ZjI2NWVmLTRmMjAtNDYxMi05Yjg1LTk4ZmExNTVjYjA3YiJ9fQ=="
width="0" height="0"/>

> - 🧐 [View latest project
report](https://app.snyk.io/org/okeamah/project/79f5fe07-5650-42a8-a92c-0ae46036ffc8?utm_source&#x3D;github&amp;utm_medium&#x3D;referral&amp;page&#x3D;upgrade-pr)
> - 📜 [Customise PR
templates](https://docs.snyk.io/scan-using-snyk/pull-requests/snyk-fix-pull-or-merge-requests/customize-pr-templates)
> - 🛠 [Adjust upgrade PR
settings](https://app.snyk.io/org/okeamah/project/79f5fe07-5650-42a8-a92c-0ae46036ffc8/settings/integration?utm_source&#x3D;github&amp;utm_medium&#x3D;referral&amp;page&#x3D;upgrade-pr)
> - 🔕 [Ignore this dependency or unsubscribe from future upgrade
PRs](https://app.snyk.io/org/okeamah/project/79f5fe07-5650-42a8-a92c-0ae46036ffc8/settings/integration?pkg&#x3D;ws&amp;utm_source&#x3D;github&amp;utm_medium&#x3D;referral&amp;page&#x3D;upgrade-pr#auto-dep-upgrades)

<!---
(snyk:metadata:{"customTemplate":{"variablesUsed":[],"fieldsUsed":[]},"dependencies":[{"name":"ws","from":"8.17.1","to":"8.18.0"}],"env":"prod","hasFixes":false,"isBreakingChange":false,"isMajorUpgrade":false,"issuesToFix":[],"prId":"99f265ef-4f20-4612-9b85-98fa155cb07b","prPublicId":"99f265ef-4f20-4612-9b85-98fa155cb07b","packageManager":"npm","priorityScoreList":[],"projectPublicId":"79f5fe07-5650-42a8-a92c-0ae46036ffc8","projectUrl":"https://app.snyk.io/org/okeamah/project/79f5fe07-5650-42a8-a92c-0ae46036ffc8?utm_source=github&utm_medium=referral&page=upgrade-pr","prType":"upgrade","templateFieldSources":{"branchName":"default","commitMessage":"default","description":"default","title":"default"},"templateVariants":[],"type":"auto","upgrade":[],"upgradeInfo":{"versionsDiff":1,"publishedDate":"2024-07-03T16:45:31.280Z"},"vulns":[]})
--->
This was referenced Aug 21, 2024
Open
Open
@qdraw qdraw mentioned this pull request Aug 25, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nikwen nikwen nikwen left review comments

@kettanaito kettanaito kettanaito approved these changes

@dharishbabu007 dharishbabu007 dharishbabu007 approved these changes

@5twelve 5twelve 5twelve approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Support sending Blob
6 participants
@lpinca @titanism @nikwen @kettanaito @dharishbabu007 @5twelve