Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2 step plan to fix most DNS-related issues in Bun #10731

Closed
Jarred-Sumner opened this issue May 1, 2024 · 4 comments
Closed

2 step plan to fix most DNS-related issues in Bun #10731

Jarred-Sumner opened this issue May 1, 2024 · 4 comments

Comments

@Jarred-Sumner
Copy link
Collaborator

Jarred-Sumner commented May 1, 2024

There are many issues related to IPv4 vs IPv6 where Bun seemingly takes forever to send a network request that should be almost instant.

This impacts bun install, fetch(), node:http, and pretty much anywhere making an outgoing HTTP request. It doesn't impact most users, but it does impact many.

I think there are two causes:

  1. getaddrinfo is blocking
  2. We do not implement Happy Eyeballs v2

I think these are two separate projects.

Make getaddrinfo non-blocking

We already have code to do this, so it should be the easiest part here. We need to separate out the JavaScript bindings from the DNS code into something we can use for both JavaScript and Zig. That means it needs to not depend on a JSC.VirtualMachine.

On macOS, we can use libinfo, on Windows, uv_getaddrinfo, and on Linux, libc getaddrinfo in a thread pool. Having 3 different implementations sounds like it'd be complicated but we already wrote the code to do this for the JS bindings so it's not actually much more work to wire it up to http.zig.

This will have the most impact on Docker & Linux, where getaddrinfo is not cached by default (macOS caches it). Like with the JavaScript bindings, we should have a cache to dedupe duplicate in-flight DNS queries. This handles making multiple fetch calls concurrently to the same host. Instead of 5 calls through getaddrinfo(), we only need to make one call.

#10728 is a small way to start this project, but I do not think I have the time to do this myself

Implement Happy Eyeballs v2

Happy Eyeballs is essentially:

  • Run getaddrinfo
  • Instead of connecting to the first one that matches, connect to all of them at the same time
  • Once one address successfully connects, close the other sockets
  • Optional: cache which one successfully connected for the TTL of that DNS query

I think this will reliably fix the issues where something can connect successfully in Node but not in Bun.

I expect this to be a little trickier to implement. Roughly how I'd do it is:

  1. Add a new poll type to uSockets for a socket which has not finished connecting yet
  2. This new poll type uses the *next ptr to denote what are the other sockets its associated with
  3. Once one of them connects successfully, close all the file descriptors for the other sockets immediately
  4. Deinit the previous sockets on the next tick of the event loop
  5. Update the poll type of the connecting socket

We can probably rely on the existing socket timeout code to handle timeouts.

Related issues: (TODO: add more)

@coindegen
Copy link

Starting a few hours ago all my bun install commands have been failing in Github Actions. Is the above issue why I'm getting these errors in CI?

------
 > [production 11/11] RUN bun install --production:
0.163 bun install v1.1.6 (e58d67b4)
37.97 
37.97 GET https://registry.npmjs.org@socket.io/component-emitter/-/component-emitter-3.1.0.tgz - 404
148.6 Segmentation fault
------
Error: failed to fetch an image or build from source: error building: failed to solve: process "/bin/sh -c bun install --production" did not complete successfully: exit code: 139

My workaround so far has been to use Node.js as my base image in my Dockerfile:

# ---- Build Stage ----
# FROM oven/bun:1.0 AS builder
FROM node:lts AS builder
RUN npm i -g bun

WORKDIR /app

ARG NPM_TOKEN

COPY package.json bun.lockb bunfig.toml .npmrc ./
# RUN bun install
RUN npm install

COPY . .
RUN bun run build
...

Is there a better workaround?

@touhidurrr
Copy link

touhidurrr commented May 7, 2024

My workaround so far has been to use Node.js as my base image in my Dockerfile:

I don't know if this is related but prisma does not work on bun if node is not installed. pretty sure --bun did not help. Maybe I should open a separate issue?

@coindegen
Copy link

Starting a few hours ago all my bun install commands have been failing in Github Actions. Is the above issue why I'm getting these errors in CI?

Is there a better workaround?

my issue has been solved since 1.1.9 and 1.1.10 releases. Thanks everyone.

@Jarred-Sumner
Copy link
Collaborator Author

This was completed some time ago

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants