@google-cloud/storage throws errors with bun

[Xiot]

What version of Bun is running?

1.0.3+25e69c71e70ac8a0a88f9cf15b4057bd7b2a633a

What platform is your computer?

Darwin 22.5.0 arm64 arm

What steps can reproduce the bug?

// test.mjs
import { Storage } from '@google-cloud/storage'

const storage = new Storage({
  keyFilename: './key.json',
})
const bucket = storage.bucket('bucket')

const info = await bucket.file('test.json').getMetadata()
console.log('info', info)

> bun test.mjs

What is the expected behavior?

the call to getMetadata() returns successfully

What do you see instead?

❯ time bun test.mjs
1 | (function (stream, autoAllocateChunkSize) {"use strict";
                       ^
TypeError: undefined is not an object
      at lazyLoadStream (:1:20)
      at getReader (:1:20)
      at readStreamIntoSink (:1:32)
      at readableStreamIntoText (:1:20)
      at /Users/chris/test/apps/web-serve/node_modules/teeny-request/build/src/index.js:223:8
      at processTicksAndRejections (:55:76)
bun test.mjs  0.19s user 0.08s system 1% cpu 15.780 total

Additional information

The call returns in about 200ms when run via node v20.6.1

[Xiot]

So I did a little bit of digging and it seems like its coming from node-fetch.

If I replace the usage of node-fetch with the global version in teeny-request/build/src/index.js file, allows it to start working.

// const node_fetch_1 = require("node-fetch");
const node_fetch_1 = {default: fetch}

Another interesting thing that I noticed, is that I put in console.log messages in all of the entrypoint files from node-fetch and none of them seemed to have been hit.

require.resolve didn't seem to help much when trying to track down the implementation, as

require.resolve('node-fetch') === 'node-fetch'

[Xiot]

Actually, my little hack of replacing the node-fetch with the native fetch works for that instance, but fails when trying to call bucket.file('test.json').download() with:

185 |         teenyRequest.stats.requestStarting();
186 |         (0, node_fetch_1.default)(uri, options).then(res => {
187 |             teenyRequest.stats.requestFinished();
188 |             responseStream = res.body;
189 |             responseStream.on('error', (err) => {
                ^
TypeError: responseStream.on is not a function. (In 'responseStream.on("error", (err) => {
          requestStream.emit("error", err);
        })', 'responseStream.on' is undefined)
      at /Users/chris/test/apps/web-serve/node_modules/teeny-request/build/src/index.js:189:12
      at processTicksAndRejections (:55:76)

Logging the value of responseStream, it looks like the bun version of ReadableStream doesn't have the EventEmitter methods (specifically the on method)

ReadableStream {
  [Symbol(state)]: 4,
  [Symbol(reader)]: undefined,
  [Symbol(storedError)]: undefined,
  [Symbol(disturbed)]: false,
  [Symbol(readableStreamController)]: null,
  [Symbol(bunNativeType)]: 1,
  [Symbol(asyncContext)]: undefined,
  [Symbol(highWaterMark)]: undefined,
  [Symbol(underlyingSource)]: undefined,
  [Symbol(start)]: [Function],
  locked: [Getter],
  cancel: [Function: cancel],
  getReader: [Function: getReader],
  pipeTo: [Function: pipeTo],
  pipeThrough: [Function: pipeThrough],
  tee: [Function: tee],
  values: [Function: values],
  [Symbol(Symbol.asyncIterator)]: [Function: lazyAsyncIterator]
}

[andriotisnikos1]

+1! im also having similar issues. I can't list buckets or files, the process exits with code 0 and no message.

Example Code:

export const gcs = new Storage({
    keyFile: process.cwd() + "/gcs.json",
    projectId: "project-id-here",

})

gcs.getBuckets().then((res) => console.log(res)).catch((err) => console.error(err))

Output:

nikos@nikos server % bun run src/index.ts                    
Server running on port 4000
nikos@nikos server % 

I have no idea as to what might be the reason, ill update when I find something

[Xiot]

@andriotisnikos1 , try adding an await when you call getBuckets.
If you aren't getting anything logged (not even a new line) then it could be caused by bun not waiting for the hanging promise to complete before exiting the process

[andriotisnikos1]

@Xiot I did try it before. as you said, im getting nothing logged to the console, the process just exits. I tried running the server with node afterwards and it worked like a charm. soooo yup, it is bun that causes the process to exit. thanks for the tip though! 🫡

[benjaminwestern]

+1 here, I am seeing the following:
bun --inspect src/index.ts
[1] 17906 segmentation fault bun src/index.ts --debug
OR
bun dev (bun run --watch src/index.ts)
error: script "dev" exited with code 11 (SIGSEGV)

When simply trying to call files from a bucket:

export const getFiles = async (bucketName: string) => {
    try {
        const bucket = storageClient.bucket(bucketName);
        const [files] = await bucket.getFiles(); // Fails at this line - doesn't make it to the catch, fails at the bun level It seems
        return files;
    }
    catch (error) {
        console.error('Error fetching GCS objects:', error);
        throw error;
    }
}

[LucyEgan]

Looks like another issue im facing with a @Google-Cloud package as i did the same hack that @Xiot did and made my issue go away.

• @google-cloud/trace-agent fails to run on bun and returns a error everything was successful(90%) or segmentation faults(10%) #6911
  From looking at the stacks its the same area though teeny-request, and the call to .text() is native and not the node-fetch one

[rupurt]

I also have problems with @google-cloud/storage but they're a little different to these. The issue seems to be in the way gaxios imports is-stream. The line reported seems to be slightly off also.

// pkgs/storage/node_modules/gaxios/build/src/gaxios.js
// source of error
219 |             if (is_stream_1.default.readable(opts.data)) {
220 |                 opts.body = opts.data;
                                    ^
TypeError: undefined is not an object (evaluating 'is_stream_1.default.readable')
      at /home/alex/workspace/rupurt/eve/pkgs/storage/node_modules/gaxios/build/src/gaxios.js:220:33

// pkgs/storage/node_modules/gaxios/build/src/gaxios.js
// import section
Object.defineProperty(exports, "__esModule", { value: true });
exports.Gaxios = void 0;
const extend_1 = __importDefault(require("extend"));
const https_1 = require("https");
const node_fetch_1 = __importDefault(require("node-fetch"));
const querystring_1 = __importDefault(require("querystring"));
const is_stream_1 = __importDefault(require("is-stream"));

This is what is_stream_1 looks like after import

is_stream_1: Module {
  __esModule: true,
  isDuplexStream: [Function: isDuplexStream],
  isReadableStream: [Function: isReadableStream],
  isStream: [Function: isStream],
  isTransformStream: [Function: isTransformStream],
  isWritableStream: [Function: isWritableStream]
}
is_stream_1.default: undefined

[rupurt]

Aha! I get the segfault when I manually replace node-fetch and is-stream with the latest versions on npm

// package.json
// ...
    "is-stream": "^3.0.0",
    "node-fetch": "^3.3.2"
// ...

[Jarred-Sumner]

There is also a bug here where retry-request seems to be happening in bun and not in node, which causes erroring requests to be retried

[rupurt]

You da man @Jarred-Sumner. Super quick turnaround. I can confirm it's now working with GCS on bun v1.0.11

[seanwessmith]

bun --revision
1.0.13+f5bf67bd1

I'm having a similar issue on bucket.upload(). The program will successfully upload the file but throws an error undefined is not an object.

import { Storage } from "@google-cloud/storage";

const main = async () => {
  const storage = new Storage({
    keyFilename: `./google-cloud-key.json`,
  });

  const bucketName = "bun-test-bucket";
  const filePath = "./test.txt";
  const bucket = storage.bucket(bucketName);

  // Sending the upload request
  bucket.upload(
    filePath,
    {
      destination: `test.txt`,
    },
    function (err, file) {
      if (err) {
        console.error(`Error uploading ${filePath}: ${err}`);
      } else {
        console.log(`txt file uploaded to ${bucketName}.`, file?.name);
      }
    }
  );
};

main();

➜  test bun run main.ts
1 | (function (controller) {"use strict";
                       ^
TypeError: undefined is not an object
      at readableStreamDefaultControllerCanCloseOrEnqueue (:1:20)
      at readableStreamDefaultControllerCallPullIfNeeded (:1:20)
      at processTicksAndRejections (:61:76)
txt file uploaded to bun-test-bucket. test.txt

[rupurt]

@seanwessmith I get the same error. Only GET requests seem to work without errors.

[benjaminwestern]

+1 here, I am seeing the following:

bun --inspect src/index.ts

[1] 17906 segmentation fault bun src/index.ts --debug

OR

bun dev (bun run --watch src/index.ts)

error: script "dev" exited with code 11 (SIGSEGV)

When simply trying to call files from a bucket:

export const getFiles = async (bucketName: string) => {

    try {

        const bucket = storageClient.bucket(bucketName);

        const [files] = await bucket.getFiles(); // Fails at this line - doesn't make it to the catch, fails at the bun level It seems

        return files;

    }

    catch (error) {

        console.error('Error fetching GCS objects:', error);

        throw error;

    }

}

I can confirm my issue with Get is now resolved with the latest version of bun 1.0.13

[Scalahansolo]

@Jarred-Sumner Just wanted to call out that I am attempting to use the node GCS storage SDK to download stuff and it seems that bun is just exiting the process with code 0. Tried to get more information through inspect or try / catches, but no luck. Just a clean exit every time.

[seanwessmith]

@Scalahansolo do you have more info, maybe an example file? I just tried this on Bun 1.0.18 and it lists the files correctly.

import { Storage } from "@google-cloud/storage";

const storageClient = new Storage({
  keyFilename: "./key.json",
});

export const getFiles = async (bucketName: string) => {
  try {
    const bucket = storageClient.bucket(bucketName);

    const [files] = await bucket.getFiles();

    console.log("Files:");
    files.forEach((file) => {
      console.log(file.name);
    });
    return files;
  } catch (error) {
    console.error("Error fetching GCS objects:", error);

    throw error;
  }
};

const bucketName = "test-bucket";
getFiles(bucketName);

bun run main.ts
Files:
backups//tmp/test.sqlite
backups/tes.sqlite
test.sqlite
test.txt

[Scalahansolo]

Hey @seanwessmith . Get files works fine for me as well. Trying to actually download the files is where I have problems with the early exit.

Downloading either through the single file or through a TransferManager have the issue of the process exit.

[MathewsJoseph25]

I'm getting this error when trying to sign a url using gcloud library:

] 10 | // Unless required by applicable law or agreed to in writing, software
[clan-app-shopify] [2024-03-05 05:07:29] 11 | // distributed under the License is distributed on an "AS IS" BASIS,
[clan-app-shopify] [2024-03-05 05:07:29] 12 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
[clan-app-shopify] [2024-03-05 05:07:29] 13 | // See the License for the specific language governing permissions and
[clan-app-shopify] [2024-03-05 05:07:29] 14 | // limitations under the License.
[clan-app-shopify] [2024-03-05 05:07:29] 15 | Object.defineProperty(exports, "__esModule", { value: true });
[clan-app-shopify] [2024-03-05 05:07:29]                         ^
[clan-app-shopify] [2024-03-05 05:07:29] SigningError: null is not an object
[clan-app-shopify] [2024-03-05 05:07:29]       at node:crypto:8827:30
[clan-app-shopify] [2024-03-05 05:07:29]       at parseKeys (node:crypto:8847:23)
[clan-app-shopify] [2024-03-05 05:07:29]       at sign (node:crypto:8983:18)
[clan-app-shopify] [2024-03-05 05:07:29]       at node:crypto:9127:38
[clan-app-shopify] [2024-03-05 05:07:29]       at sign (/app/node_modules/google-auth-library/build/src/crypto/node/crypto.js:15:20)
[clan-app-shopify] [2024-03-05 05:07:29]       at /app/node_modules/google-auth-library/build/src/auth/googleauth.js:780:39
[clan-app-shopify] [2024-03-05 05:07:29] error: script "start" exited with code 1
[clan-app-shopify] [2024-03-05 05:07:29] error: script "prod:api" exited with code 1
[clan-app-shopify] [] exited with exit code 1
[clan-app-shopify] [2024-03-05 05:07:49] $ cd apps/api && bun start

Any idea why this might be happening? My docker file:

FROM ubuntu:23.04


RUN apt-get update && \
  apt-get install -y curl unzip && \
  apt-get clean

RUN curl -sL https://deb.nodesource.com/setup_20.x | bash - && \
  apt-get install -y nodejs

RUN node --version

RUN npm install -g bun

RUN bun -v

WORKDIR /app
COPY . .
RUN bun install

CMD ["bun",  "prod:api"]

The issue happens with the oven:bun image too.

But when i run the same docker file in my local machine, everything works normally.