got-scraping not working with lambda function anymore

[nrjdalal]

Cloudwatch logs -

{
    "errorType": "Error",
    "errorMessage": "ENOENT: no such file or directory, open '/var/task/dist//data_files/headers-order.json'",
    "code": "ENOENT",
    "errno": -2,
    "syscall": "open",
    "path": "/var/task/dist//data_files/headers-order.json",
    "stack": [
        "Error: ENOENT: no such file or directory, open '/var/task/dist//data_files/headers-order.json'",
        "    at Object.openSync (node:fs:573:18)",
        "    at readFileSync (node:fs:452:35)",
        "    at new HeaderGenerator (file:///var/task/dist/index.mjs:75050:62)",
        "    at file:///var/task/dist/index.mjs:88702:17",
        "    at ModuleJob.run (node:internal/modules/esm/module_job:234:25)",
        "    at async ModuleLoader.import (node:internal/modules/esm/loader:473:24)",
        "    at async _tryAwaitImport (file:///var/runtime/index.mjs:1008:16)",
        "    at async _tryRequire (file:///var/runtime/index.mjs:1057:86)",
        "    at async _loadUserApp (file:///var/runtime/index.mjs:1081:16)",
        "    at async UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:1119:21)"
    ]
}

Code ref -

https://github.com/nrjdalal/html-scraper

[B4nan]

What nodejs version are you on? This feels like a bug they introduced in v23.0.0, seeing the double slash in the path.

[nrjdalal]

INIT_START Runtime Version: nodejs:20.v42 Runtime Version

[nrjdalal]

if I comment out imports of got-scraping and usage, the fx works fine

[B4nan]

your project looks more like you are using bun, not nodejs?

[nrjdalal]

on aws I am using node only, there is no bun support, bun is for dev only

async run() {
  const schema = z
    .object({
      API_KEY: z.string().min(8),
      SST_AWS_ACCESS_KEY_ID: z.string(),
      SST_AWS_SECRET_ACCESS_KEY: z.string(),
      PULUMI_NODEJS_PROJECT: z.string(),
      PULUMI_NODEJS_STACK: z.enum(["dev"]),
    })
    .parse(process.env)

  const apSouth1 = new sst.aws.Function(
    "html-scraper",
    {
      handler: "dist/index.handler",
      memory: "256 MB",
      runtime: "nodejs20.x", // runtime
      url: true,
      environment: {
        API_KEY: schema.API_KEY,
        ACCESS_KEY_ID: schema.SST_AWS_ACCESS_KEY_ID,
        SECRET_ACCESS_KEY: schema.SST_AWS_SECRET_ACCESS_KEY,
        FUNCTION_NAME: `${schema.PULUMI_NODEJS_PROJECT}-${schema.PULUMI_NODEJS_STACK}-${schema.PULUMI_NODEJS_PROJECT.replace("-", "")}Function`,
        FUNCTION_REGION: "ap-south-1",
      },
    },
    {
      provider: new aws.Provider("ap-south-1", { region: "ap-south-1" }),
    },
 )

runtime is node, so is in case of logs, and after fx console

aka 100% it is node