Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nitro Tasks API #1974

Open
4 of 10 tasks
pi0 opened this issue Nov 30, 2023 · 73 comments
Open
4 of 10 tasks

Nitro Tasks API #1974

pi0 opened this issue Nov 30, 2023 · 73 comments
Labels
discussion enhancement New feature or request

Comments

@pi0
Copy link
Member

pi0 commented Nov 30, 2023

Nitro tasks allow on-off operations in runtime.

Docs: https://nitro.unjs.io/guide/tasks

  • Initial Implementation (feat: experimental nitro tasks #1929)
  • Write documentation section (PR welcome!)
  • Stabilize API / return values
  • Integrate with Cron Tasks / Background Jobs
  • Cancelable tasks #2633
  • Vercel support (docs - workaround)
  • Ability to run tasks using CLI against production build
  • Task hooks (on dev start, production start and production build)
  • Ability to schedule one-time run of a task for the future
  • Allow optionally allow running multiple instances of task
@pi0 pi0 added enhancement New feature or request discussion labels Nov 30, 2023
@pi0

This comment was marked as off-topic.

@pi0

This comment was marked as off-topic.

@Aareksio
Copy link
Contributor

Aareksio commented Jan 7, 2024

Tasks should probably be async, currently the run method expects synchronous code and is not awaited.

@TheAlexLichter
Copy link
Member

@Aareksio check #2021 ☺️

@Rigo-m
Copy link
Contributor

Rigo-m commented Jan 15, 2024

If I use a nitro task in, let's say, Netlify will it leverage netlify background functions?

@cosbgn
Copy link

cosbgn commented Jan 16, 2024

I'm also interested on this. I would like to have for example my normal code on vercel-edge but run tasks on vercel lambda (5min timeout vs 25 seconds)

@imcm7
Copy link

imcm7 commented Jan 18, 2024

How to disable it, because i use dir tasks for other stuff?

@brendonmatos
Copy link
Contributor

It is planned to support running these tasks from cli without running the server?

@pi0
Copy link
Member Author

pi0 commented Feb 22, 2024

Nitro already has a CLI preset too! However it is tricky to tell now when we can do support it this way now.

@robin-dongbin
Copy link

Any plan for support seconds scheduling?

@pi0
Copy link
Member Author

pi0 commented Feb 27, 2024

@robin-dongbin You mean this https://nitro.unjs.io/guide/tasks#scheduled-tasks or something different? (like a pattern for <1 minute schedule?)

@robin-dongbin
Copy link

@robin-dongbin You mean(意味着) this nitro.unjs.io/guide/tasks#scheduled-tasks or something different? (like a pattern(模式) for <1 minute(分钟) schedule?)

Yes, that's what I want. Is it possible? I know that cron do not support that, but Laravel has support this last year.

@go4cas
Copy link
Contributor

go4cas commented Feb 27, 2024

@pi0 ... I have just tested the experimental scheduled task feature. It works really well! Thanks!

Just one question: would it be possible to have a config setting to execute a scheduled task immediately after the cron is scheduled? Example: a task running every 10 minutes: executes once the server starts, and then every 10 minutes after that? I saw in the croner docs that there is a trigger method, but not sure how it would fit into the nitro tasks?

@pi0
Copy link
Member Author

pi0 commented Feb 27, 2024

@go4cas Maybe use a Nitro plugin to triger them on startup? (also init: / startup: tasks are planned to be supported to have similar effect in the future). Also you can execute a task from another.

@robin-dongbin I also wonder if you need to trigger something this frequently, maybe a setInterval inside nitro plugins to run task with runTask might be simpler method with giving full control on executation?

@JCtapuk
Copy link

JCtapuk commented Feb 27, 2024

@go4cas Maybe use a Nitro plugin to triger them on startup? (also init: / startup: tasks are planned to be supported to have similar effect in the future). Also you can execute a task from another.

@robin-dongbin I also wonder if you need to trigger something this frequently, maybe a setInterval inside nitro plugins to run task with runTask might be simpler method with giving full control on executation?

But won’t there be conflicts if a node in a cluster with 8 processes and all 8 processes will cause a task, but this behavior should not happen? It turns out you need to find someone who has a free worker and give him a task from the master?

@pi0
Copy link
Member Author

pi0 commented Feb 27, 2024

For time being, scheduled tasks are not supported in cluster mode the exactly meet this requirement (share a safe lock among them) and it is doable.

Regardless inside your interval you can call runTask() in single node, it currently guarantees only it is running once and when we support cluster, it would be safe too.

@brendonmatos
Copy link
Contributor

brendonmatos commented Feb 29, 2024

I apologize if this isn't the correct forum. But how is supposed to run those tasks in Nuxt project from CLI? Nuxt CLI will be implemented to invoke those tasks from the URL or we should run directly from nitro cli?

@pi0
Copy link
Member Author

pi0 commented Feb 29, 2024

Yes.

@passionate-bram
Copy link
Contributor

With async tasks, and enforcing task payloads to be serializable early on it will be possible in the future to dispatch tasks to queues. Shamelessly taking the concept from Laravel: https://laravel.com/docs/10.x/queues

The nice thing about that way of working is that when queues are configurable (e.g. redis, some AWS service, etc.) then the dispatch mechanic becomes really usefull for dispatching work to different parts of your architecture.

@pi0
Copy link
Member Author

pi0 commented Mar 5, 2024

payload should be serializable indeed in preparation for future ideas like queues. and it is by design separated from context that is non seriazable for other stuff. PR welcome to clarify in the docs.

@provok-exe
Copy link

If I use a nitro task in, let's say, Netlify will it leverage netlify background functions?

@pi0
A Nitro task has only one running instance, but Netlify background functions can handle multiple instances at the same time (as I understand it in there docs).

Is there another way to use Netlify background functions with Nitro to handle multiple instances of a background function?

@pi0 pi0 mentioned this issue Jul 25, 2024
1 task
@StepanMynarik
Copy link

Using Nuxt 3.

Tried building an endpoint to get scheduled jobs using 'scheduledJobs' variable imported from "cron" package directly, see here: https://croner.56k.guru/usage/examples/#naming-jobs
My intention was to build a UI that shows the scheduled jobs, ideally even which ones are currently running with which payload etc.

The 'scheduledJobs' variable however returns an empty array always.

Is there a way to do this? To monitor scheduled and running jobs?

@luc122c
Copy link
Contributor

luc122c commented Jul 27, 2024

Hi @pi0, I'm wondering how scheduled tasks are handled in a serverless context? For example, I am currently using the Firebase preset which uses a Firebase Function as the 'server'. Since the server is not long-lived, would scheduled tasks execute each time the service is called, or are they not included in unsupported presets?

@gbyesiltas
Copy link

gbyesiltas commented Aug 24, 2024

hey, is it currently possible to create a nuxt application with tasks and get it to work with cloudflare triggers? (as it's deployed as a pages application I couldn't get it to work)

update: i got it to work after deploying the nuxt app with the cloudflare_module preset here which deploys it as a workers application

@carlos-duran
Copy link

Hello, I was trying to run nitro task run hi in a Nuxt 3 app but it looks like the .nitro folder is expected. We have .nuxt folder.

Btw, Thank you for a great API!

@TMBL-DEV
Copy link
Contributor

TMBL-DEV commented Sep 13, 2024

@carlos-duran i had the same issue i have an open pull request at the moment that should fix it

#2635

btw you can edit your buildDir in the nuxt config

// https://nuxt.com/docs/api/configuration/nuxt-config
export default defineNuxtConfig({
  compatibilityDate: '2024-04-03',
  devtools: { enabled: true },
  buildDir: '.nitro',
  nitro: {
    buildDir: '.nitro', 
    experimental: {
      tasks: true,
    }
  }
})

and replace .nuxt in the tsconfig file with .nitro

@aligzl
Copy link

aligzl commented Oct 10, 2024

hey, is it currently possible to create a nuxt application with tasks and get it to work with cloudflare triggers? (as it's deployed as a pages application I couldn't get it to work)

update: i got it to work after deploying the nuxt app with the cloudflare_module preset here which deploys it as a workers application

Can you please give an example how you did it?
Thanks

@gbyesiltas
Copy link

Can you please give an example how you did it?
Thanks

@aligzl hey, I pretty much exactly followed the docs for the cloudflare_preset module which I believe builds/deploys the app as a workers application and not as a pages application. Nitro tasks started working for me after that

@aligzl
Copy link

aligzl commented Oct 10, 2024

hmm, i am using nuxt hub, i guess i will mess it up if i touch it :)

@zacwebb
Copy link

zacwebb commented Oct 10, 2024

hmm, i am using nuxt hub, i guess i will mess it up if i touch it :)

I'm in the same boat with NuxtHub. With the new Cloudflare worker updates, support for this will be coming soon to NuxtHub I believe (cc @atinux 👀)

@ymansurozer
Copy link

The only thing that i need to figure out is how to get cloudflare d1 working in the tasks context

The context inside the TaskEvent seems to be the same as a usual context in an API route for Cloudflare Workers deployment. So you can access the Cloudflare env in that, too. @TMBL-DEV @juretopolak

@gsxdsm
Copy link

gsxdsm commented Nov 25, 2024

Love to see this can't wait until it's fully ready

@shoooe
Copy link

shoooe commented Dec 13, 2024

I tried this 3 weeks ago and now I can't make them work anymore for some reason following the same guide I had specified here: solidjs/solid-docs#964

I also tried in Tanstack Start which is also based on Nitro and it also doesn't work there.

Did something change somehow in the last 3 weeks 😕? Seems like the last release was more than 3 weeks ago so I'm assuming no.

Anybody else has the same problem?

Or if you made it work could you post your app.config.ts and tasks folder?

@pi0
Copy link
Member Author

pi0 commented Dec 13, 2024

@shoooe both tanstack-start and solid are based on a wrapper "Vinxi" which sadly abstracts away full features of Nitro (consider even dev server of nitro is not used when using vinxi!). Currently Nuxt, AnalogJS or pure Nitro projects can use all features. Hopefully this changes in the near future for them.

@kstraszewski
Copy link

Im using Nitro Tasks on Nuxt3, and i have an issue when i deploy them on Vercel. Everything works great locally, but when i deploy app on vercel the Task mysteriously dies and i do not have any information why. When i've added console.logs to the function i see that at some point they stops. Any idea how to debug it, or why it might happened?

@luc122c
Copy link
Contributor

luc122c commented Jan 5, 2025

on vercel the Task mysteriously dies and i do not have any information why.

I would imagine it's timing out. Vercel is not really the best place for long-lived tasks.

@kstraszewski
Copy link

kstraszewski commented Jan 5, 2025

on vercel the Task mysteriously dies and i do not have any information why.

I would imagine it's timing out. Vercel is not really the best place for long-lived tasks.

I have this in my nuxt.config

...
    vercel: {
      functions: {
        maxDuration: 120,
      },
    },    

Task should take about 30-45sec.
I do not see anything in the vercel logs.

It looks like handling async calls to external api, crashes the task.

Is there any recomended place to deploy Nuxt3 with working tasks?

@pi0
Copy link
Member Author

pi0 commented Jan 6, 2025

Added vercel support to tracker 🤞🏼

In the meantime, you can instruct nitro config and create a custom handler for vercel:

export default defineNitroConfig({
  vercel: {
    config: {
      crons: [
        {
          // TODO: create server/api/_cron.ts event handler
          path: "/api/_cron", 
          schedule: "0 0 * * *",
        }
      ],
    },
  },
});

@kstraszewski
Copy link

Added vercel support to tracker 🤞🏼

In the meantime, you can instruct nitro config and create a custom handler for vercel:

export default defineNitroConfig({
  vercel: {
    config: {
      crons: [
        {
          // TODO: create server/api/_cron.ts event handler
          path: "/api/_cron", 
          schedule: "0 0 * * *",
        }
      ],
    },
  },
});

For me it is issue with single Nitro Task, run by built-in runTask function not scheduled crons. Sometimes it works after even 10-15minutes, but locally task runs in about 15sec.

I found out openAi returns me error:
APICallError [Al_APICallError]: Cannot connect to APl: other side closed

@gritworks
Copy link

gritworks commented Jan 6, 2025

thanks pi0.

Also seeing support for this way of defining the cronjob in the nuxt.config file.
I guess this will work also the same way:

nitro: {
      vercel:{
        config:{
            "crons": [{
            "path": "/tasks/mytask",
            "schedule": "0 0 2 * *"
          }]
        }
      } 
  }

@danielroe danielroe marked this as a duplicate of nuxt/nuxt#30471 Jan 7, 2025
@medz
Copy link

medz commented Jan 15, 2025

@pi0 I have a question. I use Nitro development server and deploy it with docker. There may be multiple pods. Do we have any way to ensure that the task will only run once?

For example, Redis shared lock, etc.

@ooxoxx
Copy link

ooxoxx commented Jan 23, 2025

@pi0 hi, thanks for the hard work on this feature. I really need that for a data scrawling scenario. Is there any workarounds about long-running tasks on production environments like netlify right now?
Here is my major requirements:

  1. single netlify function instance for caching/state-control. (Tasks result in cache/state changing in application level.)
  2. get rid of 10 seconds limit of api calling from netlify function. (maybe take advantage of their background function feature?)
    I think it's literally not a very simple quest. Should we make the entire server.mjs function a "background" one?

@tomhaerter
Copy link

I'm curious if there's any plan for a queue system like Rails Solid Queue or Laravel Queues. (I saw the previous comments about Laravel Queues)

Things like these would be great:

  • Queue Worker(s) (which can run independently of the main Nitro server, for horizontal scaling see Rails Solid Queue)
  • Storage Adapters (SQL, Redis, RabbitMQ, Amazon SQS, etc.)
  • Retrying of tasks

@oom-
Copy link

oom- commented Feb 18, 2025

hi @pi0, does Allow optionally allow running multiple instances of task, means that by default a task can be run only once at a time ?

If the task is scheduled to be every min, something like that? :

00h01: 🕐 run task1 <-- start task1
00h02: 🔴 run task1 <--- if task1 didn't finished, it will not be started there
00h03: 🔴 run task1 <--- if task1 didn't finished, it will not be started there
-- task1 is done--
00h04: 🕐 run task1 <--start task1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discussion enhancement New feature or request
Projects
None yet
Development

No branches or pull requests