Job takes too long?

[AdamKyle]

Here is my controller action:

public function export() {

    (new ItemsExport)->queue('items.xlsx')->chain([
        new ExportItemsEmail(auth()->user()),
    ]);

    return redirect()->back()->with('success', 'File will be emailed to you once done.');
}

Super simple, we queue up the export (there are over 5k records, can be more)

class ItemsExport implements FromQuery, ShouldQueue {

    use Exportable;

    public function query() {
        return Item::query();
    }
}

Again nothing fancy.

Issue: Horizon fails

I get this error in horizon:

Illuminate\Queue\MaxAttemptsExceededException: Maatwebsite\Excel\Jobs\AppendQueryToSheet has been attempted too many times or run too long. The job may have previously timed out. in /home/adam/Documents/flare/vendor/laravel/framework/src/Illuminate/Queue/Worker.php:736

From the docs this is support to chunk it and spin up many jobs. How ever in Horizon I see 2-3 jobs go through pending like no tomorrow, then one that stays before dying because of the above error.

Am I doing this wrong?

[patrickbrouwers]

I'd personally not recommend using the built in queuing, each chunk has to re-open the file which takes more time and more memory on each chunk-job. I believe it's better/easier to just wrap the export in a job of your own, increasing the timeout and running it on a long-running redis process (using Horizon as you are already doing).

Next release will demonstrate this approach in docs, however currently don't have that info ready, but hope you are able to figure it out with articles like https://medium.com/@williamvicary/long-running-jobs-with-laravel-horizon-7655e34752f7

[wvdongen]

I've got the same problem Illuminate\Queue\MaxAttemptsExceededException: Maatwebsite\Excel\Jobs\AppendQueryToSheet has been attempted too many times or run too long. The job may have previously timed out.. Using the most recent version of Laravel (9) and this plugin.

Seems to me the Laravel job property $timeout should be respected? See https://laravel.com/docs/9.x/queues#timeout

[alejandrosanchez18]

Any solution? I have the same problem

[pmediavictor]

Any solution?

I am having same issue.

Illuminate\Queue\MaxAttemptsExceededException: Maatwebsite\Excel\Jobs\AppendDataToSheet has been attempted too many times or run too long. The job may have previously timed out.

[oldravian]

I'm also having the same issue.
timeout parameter in

php artisan queue:work is not working

[coderste]

Any chance of an answer for this one?

I have the export run in it's own Job and the timeout set to 10 minutes but it seems like the export functionality of the package is still trying to get it done within the 1 minute default and therefore fails with the below because of the large dataset.

Illuminate\Queue\MaxAttemptsExceededException: Maatwebsite\Excel\Jobs\AppendQueryToSheet has been attempted too many times or run too long. The job may have previously timed out.

[patrickbrouwers]

If you run it in your own job and don't use any queueing logic of the package, it will run as long as you want. Make sure to follow all steps in e.g. https://medium.com/@williamvicary/long-running-jobs-with-laravel-horizon-7655e34752f7, simply extending the timeout to 10 minutes won't work, as Laravel's queue worker also kills jobs after a certain time (retry_after) (see article for more info)

[coderste]

I have it running in it's own job now with a timeout set to 1800 seconds and then the retryUntil() function returning 35 minutes. I sadly don't have horizon installed so just using the default redis queuing system. Trying to build a large CSV export of >250k records but these timeouts still seem to fail with the same error.

The query is all pretty much hand written (so it's optimised as much as possible). Any ideas?

[patrickbrouwers]

You can find the information without horizon here: https://cosme.dev/post/how-to-handle-longrunning-jobs-in-laravel You probably have the retry_after at 60s

[coderste]

Will give it a go thank you for the response 😄

[chatisk]

@patrickbrouwers

Yeah but, you can't split the Export class into smaller jobs. That would create multiple files, which misses the whole point of the package