Collabora Online - Built-in CODE Server - log size #187

amg-web · 2022-06-18T08:51:32Z

we are using Collabora Online - Built-in CODE Server
after 2 months without restart log file became very big (~2.8 Gb) in /tmp partition using big part of space
also remaining some flog files after restarts.
need to configure log rotation and removal to reduce logs size.

mokkin · 2024-06-03T09:06:16Z

I didn't realize that before, but we now also have this issue running on NC 29.0.1 with a size of the /tmp/coolwsd.*/jails directory of around 54G
The uptime is 34 days and we are using Collabora / Nextcloud Office only occasionally.

red3333 · 2024-06-13T12:35:32Z

Same problem here.
I just realized that /tmp/systemd-private-*-apache2.service-*/tmp/coolwsd.*/jails also reached 54GB. Uptime around 109 days (but apache2 was restarted several times).
I just stopped apache2, deleted its /tmp/systemd-private-*-apache2.service-*/ directory and restarted it. Let's see if it comes back...

Configuration:
Nextcloud 29.0.2
Collabora Online Development Edition 24.04.2.1 80a6f97
Apache/2.4.59 (Debian)

Cris70 · 2024-06-17T10:48:50Z

Same here, right after updating to Nextcloud v29.0.2 from Nextcloud 28.
It filled up my /tmp disk.
I removed part of the offending files from /tmp, but the directory is still growing at an alarming rate.
How to fix?

djonik1562 · 2024-07-31T18:36:31Z

I have same problem.
It's growning really fast.

Githopp192 · 2024-08-04T12:55:12Z

same problem here ==>

#176

/tmp/coolwsd.*/jails is growing very fast - 100GB in 24h !

Without monitoring the system - this would lead to a severe server crash

epidemiaf1 · 2024-08-08T22:09:09Z

Yeap, I can confirm this is an issue with Nextcloud 29.0.4 and Collabora Online 24.4.502. Server crashed this morning and had to hard reboot it. After 5 hours, it's already at 17GB (output from du 17G ./coolwsd.*/jails).

bastien30 · 2024-08-09T09:37:45Z

Hello,

Same here, after updating to Nextcloud 29 from 28 on docker-compose stack.

collabora/code:latest and nextcloud:stable from docker hub.

Githopp192 · 2024-08-09T11:05:55Z

Same here, right after updating to Nextcloud v29.0.2 from Nextcloud 28. It filled up my /tmp disk. I removed part of the offending files from /tmp, but the directory is still growing at an alarming rate. How to fix?

@Cris70 - what i did as workaround .. developped some script, which will:

check /tmp/php-fpm ever 15 minutes
defined a critical threshold value for /tmp and very critical threshold value for /tmp
if critcal threshold value is reached, script will check if this is within business hours, when not, then
- set cloud maintenance: on, wait 5 minutes
- restart apache,php-fpm, redis - this will clear the *coolwsd" stuff
- - set cloud maintenance: off
if very critcal threshold value is reached, script force to immediately :

set cloud maintenance: on, wait 5 minutes
restart apache,php-fpm, redis - this will clear the *coolwsd" stuff
set cloud maintenance: off

epidemiaf1 · 2024-08-09T13:11:39Z

We do use Nextcloud office but it isn't a critical part of the setup. I temporarily disabled it and the Collabora Online server within Nextcloud. Had to restart apache and php8.2-fpm service as it kept growing the jail files regardless.

ulfkosack · 2024-08-16T11:40:11Z

Same here after update to NC from 28.0.4 to 29.0.5
I've disabled CODE Server and Nextcloud Office

cableTh0rn · 2024-08-20T07:54:44Z

Same issue here
/tmp/ folder is filled up with coolwsd/jail whatever over 700GB
restaring nginx, php-fpm ,redis did not remove tmp files
I had to delete temp stuff manually then restarted services
kindda disappointed

Githopp192 · 2024-08-20T07:58:00Z

Same issue here /tmp/ folder is filled up with coolwsd/jail whatever over 700GB restaring nginx, php-fpm ,redis did not remove tmp files I had to delete temp stuff manually then restarted services kindda disappointed

==> since i disabled the maintenance specifc run ('maintenance_window_start' => 100, (disabling it), the issue did not re-occur !?

(issue = long running cron & increasing /tmp/php stuff)

nextcloud/server#47132

cableTh0rn · 2024-08-21T06:52:04Z

Same issue here /tmp/ folder is filled up with coolwsd/jail whatever over 700GB restaring nginx, php-fpm ,redis did not remove tmp files I had to delete temp stuff manually then restarted services kindda disappointed

==> since i disabled the maintenance specifc run ('maintenance_window_start' => 100, (disabling it), the issue did not re-occur !?

(issue = long running cron & increasing /tmp/php stuff)

nextcloud/server#47132

I run a check script to log the time tmp files gets bigger. It is just the maintenace_window time, 100GB in just one hour

'''
Wed Aug 21 04:38:42 +03 2024 --> 1.9G /tmp/
Wed Aug 21 04:38:52 +03 2024 --> 1.9G /tmp/
Wed Aug 21 04:39:02 +03 2024 --> 1.9G /tmp/
Wed Aug 21 04:39:12 +03 2024 --> 1.9G /tmp/
Wed Aug 21 04:39:22 +03 2024 --> 1.9G /tmp/
Wed Aug 21 04:39:32 +03 2024 --> 1.9G /tmp/
Wed Aug 21 04:39:42 +03 2024 --> 1.9G /tmp/
Wed Aug 21 04:39:52 +03 2024 --> 1.9G /tmp/
Wed Aug 21 04:40:02 +03 2024 --> 1.9G /tmp/
Wed Aug 21 04:40:12 +03 2024 --> 2.3G /tmp/
Wed Aug 21 04:40:22 +03 2024 --> 2.9G /tmp/
Wed Aug 21 04:40:33 +03 2024 --> 3.5G /tmp/
Wed Aug 21 04:40:43 +03 2024 --> 3.7G /tmp/
Wed Aug 21 04:40:53 +03 2024 --> 3.7G /tmp/
Wed Aug 21 04:41:03 +03 2024 --> 3.7G /tmp/
Wed Aug 21 04:41:13 +03 2024 --> 3.7G /tmp/
Wed Aug 21 04:41:23 +03 2024 --> 3.8G /tmp/
Wed Aug 21 04:41:33 +03 2024 --> 3.7G /tmp/
Wed Aug 21 04:41:43 +03 2024 --> 3.7G /tmp/
Wed Aug 21 04:41:53 +03 2024 --> 3.7G /tmp/
Wed Aug 21 04:42:03 +03 2024 --> 3.7G /tmp/
Wed Aug 21 04:42:13 +03 2024 --> 3.8G /tmp/
Wed Aug 21 04:42:23 +03 2024 --> 3.8G /tmp/
Wed Aug 21 04:42:33 +03 2024 --> 4.1G /tmp/
Wed Aug 21 04:42:43 +03 2024 --> 4.5G /tmp/
Wed Aug 21 04:42:54 +03 2024 --> 4.8G /tmp/
Wed Aug 21 04:43:04 +03 2024 --> 5.8G /tmp/
Wed Aug 21 04:43:14 +03 2024 --> 7.2G /tmp/
Wed Aug 21 04:43:24 +03 2024 --> 8.0G /tmp/
Wed Aug 21 04:43:34 +03 2024 --> 9.2G /tmp/
...
Wed Aug 21 04:43:34 +03 2024 --> 9.2G /tmp/
Wed Aug 21 04:43:44 +03 2024 --> 11G /tmp/
Wed Aug 21 04:43:54 +03 2024 --> 12G /tmp/
Wed Aug 21 04:44:04 +03 2024 --> 13G /tmp/
Wed Aug 21 04:44:15 +03 2024 --> 14G /tmp/
Wed Aug 21 04:44:25 +03 2024 --> 15G /tmp/
Wed Aug 21 04:44:35 +03 2024 --> 16G /tmp/
Wed Aug 21 04:44:45 +03 2024 --> 17G /tmp/
Wed Aug 21 04:44:55 +03 2024 --> 18G /tmp/
Wed Aug 21 04:45:05 +03 2024 --> 19G /tmp/
Wed Aug 21 04:45:15 +03 2024 --> 20G /tmp/
Wed Aug 21 04:45:25 +03 2024 --> 21G /tmp/
Wed Aug 21 04:45:35 +03 2024 --> 21G /tmp/
Wed Aug 21 04:45:45 +03 2024 --> 22G /tmp/
Wed Aug 21 04:45:55 +03 2024 --> 23G /tmp/
Wed Aug 21 04:46:06 +03 2024 --> 25G /tmp/
Wed Aug 21 04:46:16 +03 2024 --> 26G /tmp/
Wed Aug 21 04:46:26 +03 2024 --> 27G /tmp/
Wed Aug 21 04:46:36 +03 2024 --> 28G /tmp/
Wed Aug 21 04:46:46 +03 2024 --> 28G /tmp/
Wed Aug 21 04:46:56 +03 2024 --> 30G /tmp/
Wed Aug 21 04:47:06 +03 2024 --> 31G /tmp/
Wed Aug 21 04:47:16 +03 2024 --> 32G /tmp/
Wed Aug 21 04:47:26 +03 2024 --> 33G /tmp/
Wed Aug 21 04:47:36 +03 2024 --> 34G /tmp/
Wed Aug 21 04:47:47 +03 2024 --> 35G /tmp/
Wed Aug 21 04:47:57 +03 2024 --> 36G /tmp/
Wed Aug 21 04:48:07 +03 2024 --> 36G /tmp/
Wed Aug 21 04:48:17 +03 2024 --> 37G /tmp/
...
Wed Aug 21 04:58:24 +03 2024 --> 97G /tmp/
Wed Aug 21 04:58:34 +03 2024 --> 98G /tmp/
Wed Aug 21 04:58:45 +03 2024 --> 99G /tmp/
Wed Aug 21 04:58:55 +03 2024 --> 99G /tmp/
Wed Aug 21 04:59:05 +03 2024 --> 99G /tmp/
Wed Aug 21 04:59:15 +03 2024 --> 99G /tmp/
Wed Aug 21 04:59:25 +03 2024 --> 99G /tmp/
Wed Aug 21 04:59:35 +03 2024 --> 99G /tmp/
Wed Aug 21 04:59:45 +03 2024 --> 99G /tmp/
Wed Aug 21 04:59:55 +03 2024 --> 99G /tmp/
Wed Aug 21 05:00:06 +03 2024 --> 99G /tmp/
Wed Aug 21 05:00:16 +03 2024 --> 99G /tmp/

joshtrichards · 2024-08-21T13:21:30Z

Same here, after updating to Nextcloud 29 from 28 on docker-compose stack.
collabora/code:latest and nextcloud:stable from docker hub.

@bastien30

If you're experiencing this in collabora/code:latest than this isn't a richdocumentscode matter (which might be a very useful clue actually; at least if your situation has the same underlying cause as other reporters here).

joshtrichards · 2024-08-21T13:34:31Z

The logging in Built-in / richdocumentscode is just the same as default for CODE in general from the looks of it: warning level.

https://sdk.collaboraonline.com/docs/installation/Configuration.html?highlight=logging#logging

Since Built-in is mostly for testing and personal use, log rotation was likely not an original consideration.

Some of these reports here suggest very rapid log file growth. It would be helpful if one of you reporting this can inventory what precisely is showing up in the logs that is generating so much usage. Is it an unusual error/warning situation? Is it specific to certain environments? Do you have many users? etc.

bastien30 · 2024-08-21T13:44:28Z

Same here, after updating to Nextcloud 29 from 28 on docker-compose stack.
collabora/code:latest and nextcloud:stable from docker hub.

@bastien30

If you're experiencing this in collabora/code:latest than this isn't a richdocumentscode matter (which might be a very useful clue actually; at least if your situation has the same underlying cause as other reporters here).

Thanks, it seems the problem is gone for me, after uninstalling/reinstalling CODE application from within Nextcloud web interface.

joshtrichards mentioned this issue Nov 6, 2023

coolwsd logging is filling /tmp -- even faster by logging it is getting low on space in /tmp... nextcloud/richdocuments#2676

Closed

Githopp192 mentioned this issue Aug 8, 2024

cron: Long running jobs - Memory & cpu footprint will increase continously nextcloud/server#47132

Closed

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Collabora Online - Built-in CODE Server - log size #187

Collabora Online - Built-in CODE Server - log size #187

amg-web commented Jun 18, 2022

mokkin commented Jun 3, 2024

red3333 commented Jun 13, 2024 •

edited

Loading

Cris70 commented Jun 17, 2024 •

edited

Loading

djonik1562 commented Jul 31, 2024

Githopp192 commented Aug 4, 2024

epidemiaf1 commented Aug 8, 2024 •

edited

Loading

bastien30 commented Aug 9, 2024

Githopp192 commented Aug 9, 2024 •

edited

Loading

epidemiaf1 commented Aug 9, 2024

ulfkosack commented Aug 16, 2024

cableTh0rn commented Aug 20, 2024

Githopp192 commented Aug 20, 2024 •

edited

Loading

cableTh0rn commented Aug 21, 2024

joshtrichards commented Aug 21, 2024 •

edited

Loading

joshtrichards commented Aug 21, 2024

bastien30 commented Aug 21, 2024

Collabora Online - Built-in CODE Server - log size #187

Collabora Online - Built-in CODE Server - log size #187

Comments

amg-web commented Jun 18, 2022

mokkin commented Jun 3, 2024

red3333 commented Jun 13, 2024 • edited Loading

Cris70 commented Jun 17, 2024 • edited Loading

djonik1562 commented Jul 31, 2024

Githopp192 commented Aug 4, 2024

epidemiaf1 commented Aug 8, 2024 • edited Loading

bastien30 commented Aug 9, 2024

Githopp192 commented Aug 9, 2024 • edited Loading

epidemiaf1 commented Aug 9, 2024

ulfkosack commented Aug 16, 2024

cableTh0rn commented Aug 20, 2024

Githopp192 commented Aug 20, 2024 • edited Loading

cableTh0rn commented Aug 21, 2024

joshtrichards commented Aug 21, 2024 • edited Loading

joshtrichards commented Aug 21, 2024

bastien30 commented Aug 21, 2024

red3333 commented Jun 13, 2024 •

edited

Loading

Cris70 commented Jun 17, 2024 •

edited

Loading

epidemiaf1 commented Aug 8, 2024 •

edited

Loading

Githopp192 commented Aug 9, 2024 •

edited

Loading

Githopp192 commented Aug 20, 2024 •

edited

Loading

joshtrichards commented Aug 21, 2024 •

edited

Loading