Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stop_signal (SIGTERM) is not sent to old container after livepush #1079

Closed
adipascu opened this issue Sep 9, 2019 · 2 comments
Closed

stop_signal (SIGTERM) is not sent to old container after livepush #1079

adipascu opened this issue Sep 9, 2019 · 2 comments
Assignees

Comments

@adipascu
Copy link

adipascu commented Sep 9, 2019

I am doing local development, I deploy my code using balena push xxxx.local.
After each new push, the old container does not receive SIGTERM, this causes code that does GPU memory allocation to leak.

Here are some examples written in node.js .

setInterval(Function.prototype, 1000); // keep app open
process.on('SIGTERM', () => {
  console.log("Received SIGTERM"); // <- This is not executed
  process.exit(0);
});

Here is an example that leaks using the pigpio library, it relies on SIGTERM internally to do cleanup.

setInterval(Function.prototype, 1000); // keep app open
try {
  require('fs').unlinkSync('/var/run/pigpio.pid');
  console.warn('Cleaned up previous pigpio lock file, we leaked memory in a previous container :(');
} catch (e) {
  // Should to fail, lock file should not exist
}
const { Gpio } = require('pigpio');
new Gpio(18, { mode: Gpio.OUTPUT }).digitalWrite(false);
@CameronDiver
Copy link
Contributor

So I think the reason that this is occurring is that livepush sends a SIGKILL by default, to speed up the restart process. It sounds like we need a method of specifying whether we kill directly, or follow the standard docker process of sending SIGTERM and then killing if the container does not reply after a set amount of time.

@CameronDiver
Copy link
Contributor

CameronDiver commented Sep 13, 2019

I've created an issue on the livepush repo here to track this: balena-io-modules/livepush#49 and I'm going to close this issue, as it's not related to the supervisor.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants