Skip to content

Commit

Permalink
Description and comments refined
Browse files Browse the repository at this point in the history
  • Loading branch information
luav committed Jun 21, 2018
1 parent 842dc94 commit 7231a05
Show file tree
Hide file tree
Showing 3 changed files with 3 additions and 6 deletions.
1 change: 0 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,6 @@ Large executing jobs can be postponed for the later execution with less number o

Demo of the *scheduling with memory constraints* for the worker processes:
![mpepool_memory](images/mpepool_mem.png)
> Zombie processes still may appear on abnormal termination of worker processes during rescheduling due to some bug in the Python interpreter. Anyway all resources are clearer up and the zombies disappear on the execution pool completion.

Demo of the *scheduling with cache L1 maximization* for single-threaded processes on the server with cross-node CPUs enumeration. Whole physical CPU core consisting of two hardware threads assigned to each worker process, so the L1 cache is dedicated (not shared), but the maximal loading over all CPUs is 50%:
![mpepool_cacheL1_1](images/mpepool_cacheL1_1.png)
Expand Down
7 changes: 3 additions & 4 deletions mpepool.py
Original file line number Diff line number Diff line change
Expand Up @@ -2227,7 +2227,7 @@ def __start(self, job, concur=True):
# Kill non-terminated process
if active:
if job.proc.poll() is None:
print(' Killing ~zombie "{}" #{} ...'.format(job.name, job.proc.pid), file=sys.stderr)
print(' Killing ~hanged "{}" #{} ...'.format(job.name, job.proc.pid), file=sys.stderr)
job.proc.kill()
self.__complete(job, False)
# ATTENTION: re-raise exception for the BaseException but not Exception sub-classes
Expand Down Expand Up @@ -2523,7 +2523,6 @@ def __reviseWorkers(self):
memall = self.memlimit + memov # Note: memov is negative here

# Process completed (and terminated) jobs: execute callbacks and remove the workers
#cterminated = False # Completed terminated procs processed
for job in completed:
# Note: check for the termination in all cycles
if not self.alive:
Expand All @@ -2533,7 +2532,6 @@ def __reviseWorkers(self):
exectime = job.tstop - job.tstart
# Restart the job if it was terminated and should be restarted
if not job.terminates:
#cterminated = True
continue
print('WARNING, "{}" #{} is terminated because of the {} violation'
', chtermtime: {}, consumes {:.4f} / {:.4f} GB, timeout {:.4f} sec, executed: {:.4f} sec ({} h {} m {:.4f} s)'
Expand Down Expand Up @@ -2575,7 +2573,8 @@ def __reviseWorkers(self):
self.__postpone(job, True)
# Note: the number of workers is not reduced to less than 1

# Note: active_children() does not impact on the existence of zombie procs
# Note: active_children() does not impact on the existence of zombie procs,
# proc table clearup implemented in complete() using wait()
#if cterminated:
# # Note: required to join terminated child procs and avoid zombies
# # Return list of all live children of the current process,joining any processes which have already finished
Expand Down
1 change: 0 additions & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@

setup(
name=pkgname, # This is the name of your PyPI-package.
# version='2.2.0', # Update the version number for new releases
version='3.1.0', # Update the version number for new releases
description=('A lightweight multi-process Execution Pool with load balancing'
' and customizable resource consumption constraints.'), # Required, "Summary" metadata field
Expand Down

0 comments on commit 7231a05

Please sign in to comment.