diff --git a/released_box/README.md b/released_box/README.md index 1e7cce5..347018b 100644 --- a/released_box/README.md +++ b/released_box/README.md @@ -51,6 +51,7 @@ train_val: - patience: *int=10*, define value coordinate with tolerance to expand detect length. [10, 100]. - backend: *str='threading'*, configure accelerate backend used in inner process. 'threading', 'multiprocessing', 'loky'. - n_jobs: *int=-1*, define numbers of jobs with manually set by users need. -1 or any int value > 0. (if n_jobs=1, parallel processing will be turn off to save cuda memory.) +- prefer: *str='threads'*, configure soft hint to choose the default backend. 'threads', 'processes'. (prefer 'threading' & 'threads' when users try fails by setting 'loky' and 'processes' or turn to v1.6.1.) - early_stop: *bool=False*, define whether to enable early_stop process. False or True. test: @@ -68,7 +69,7 @@ save or load: | `__init__` | input_: int
num_classes: int
hidden_layer_sizes: Tuple[int]=(100,)
device: str='cuda'
*
activation: str='relu'
inplace_on: bool=False
criterion: str='CrossEntropyLoss'
solver: str='adam'
batch_size: int=32
learning_rate_init: float=1e-2
lr_scheduler: Optional[str]=None | Initialize Classifier or Regressier Based on Basic Information of the Dataset Obtained through Data Preprocessing and Feature Engineering. | | print_config | / | Return Initialized Parameters of Multi-layer Perceptron and Graph. | | data_loader | features: TabularData
labels: TabularData
ratio_set: Dict[str, int]={'train': 8, 'test': 1, 'val': 1}
worker_set: Dict[str, int]={'train': 8, 'test': 2, 'val': 1}
random_seed: Optional[int]=None | Using `ratio_set` and `worker_set` to Load the Numpy Dataset into `torch.utils.data.DataLoader`. | -| train_val | num_epochs: int=2
interval: int=100
tolerance: float=1e-3
patience: int=10
backend: str='threading'
n_jobs: int=-1
early_stop: bool=False | Using `num_epochs`, `tolerance`, `patience` to Control Training Process and `interval` to Adjust Print Interval with Accelerated Validation Combined with `backend` and `n_jobs`. | +| train_val | num_epochs: int=2
interval: int=100
tolerance: float=1e-3
patience: int=10
backend: str='threading'
n_jobs: int=-1
prefer: str='threads'
early_stop: bool=False | Using `num_epochs`, `tolerance`, `patience` to Control Training Process and `interval` to Adjust Print Interval with Accelerated Validation Combined with `backend` and `n_jobs`. | | test | sort_by: str='accuracy'
sort_state: bool=True | Sort Returned Test Result about Correct Classes with `sort_by` and `sort_state` Which Only Appears in Classification. | | save | con: bool=True
dir: str='./model' | Save Trained Model Parameters with Model `state_dict` Control by `con`. | | load | con: bool=True
dir: str='./model' | Load Trained Model Parameters with Model `state_dict` Control by `con`. | @@ -83,7 +84,7 @@ save or load: | print_config | / | Return Initialized Parameters of Multi-layer Perceptron and Graph. | | data_loader | features: TabularData
labels: TabularData
ratio_set: Dict[str, int]={'train': 8, 'test': 1, 'val': 1}
worker_set: Dict[str, int]={'train': 8, 'test': 2, 'val': 1}
random_seed: Optional[int]=None | Using `ratio_set` and `worker_set` to Load the Regression Dataset with Numpy format into `torch.utils.data.DataLoader`. | | set_freeze | require_grad: Dict[int, bool] | freeze some layers by given `requires_grad=False` if trained model will be loaded to execute experiments. | -| train_val | num_epochs: int=2
interval: int=100
tolerance: float=1e-3
patience: int=10
backend: str='threading'
n_jobs: int=-1
early_stop: bool=False | Using `num_epochs`, `tolerance`, `patience` to Control Training Process and `interval` to Adjust Print Interval with Accelerated Validation Combined with `backend` and `n_jobs`. | +| train_val | num_epochs: int=2
interval: int=100
tolerance: float=1e-3
patience: int=10
backend: str='threading'
n_jobs: int=-1
prefer: str='threads'
early_stop: bool=False | Using `num_epochs`, `tolerance`, `patience` to Control Training Process and `interval` to Adjust Print Interval with Accelerated Validation Combined with `backend` and `n_jobs`. | | test | / | Test Module Only Show with Loss at 3 Stages: Train, Test, Val | | save | con: bool=True
dir: str='./model' | Save Trained Model Parameters with Model `state_dict` Control by `con`. | | load | con: bool=True
dir: str='./model' | Load Trained Model Parameters with Model `state_dict` Control by `con`. | @@ -96,7 +97,7 @@ save or load: | print_config | / | Return Initialized Parameters of Multi-layer Perceptron and Graph. | | data_loader | features: TabularData
labels: TabularData
ratio_set: Dict[str, int]={'train': 8, 'test': 1, 'val': 1}
worker_set: Dict[str, int]={'train': 8, 'test': 2, 'val': 1}
random_seed: Optional[int]=None | Using `ratio_set` and `worker_set` to Load the Binary-classification Dataset with Numpy format into `torch.utils.data.DataLoader`. | | set_freeze | require_grad: Dict[int, bool] | freeze some layers by given `requires_grad=False` if trained model will be loaded to execute experiments. | -| train_val | num_epochs: int=2
interval: int=100
tolerance: float=1e-3
patience: int=10
backend: str='threading'
n_jobs: int=-1
early_stop: bool=False | Using `num_epochs`, `tolerance`, `patience` to Control Training Process and `interval` to Adjust Print Interval with Accelerated Validation Combined with `backend` and `n_jobs`. | +| train_val | num_epochs: int=2
interval: int=100
tolerance: float=1e-3
patience: int=10
backend: str='threading'
n_jobs: int=-1
prefer: str='threads'
early_stop: bool=False | Using `num_epochs`, `tolerance`, `patience` to Control Training Process and `interval` to Adjust Print Interval with Accelerated Validation Combined with `backend` and `n_jobs`. | | test | sort_by: str='accuracy'
sort_state: bool=True | Test Module con with Correct Class and Loss at 3 Stages: Train, Test, Val | | save | con: bool=True
dir: str='./model' | Save Trained Model Parameters with Model `state_dict` Control by `con`. | | load | con: bool=True
dir: str='./model' | Load Trained Model Parameters with Model `state_dict` Control by `con`. | @@ -109,7 +110,7 @@ save or load: | print_config | / | Return Initialized Parameters of Multi-layer Perceptron and Graph. | | data_loader | features: TabularData
labels: TabularData
ratio_set: Dict[str, int]={'train': 8, 'test': 1, 'val': 1}
worker_set: Dict[str, int]={'train': 8, 'test': 2, 'val': 1}
random_seed: Optional[int]=None | Using `ratio_set` and `worker_set` to Load the Multi-classification Dataset with Numpy format into `torch.utils.data.DataLoader`. | | set_freeze | require_grad: Dict[int, bool] | freeze some layers by given `requires_grad=False` if trained model will be loaded to execute experiments. | -| train_val | num_epochs: int=2
interval: int=100
tolerance: float=1e-3
patience: int=10
backend: str='threading'
n_jobs: int=-1
early_stop: bool=False | Using `num_epochs`, `tolerance`, `patience` to Control Training Process and `interval` to Adjust Print Interval with Accelerated Validation Combined with `backend` and `n_jobs`. | +| train_val | num_epochs: int=2
interval: int=100
tolerance: float=1e-3
patience: int=10
backend: str='threading'
n_jobs: int=-1
prefer: str='threads'
early_stop: bool=False | Using `num_epochs`, `tolerance`, `patience` to Control Training Process and `interval` to Adjust Print Interval with Accelerated Validation Combined with `backend` and `n_jobs`. | | test | sort_by: str='accuracy'
sort_state: bool=True | Sort Returned Test Result about Correct Classes with `sort_by` and `sort_state` Which Only Appears in Classification. | | save | con: bool=True
dir: str='./model' | Save Trained Model Parameters with Model `state_dict` Control by `con`. | | load | con: bool=True
dir: str='./model' | Load Trained Model Parameters with Model `state_dict` Control by `con`. | @@ -122,7 +123,7 @@ save or load: | print_config | / | Return Initialized Parameters of Multi-layer Perceptron and Graph. | | data_loader | features: TabularData
labels: TabularData
ratio_set: Dict[str, int]={'train': 8, 'test': 1, 'val': 1}
worker_set: Dict[str, int]={'train': 8, 'test': 2, 'val': 1}
random_seed: Optional[int]=None | Using `ratio_set` and `worker_set` to Load the Multi-outputs Dataset with Numpy format into `torch.utils.data.DataLoader`. | | set_freeze | require_grad: Dict[int, bool] | freeze some layers by given `requires_grad=False` if trained model will be loaded to execute experiments. | -| train_val | num_epochs: int=2
interval: int=100
tolerance: float=1e-3
patience: int=10
backend: str='threading'
n_jobs: int=-1
early_stop: bool=False | Using `num_epochs`, `tolerance`, `patience` to Control Training Process and `interval` to Adjust Print Interval with Accelerated Validation Combined with `backend` and `n_jobs`. | +| train_val | num_epochs: int=2
interval: int=100
tolerance: float=1e-3
patience: int=10
backend: str='threading'
n_jobs: int=-1
prefer: str='threads'
early_stop: bool=False | Using `num_epochs`, `tolerance`, `patience` to Control Training Process and `interval` to Adjust Print Interval with Accelerated Validation Combined with `backend` and `n_jobs`. | | test | / | Test Module Only Show with Loss at 3 Stages: Train, Test, Val | | save | con: bool=True
dir: str='./model' | Save Trained Model Parameters with Model `state_dict` Control by `con`. | | load | con: bool=True
dir: str='./model' | Load Trained Model Parameters with Model `state_dict` Control by `con`. | diff --git a/released_box/perming/__init__.py b/released_box/perming/__init__.py index 0c684bc..96ec0ac 100644 --- a/released_box/perming/__init__.py +++ b/released_box/perming/__init__.py @@ -27,4 +27,4 @@ 'Multi-outputs': Ranker } -__version__ = '1.9.2' \ No newline at end of file +__version__ = '1.9.3' \ No newline at end of file diff --git a/released_box/perming/_utils.py b/released_box/perming/_utils.py index 7a745df..65cad50 100644 --- a/released_box/perming/_utils.py +++ b/released_box/perming/_utils.py @@ -1,7 +1,7 @@ # Copyright (c) 2023 linjing-lab import torch, random, numpy, gc -from joblib import parallel_backend, Parallel, delayed +from joblib import parallel_config, Parallel, delayed from collections import OrderedDict from ._typing import TabularData, List, Tuple, Dict, Optional, Any from ._version import parse_torch_version @@ -220,6 +220,7 @@ def train_val(self, patience: int=10, backend: str='threading', n_jobs: int=-1, + prefer: str='threads', early_stop: bool=False) -> None: ''' Training and Validation with `train_loader` and `val_container`. @@ -229,6 +230,7 @@ def train_val(self, :param patience: int, patience of no improvement waiting for training to stop. default: 10. :param backend: str, 'threading', 'multiprocessing', 'loky'. default: 'threading'. :param n_jobs: int, accelerate processing of validation. default: -1. + :param prefer: str, set 'threads' when 'threading' and set 'processes' when 'multiprocessing' or 'loky'. default: 'threads'. :param early_stop: bool, whether to enable early_stop in train_val. default: False. ''' assert num_epochs > 0 and interval > 0, 'With num_epochs > 0 and interval > 0 to train parameters into outputs.' @@ -258,7 +260,7 @@ def train_val(self, # validation with val_container self.val_loss = 0 # int value at cpu - with parallel_backend(backend, n_jobs=n_jobs): + with parallel_config(backend, n_jobs=n_jobs, prefer=prefer): Parallel()(delayed(self._val_acc)(val_set) for val_set in self.val_container) self.val_loss /= val_length val_counts = i + 1 + total_step * epoch # times of val_loss renewed diff --git a/released_box/requirements.txt b/released_box/requirements.txt index 87a6db0..fa2cf67 100644 --- a/released_box/requirements.txt +++ b/released_box/requirements.txt @@ -1,4 +1,4 @@ -joblib>=1.1.0 +joblib>=1.3.2 numpy>=1.24.0 setuptools>=63.4.1 torch==1.7.1+cu101 diff --git a/released_box/setup.py b/released_box/setup.py index e1cce8b..a7c6095 100644 --- a/released_box/setup.py +++ b/released_box/setup.py @@ -59,7 +59,7 @@ ], install_requires=[ 'numpy>=1.21.0', - 'joblib>=1.1.0' + 'joblib>=1.3.2' ] # extras_require=[] ) \ No newline at end of file diff --git a/released_box/tests/Multi-classification Task.ipynb b/released_box/tests/Multi-classification Task.ipynb index 7ae4582..281708b 100644 --- a/released_box/tests/Multi-classification Task.ipynb +++ b/released_box/tests/Multi-classification Task.ipynb @@ -374,7 +374,7 @@ } ], "source": [ - "main.train_val(num_epochs=1, interval=100, early_stop=True) # set n_jobs > 1 within number of processes" + "main.train_val(num_epochs=1, interval=100, backend='threading', prefer='threads', early_stop=True) # set n_jobs > 1 within number of processes" ] }, {