This project uses configuration files instead of command line arguments to simplify script execution. A default config is provided in the configs directory. Additionally, a new config file with some preconfigurations will automatically be created after preprocessing a new dataset.
The following documentation describes all configuration options. Please remember to modify the configuration before running the according scripts.
The system path of the dataset to use, containing the images, background, and annotations. During inference, this is the dataset containing the source actor annotations (poses).
If set to false, the dataset will be preloaded into RAM.
Percentage of data (starting from the beginning) that will be used for training. The rest will be used as validation set. For example, a value of 1.0 indicates that all samples are used for training, while a 0.8 will cut the last 20% of the video for validation/testing.
Number of body part segmentation labels.We use the ATR dataset containing 18 unique labels.
Number of channels (limbs) in the skeleton representation. Default is 11.
List containing all segmentation labels refering to clothes. Only labels in this list will be processed by the garmnent structure network.
Segmentation label for the scene background
Path of the scene background. During inference, this should be the background used for training the requested render network component.
Directory name of the dataset used to train the executed network, if pose normalization is required. To use the unadjusted skeletons (e.g. during training), this parameter should be set to self.
Image width of the target actor dataset (equals current dataset width during training).
Image height of the target actor dataset (equals current dataset width during training).
Name of the current training run, used for checkpoint and tensorboard log file naming.
Output path for tensorboard logs.
Putput path for training checkpoints.
Distance between training checkpoints (in epochs). E.g. a value of 5 will generate a training checkpoint every 5 epochs. Use -1 to disable.
List of relative trainingset indices (in range [0, 1]) used for training progress visualization in tensorboard.
List of relative validationset indices (in range [0, 1]) used for training progress visualization in tensorboard.
Index of GPU to train on.
Number of training epochs
Activates GPU training
VGG layers used in perceptive loss
Contains parameters for the rendering (appearance) network component.
First beta parameter for ADAM optimization.
Second beta parameter for ADAM optimization.
Set to False to skip training the rendering (appearance) component.
Path of rendering network checkpoint to resume training from. Use None to train from scratch.
The optimizer learning rate.
Decay factor applied to the learning rate.
Number of of optimization steps before learning rate decay is applied.
Perceptive (VGG) loss lambda for background fusion.
Reconstruction (L1) loss lambda for background fusion.
Perceptive (VGG) loss lambda for actor generation.
Reconstruction (L1) loss lambda for actor generation.
Contains parameters for the segmentation (shape) network component.
First beta parameter for ADAM optimization.
Second beta parameter for ADAM optimization.
Set to False to skip training the segmentation (shape) component.
Path of segmentation network checkpoint to resume training from. Use None to train from scratch.
The optimizer learning rate.
Decay factor applied to the learning rate.
Number of of optimization steps before learning rate decay is applied.
Binary cross entropy (BCE) lambda for segmentation learning.
Contains parameters for the structure network component.
First beta parameter for ADAM optimization.
Second beta parameter for ADAM optimization.
Set to False to skip training the clothing structure component.
Path of structure network checkpoint to resume training from. Use None to train from scratch.
The optimizer learning rate.
Decay factor applied to the learning rate.
Number of of optimization steps before learning rate decay is applied.
Reconstruction (L1) loss lambda for structure learning.
Activates inference on GPU
Index of GPU to run networks on.
Path of the segmentation (shape) network checkpoint used for reenactment.
Path of the structure network checkpoint used for reenactment.
Path of the render (appearance) network checkpoint used for reenactment.
If set to true, pseudo gt shape annotations from the dataset will be used for reenactment instead of network estimates
If set to true, pseudo gt structure annotations from the dataset will be used for reenactment instead of network estimates
Number of initial iterations to compensate first zero input in recurrent components.
Constant factor applied to the input structure.
Path of the output directory.
If set to true, the ground truth source actor image used to extract the input motion sequence will be horizontally concatenated to the final output image for direct comparison.
If true, the estimated shape will be saved to the output directory.
If true, the estimated structure will be saved to the output directory.
If true, the dataset validation split will be used for reenactment.
If true, a video is automatically generated from the output images using ffmpeg.
The framerate used for output video generation.