From 53c692ad5da15b65137dd1805f28b4752a7497b1 Mon Sep 17 00:00:00 2001 From: Prabhu Teja Date: Mon, 9 Oct 2023 10:44:02 +0200 Subject: [PATCH 1/3] Documentation changes --- doc/benchmarking/renate_benchmarks.rst | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/doc/benchmarking/renate_benchmarks.rst b/doc/benchmarking/renate_benchmarks.rst index f914eb84..871db375 100644 --- a/doc/benchmarking/renate_benchmarks.rst +++ b/doc/benchmarking/renate_benchmarks.rst @@ -97,7 +97,14 @@ The full list of models and model names including a short description is provide - Wrapper around Hugging Face transformers. - * ``pretrained_model_name_or_path``: Hugging Face `transformer ID `__. * ``num_outputs``: The number of classes. - + * - `~renate.benchmark.models.l2p.LearningToPromptTransformer` + - `Learning to Prompt Transformer `_. Supports both text and vision transformers. + - * ``pretrained_model_name_or_path``: Hugging Face `transformer ID `__. + * ``num_outputs``: The number of classes. + * ``pool_size``: Total number of prompts in the prompt pool. + * ``pool_selection_size``: Number of prompts to select for each input from the pool. + * ``prompt_size``: Number of input tokens each prompt is equivalent to. + * ``prompt_key_dim``: Dimenensionality of the features used for prompt matching. .. _benchmarking-renate-benchmarks-datasets: @@ -173,6 +180,14 @@ The following table contains the list of supported datasets. - multiple - Any `Hugging Face dataset `__ can be used. Just prepend the prefix ``hfd-``, e.g., ``hfd-rotten_tomatoes``. Select input and target columns via ``config_space``, e.g., add ``"input_column": "text", "target_column": "label"`` for the `rotten_tomatoes `__ example. - Please refer to `the official documentation `__. + * - CDDB + - Image Classification + - 2 classes from 5 domains of generated images using various techniques. Numbers vary across domains. + - Li, Chuqiao, et al. A continual deepfake detection benchmark: Dataset, methods, and essentials. IEEE/CVF Winter Conference on Applications of Computer Vision. 2023. + * - CORe50 + - Object Recognition + - 10 classes of 7 training domains, 3 other domains used for evaluation + - Vincenzo Lomonaco and Davide Maltoni: CORe50: a new Dataset and Benchmark for continual Object Recognition. 1st Annual Conference on Robot Learning, PMLR 78:17-26, 2017. .. _benchmarking-renate-benchmarks-scenarios: From d55c2ec77215eb79659559aea28d10ff0bdec56b Mon Sep 17 00:00:00 2001 From: Prabhu Teja Date: Mon, 9 Oct 2023 10:49:36 +0200 Subject: [PATCH 2/3] update to text --- doc/benchmarking/renate_benchmarks.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/benchmarking/renate_benchmarks.rst b/doc/benchmarking/renate_benchmarks.rst index 871db375..d7fc9597 100644 --- a/doc/benchmarking/renate_benchmarks.rst +++ b/doc/benchmarking/renate_benchmarks.rst @@ -185,7 +185,7 @@ The following table contains the list of supported datasets. - 2 classes from 5 domains of generated images using various techniques. Numbers vary across domains. - Li, Chuqiao, et al. A continual deepfake detection benchmark: Dataset, methods, and essentials. IEEE/CVF Winter Conference on Applications of Computer Vision. 2023. * - CORe50 - - Object Recognition + - Object Recognition - image classfication - 10 classes of 7 training domains, 3 other domains used for evaluation - Vincenzo Lomonaco and Davide Maltoni: CORe50: a new Dataset and Benchmark for continual Object Recognition. 1st Annual Conference on Robot Learning, PMLR 78:17-26, 2017. From 902006ff75e349e5cb703232d01ae2f9a6041b8a Mon Sep 17 00:00:00 2001 From: Prabhu Teja Date: Mon, 9 Oct 2023 11:54:21 +0200 Subject: [PATCH 3/3] addressing comments --- doc/benchmarking/renate_benchmarks.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/benchmarking/renate_benchmarks.rst b/doc/benchmarking/renate_benchmarks.rst index d7fc9597..3bb602e6 100644 --- a/doc/benchmarking/renate_benchmarks.rst +++ b/doc/benchmarking/renate_benchmarks.rst @@ -181,12 +181,12 @@ The following table contains the list of supported datasets. - Any `Hugging Face dataset `__ can be used. Just prepend the prefix ``hfd-``, e.g., ``hfd-rotten_tomatoes``. Select input and target columns via ``config_space``, e.g., add ``"input_column": "text", "target_column": "label"`` for the `rotten_tomatoes `__ example. - Please refer to `the official documentation `__. * - CDDB - - Image Classification - - 2 classes from 5 domains of generated images using various techniques. Numbers vary across domains. + - Image Classification: deepfake detection + - 2 classes, 5 domains, each generated using image generation techniques: GauGAN, BigGAN, WildDeepfake, WhichFaceReal, SAN respectively from HARD evaluation scenario. Numbers vary across domains. - Li, Chuqiao, et al. A continual deepfake detection benchmark: Dataset, methods, and essentials. IEEE/CVF Winter Conference on Applications of Computer Vision. 2023. - * - CORe50 - - Object Recognition - image classfication - - 10 classes of 7 training domains, 3 other domains used for evaluation + * - Core50 + - Image Classfication + - 50 classes, 8 (0-7) domains for training, a single test set for evaluation. - Vincenzo Lomonaco and Davide Maltoni: CORe50: a new Dataset and Benchmark for continual Object Recognition. 1st Annual Conference on Robot Learning, PMLR 78:17-26, 2017. .. _benchmarking-renate-benchmarks-scenarios: