Run Torchvision problems with Benchmark[Problem/Runner/Metric]; consolidate PyTorchCNN problems #2688
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D61414680 |
|
This pull request was exported from Phabricator. Differential Revision: D61414680 |
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Aug 21, 2024
…lidate PyTorchCNN problems (facebook#2688) Summary: Pull Request resolved: facebook#2688 Context: Consolidating on common abstractions will make it easier to add functionality to all classes. Also, this will make the code smaller and easier to navigate. This diff is substantially LOC-negative outside of tests. This PR: * Updates functionality in torchvision.py that replaces MNIST datasets with fakes when run in a test environment; the data sets are now realistic enough to be usable. * Merge the functionality in `pytorch_cnn.py` into `torchvision.py`; it was only ever used in order to support `torchvision.py`. * Remove `PyTorchCNNTorchvisionBenchmarkProblem` and its special serialization logic; it is replaced with `BenchmarkProblem` * Remove `PyTorchCNNTorchvisionRunner`, `PyTorchCNNBenchmarkProblem`, `PyTorchCNNMetric`, and `PyTorchCNNRunner` * Introduce `PyTorchCNNTorchvisionParamBasedProblem`. It does not need special serialization logic because it is a dataclass with datasets constructed in the `__post_init__`. When an instance is serialized, the data sets are not serialized; they are reconstructed when the instance is decoded. Using a dataclass here also allows for an automatic and more rigorous equality check. * Use `BenchmarkRunner`; as per D61483962, this means that this problem now has a ground truth, which won't change its behavior since it doesn't have noise added. Differential Revision: D61414680
esantorella
force-pushed
the
export-D61414680
branch
from
391d538 to
b4284d7
Compare
|
This pull request was exported from Phabricator. Differential Revision: D61414680 |
esantorella
force-pushed
the
export-D61414680
branch
from
b4284d7 to
e550e56
Compare
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Aug 22, 2024
…lidate PyTorchCNN problems (facebook#2688) Summary: Pull Request resolved: facebook#2688 Context: Consolidating on common abstractions will make it easier to add functionality to all classes. Also, this will make the code smaller and easier to navigate. This diff is substantially LOC-negative outside of tests. This PR: * Updates functionality in torchvision.py that replaces MNIST datasets with fakes when run in a test environment; the data sets are now realistic enough to be usable. * Merge the functionality in `pytorch_cnn.py` into `torchvision.py`; it was only ever used in order to support `torchvision.py`. * Remove `PyTorchCNNTorchvisionBenchmarkProblem` and its special serialization logic; it is replaced with `BenchmarkProblem` * Remove `PyTorchCNNTorchvisionRunner`, `PyTorchCNNBenchmarkProblem`, `PyTorchCNNMetric`, and `PyTorchCNNRunner` * Introduce `PyTorchCNNTorchvisionParamBasedProblem`. It does not need special serialization logic because it is a dataclass with datasets constructed in the `__post_init__`. When an instance is serialized, the data sets are not serialized; they are reconstructed when the instance is decoded. Using a dataclass here also allows for an automatic and more rigorous equality check. * Use `BenchmarkRunner`; as per D61483962, this means that this problem now has a ground truth, which won't change its behavior since it doesn't have noise added. Reviewed By: Balandat Differential Revision: D61414680
|
This pull request was exported from Phabricator. Differential Revision: D61414680 |
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Aug 22, 2024
…lidate PyTorchCNN problems (facebook#2688) Summary: Pull Request resolved: facebook#2688 Context: Consolidating on common abstractions will make it easier to add functionality to all classes. Also, this will make the code smaller and easier to navigate. This diff is substantially LOC-negative outside of tests. This PR: * Updates functionality in torchvision.py that replaces MNIST datasets with fakes when run in a test environment; the data sets are now realistic enough to be usable. * Merge the functionality in `pytorch_cnn.py` into `torchvision.py`; it was only ever used in order to support `torchvision.py`. * Remove `PyTorchCNNTorchvisionBenchmarkProblem` and its special serialization logic; it is replaced with `BenchmarkProblem` * Remove `PyTorchCNNTorchvisionRunner`, `PyTorchCNNBenchmarkProblem`, `PyTorchCNNMetric`, and `PyTorchCNNRunner` * Introduce `PyTorchCNNTorchvisionParamBasedProblem`. It does not need special serialization logic because it is a dataclass with datasets constructed in the `__post_init__`. When an instance is serialized, the data sets are not serialized; they are reconstructed when the instance is decoded. Using a dataclass here also allows for an automatic and more rigorous equality check. * Use `BenchmarkRunner`; as per D61483962, this means that this problem now has a ground truth, which won't change its behavior since it doesn't have noise added. Reviewed By: Balandat Differential Revision: D61414680
esantorella
force-pushed
the
export-D61414680
branch
from
e550e56 to
eb58242
Compare
|
This pull request was exported from Phabricator. Differential Revision: D61414680 |
esantorella
force-pushed
the
export-D61414680
branch
from
eb58242 to
1d0d80f
Compare
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Aug 22, 2024
…lidate PyTorchCNN problems (facebook#2688) Summary: Pull Request resolved: facebook#2688 Context: Consolidating on common abstractions will make it easier to add functionality to all classes. Also, this will make the code smaller and easier to navigate. This diff is substantially LOC-negative outside of tests. This PR: * Updates functionality in torchvision.py that replaces MNIST datasets with fakes when run in a test environment; the data sets are now realistic enough to be usable. * Merge the functionality in `pytorch_cnn.py` into `torchvision.py`; it was only ever used in order to support `torchvision.py`. * Remove `PyTorchCNNTorchvisionBenchmarkProblem` and its special serialization logic; it is replaced with `BenchmarkProblem` * Remove `PyTorchCNNTorchvisionRunner`, `PyTorchCNNBenchmarkProblem`, `PyTorchCNNMetric`, and `PyTorchCNNRunner` * Introduce `PyTorchCNNTorchvisionParamBasedProblem`. It does not need special serialization logic because it is a dataclass with datasets constructed in the `__post_init__`. When an instance is serialized, the data sets are not serialized; they are reconstructed when the instance is decoded. Using a dataclass here also allows for an automatic and more rigorous equality check. * Use `BenchmarkRunner`; as per D61483962, this means that this problem now has a ground truth, which won't change its behavior since it doesn't have noise added. Reviewed By: Balandat Differential Revision: D61414680
|
This pull request was exported from Phabricator. Differential Revision: D61414680 |
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Aug 22, 2024
…lidate PyTorchCNN problems (facebook#2688) Summary: Pull Request resolved: facebook#2688 Context: Consolidating on common abstractions will make it easier to add functionality to all classes. Also, this will make the code smaller and easier to navigate. This diff is substantially LOC-negative outside of tests. This PR: * Updates functionality in torchvision.py that replaces MNIST datasets with fakes when run in a test environment; the data sets are now realistic enough to be usable. * Merge the functionality in `pytorch_cnn.py` into `torchvision.py`; it was only ever used in order to support `torchvision.py`. * Remove `PyTorchCNNTorchvisionBenchmarkProblem` and its special serialization logic; it is replaced with `BenchmarkProblem` * Remove `PyTorchCNNTorchvisionRunner`, `PyTorchCNNBenchmarkProblem`, `PyTorchCNNMetric`, and `PyTorchCNNRunner` * Introduce `PyTorchCNNTorchvisionParamBasedProblem`. It does not need special serialization logic because it is a dataclass with datasets constructed in the `__post_init__`. When an instance is serialized, the data sets are not serialized; they are reconstructed when the instance is decoded. Using a dataclass here also allows for an automatic and more rigorous equality check. * Use `BenchmarkRunner`; as per D61483962, this means that this problem now has a ground truth, which won't change its behavior since it doesn't have noise added. Reviewed By: Balandat Differential Revision: D61414680
esantorella
force-pushed
the
export-D61414680
branch
from
1d0d80f to
3117f41
Compare
|
This pull request was exported from Phabricator. Differential Revision: D61414680 |
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Aug 23, 2024
…lidate PyTorchCNN problems (facebook#2688) Summary: Pull Request resolved: facebook#2688 Context: Consolidating on common abstractions will make it easier to add functionality to all classes. Also, this will make the code smaller and easier to navigate. This diff is substantially LOC-negative outside of tests. This PR: * Updates functionality in torchvision.py that replaces MNIST datasets with fakes when run in a test environment; the data sets are now realistic enough to be usable. * Merge the functionality in `pytorch_cnn.py` into `torchvision.py`; it was only ever used in order to support `torchvision.py`. * Remove `PyTorchCNNTorchvisionBenchmarkProblem` and its special serialization logic; it is replaced with `BenchmarkProblem` * Remove `PyTorchCNNTorchvisionRunner`, `PyTorchCNNBenchmarkProblem`, `PyTorchCNNMetric`, and `PyTorchCNNRunner` * Introduce `PyTorchCNNTorchvisionParamBasedProblem`. It does not need special serialization logic because it is a dataclass with datasets constructed in the `__post_init__`. When an instance is serialized, the data sets are not serialized; they are reconstructed when the instance is decoded. Using a dataclass here also allows for an automatic and more rigorous equality check. * Use `BenchmarkRunner`; as per D61483962, this means that this problem now has a ground truth, which won't change its behavior since it doesn't have noise added. Reviewed By: Balandat Differential Revision: D61414680
esantorella
force-pushed
the
export-D61414680
branch
from
3117f41 to
703505e
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2688 +/- ##
==========================================
+ Coverage 95.28% 95.39% +0.10%
==========================================
Files 493 492 -1
Lines 47919 47916 -3
==========================================
+ Hits 45662 45711 +49
+ Misses 2257 2205 -52 ☔ View full report in Codecov by Sentry. |
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Aug 23, 2024
…lidate PyTorchCNN problems (facebook#2688) Summary: Pull Request resolved: facebook#2688 Context: Consolidating on common abstractions will make it easier to add functionality to all classes. Also, this will make the code smaller and easier to navigate. This diff is substantially LOC-negative outside of tests. This PR: * Updates functionality in torchvision.py that replaces MNIST datasets with fakes when run in a test environment; the data sets are now realistic enough to be usable. * Merge the functionality in `pytorch_cnn.py` into `torchvision.py`; it was only ever used in order to support `torchvision.py`. * Remove `PyTorchCNNTorchvisionBenchmarkProblem` and its special serialization logic; it is replaced with `BenchmarkProblem` * Remove `PyTorchCNNTorchvisionRunner`, `PyTorchCNNBenchmarkProblem`, `PyTorchCNNMetric`, and `PyTorchCNNRunner` * Introduce `PyTorchCNNTorchvisionParamBasedProblem`. It does not need special serialization logic because it is a dataclass with datasets constructed in the `__post_init__`. When an instance is serialized, the data sets are not serialized; they are reconstructed when the instance is decoded. Using a dataclass here also allows for an automatic and more rigorous equality check. * Use `BenchmarkRunner`; as per D61483962, this means that this problem now has a ground truth, which won't change its behavior since it doesn't have noise added. Differential Revision: D61414680 Reviewed By: Balandat
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Aug 24, 2024
…lidate PyTorchCNN problems (facebook#2688) Summary: Pull Request resolved: facebook#2688 Context: Consolidating on common abstractions will make it easier to add functionality to all classes. Also, this will make the code smaller and easier to navigate. This diff is substantially LOC-negative outside of tests. This PR: * Updates functionality in torchvision.py that replaces MNIST datasets with fakes when run in a test environment; the data sets are now realistic enough to be usable. * Merge the functionality in `pytorch_cnn.py` into `torchvision.py`; it was only ever used in order to support `torchvision.py`. * Remove `PyTorchCNNTorchvisionBenchmarkProblem` and its special serialization logic; it is replaced with `BenchmarkProblem` * Remove `PyTorchCNNTorchvisionRunner`, `PyTorchCNNBenchmarkProblem`, `PyTorchCNNMetric`, and `PyTorchCNNRunner` * Introduce `PyTorchCNNTorchvisionParamBasedProblem`. It does not need special serialization logic because it is a dataclass with datasets constructed in the `__post_init__`. When an instance is serialized, the data sets are not serialized; they are reconstructed when the instance is decoded. Using a dataclass here also allows for an automatic and more rigorous equality check. * Use `BenchmarkRunner`; as per D61483962, this means that this problem now has a ground truth, which won't change its behavior since it doesn't have noise added. Differential Revision: D61414680 Reviewed By: Balandat
|
This pull request was exported from Phabricator. Differential Revision: D61414680 |
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Aug 24, 2024
…lidate PyTorchCNN problems (facebook#2688) Summary: Pull Request resolved: facebook#2688 Context: Consolidating on common abstractions will make it easier to add functionality to all classes. Also, this will make the code smaller and easier to navigate. This diff is substantially LOC-negative outside of tests. This PR: * Updates functionality in torchvision.py that replaces MNIST datasets with fakes when run in a test environment; the data sets are now realistic enough to be usable. * Merge the functionality in `pytorch_cnn.py` into `torchvision.py`; it was only ever used in order to support `torchvision.py`. * Remove `PyTorchCNNTorchvisionBenchmarkProblem` and its special serialization logic; it is replaced with `BenchmarkProblem` * Remove `PyTorchCNNTorchvisionRunner`, `PyTorchCNNBenchmarkProblem`, `PyTorchCNNMetric`, and `PyTorchCNNRunner` * Introduce `PyTorchCNNTorchvisionParamBasedProblem`. It does not need special serialization logic because it is a dataclass with datasets constructed in the `__post_init__`. When an instance is serialized, the data sets are not serialized; they are reconstructed when the instance is decoded. Using a dataclass here also allows for an automatic and more rigorous equality check. * Use `BenchmarkRunner`; as per D61483962, this means that this problem now has a ground truth, which won't change its behavior since it doesn't have noise added. Reviewed By: Balandat Differential Revision: D61414680
esantorella
force-pushed
the
export-D61414680
branch
from
703505e to
e8b10b8
Compare
|
This pull request was exported from Phabricator. Differential Revision: D61414680 |
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Aug 24, 2024
…lidate PyTorchCNN problems (facebook#2688) Summary: Pull Request resolved: facebook#2688 Context: Consolidating on common abstractions will make it easier to add functionality to all classes. Also, this will make the code smaller and easier to navigate. This diff is substantially LOC-negative outside of tests. This PR: * Updates functionality in torchvision.py that replaces MNIST datasets with fakes when run in a test environment; the data sets are now realistic enough to be usable. * Merge the functionality in `pytorch_cnn.py` into `torchvision.py`; it was only ever used in order to support `torchvision.py`. * Remove `PyTorchCNNTorchvisionBenchmarkProblem` and its special serialization logic; it is replaced with `BenchmarkProblem` * Remove `PyTorchCNNTorchvisionRunner`, `PyTorchCNNBenchmarkProblem`, `PyTorchCNNMetric`, and `PyTorchCNNRunner` * Introduce `PyTorchCNNTorchvisionParamBasedProblem`. It does not need special serialization logic because it is a dataclass with datasets constructed in the `__post_init__`. When an instance is serialized, the data sets are not serialized; they are reconstructed when the instance is decoded. Using a dataclass here also allows for an automatic and more rigorous equality check. * Use `BenchmarkRunner`; as per D61483962, this means that this problem now has a ground truth, which won't change its behavior since it doesn't have noise added. Reviewed By: Balandat Differential Revision: D61414680
esantorella
force-pushed
the
export-D61414680
branch
from
e8b10b8 to
dea323a
Compare
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Aug 24, 2024
…lidate PyTorchCNN problems (facebook#2688) Summary: Pull Request resolved: facebook#2688 Context: Consolidating on common abstractions will make it easier to add functionality to all classes. Also, this will make the code smaller and easier to navigate. This diff is substantially LOC-negative outside of tests. This PR: * Updates functionality in torchvision.py that replaces MNIST datasets with fakes when run in a test environment; the data sets are now realistic enough to be usable. * Merge the functionality in `pytorch_cnn.py` into `torchvision.py`; it was only ever used in order to support `torchvision.py`. * Remove `PyTorchCNNTorchvisionBenchmarkProblem` and its special serialization logic; it is replaced with `BenchmarkProblem` * Remove `PyTorchCNNTorchvisionRunner`, `PyTorchCNNBenchmarkProblem`, `PyTorchCNNMetric`, and `PyTorchCNNRunner` * Introduce `PyTorchCNNTorchvisionParamBasedProblem`. It does not need special serialization logic because it is a dataclass with datasets constructed in the `__post_init__`. When an instance is serialized, the data sets are not serialized; they are reconstructed when the instance is decoded. Using a dataclass here also allows for an automatic and more rigorous equality check. * Use `BenchmarkRunner`; as per D61483962, this means that this problem now has a ground truth, which won't change its behavior since it doesn't have noise added. Differential Revision: D61414680 Reviewed By: Balandat
|
This pull request was exported from Phabricator. Differential Revision: D61414680 |
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Aug 24, 2024
…lidate PyTorchCNN problems (facebook#2688) Summary: Pull Request resolved: facebook#2688 Context: Consolidating on common abstractions will make it easier to add functionality to all classes. Also, this will make the code smaller and easier to navigate. This diff is substantially LOC-negative outside of tests. This PR: * Updates functionality in torchvision.py that replaces MNIST datasets with fakes when run in a test environment; the data sets are now realistic enough to be usable. * Merge the functionality in `pytorch_cnn.py` into `torchvision.py`; it was only ever used in order to support `torchvision.py`. * Remove `PyTorchCNNTorchvisionBenchmarkProblem` and its special serialization logic; it is replaced with `BenchmarkProblem` * Remove `PyTorchCNNTorchvisionRunner`, `PyTorchCNNBenchmarkProblem`, `PyTorchCNNMetric`, and `PyTorchCNNRunner` * Introduce `PyTorchCNNTorchvisionParamBasedProblem`. It does not need special serialization logic because it is a dataclass with datasets constructed in the `__post_init__`. When an instance is serialized, the data sets are not serialized; they are reconstructed when the instance is decoded. Using a dataclass here also allows for an automatic and more rigorous equality check. * Use `BenchmarkRunner`; as per D61483962, this means that this problem now has a ground truth, which won't change its behavior since it doesn't have noise added. Reviewed By: Balandat Differential Revision: D61414680
esantorella
force-pushed
the
export-D61414680
branch
from
dea323a to
c14e2b1
Compare
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Aug 24, 2024
…lidate PyTorchCNN problems (facebook#2688) Summary: Pull Request resolved: facebook#2688 Context: Consolidating on common abstractions will make it easier to add functionality to all classes. Also, this will make the code smaller and easier to navigate. This diff is substantially LOC-negative outside of tests. This PR: * Updates functionality in torchvision.py that replaces MNIST datasets with fakes when run in a test environment; the data sets are now realistic enough to be usable. * Merge the functionality in `pytorch_cnn.py` into `torchvision.py`; it was only ever used in order to support `torchvision.py`. * Remove `PyTorchCNNTorchvisionBenchmarkProblem` and its special serialization logic; it is replaced with `BenchmarkProblem` * Remove `PyTorchCNNTorchvisionRunner`, `PyTorchCNNBenchmarkProblem`, `PyTorchCNNMetric`, and `PyTorchCNNRunner` * Introduce `PyTorchCNNTorchvisionParamBasedProblem`. It does not need special serialization logic because it is a dataclass with datasets constructed in the `__post_init__`. When an instance is serialized, the data sets are not serialized; they are reconstructed when the instance is decoded. Using a dataclass here also allows for an automatic and more rigorous equality check. * Use `BenchmarkRunner`; as per D61483962, this means that this problem now has a ground truth, which won't change its behavior since it doesn't have noise added. Differential Revision: D61414680 Reviewed By: Balandat
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Aug 25, 2024
…lidate PyTorchCNN problems (facebook#2688) Summary: Pull Request resolved: facebook#2688 Context: Consolidating on common abstractions will make it easier to add functionality to all classes. Also, this will make the code smaller and easier to navigate. This diff is substantially LOC-negative outside of tests. This PR: * Updates functionality in torchvision.py that replaces MNIST datasets with fakes when run in a test environment; the data sets are now realistic enough to be usable. * Merge the functionality in `pytorch_cnn.py` into `torchvision.py`; it was only ever used in order to support `torchvision.py`. * Remove `PyTorchCNNTorchvisionBenchmarkProblem` and its special serialization logic; it is replaced with `BenchmarkProblem` * Remove `PyTorchCNNTorchvisionRunner`, `PyTorchCNNBenchmarkProblem`, `PyTorchCNNMetric`, and `PyTorchCNNRunner` * Introduce `PyTorchCNNTorchvisionParamBasedProblem`. It does not need special serialization logic because it is a dataclass with datasets constructed in the `__post_init__`. When an instance is serialized, the data sets are not serialized; they are reconstructed when the instance is decoded. Using a dataclass here also allows for an automatic and more rigorous equality check. * Use `BenchmarkRunner`; as per D61483962, this means that this problem now has a ground truth, which won't change its behavior since it doesn't have noise added. Differential Revision: D61414680 Reviewed By: Balandat
|
This pull request was exported from Phabricator. Differential Revision: D61414680 |
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Aug 25, 2024
…lidate PyTorchCNN problems (facebook#2688) Summary: Pull Request resolved: facebook#2688 Context: Consolidating on common abstractions will make it easier to add functionality to all classes. Also, this will make the code smaller and easier to navigate. This diff is substantially LOC-negative outside of tests. This PR: * Updates functionality in torchvision.py that replaces MNIST datasets with fakes when run in a test environment; the data sets are now realistic enough to be usable. * Merge the functionality in `pytorch_cnn.py` into `torchvision.py`; it was only ever used in order to support `torchvision.py`. * Remove `PyTorchCNNTorchvisionBenchmarkProblem` and its special serialization logic; it is replaced with `BenchmarkProblem` * Remove `PyTorchCNNTorchvisionRunner`, `PyTorchCNNBenchmarkProblem`, `PyTorchCNNMetric`, and `PyTorchCNNRunner` * Introduce `PyTorchCNNTorchvisionParamBasedProblem`. It does not need special serialization logic because it is a dataclass with datasets constructed in the `__post_init__`. When an instance is serialized, the data sets are not serialized; they are reconstructed when the instance is decoded. Using a dataclass here also allows for an automatic and more rigorous equality check. * Use `BenchmarkRunner`; as per D61483962, this means that this problem now has a ground truth, which won't change its behavior since it doesn't have noise added. Reviewed By: Balandat Differential Revision: D61414680
esantorella
force-pushed
the
export-D61414680
branch
from
c14e2b1 to
8e2a32a
Compare
|
This pull request was exported from Phabricator. Differential Revision: D61414680 |
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Aug 26, 2024
…lidate PyTorchCNN problems (facebook#2688) Summary: Pull Request resolved: facebook#2688 Context: Consolidating on common abstractions will make it easier to add functionality to all classes. Also, this will make the code smaller and easier to navigate. This diff is substantially LOC-negative outside of tests. This PR: * Updates functionality in torchvision.py that replaces MNIST datasets with fakes when run in a test environment; the data sets are now realistic enough to be usable. * Merge the functionality in `pytorch_cnn.py` into `torchvision.py`; it was only ever used in order to support `torchvision.py`. * Remove `PyTorchCNNTorchvisionBenchmarkProblem` and its special serialization logic; it is replaced with `BenchmarkProblem` * Remove `PyTorchCNNTorchvisionRunner`, `PyTorchCNNBenchmarkProblem`, `PyTorchCNNMetric`, and `PyTorchCNNRunner` * Introduce `PyTorchCNNTorchvisionParamBasedProblem`. It does not need special serialization logic because it is a dataclass with datasets constructed in the `__post_init__`. When an instance is serialized, the data sets are not serialized; they are reconstructed when the instance is decoded. Using a dataclass here also allows for an automatic and more rigorous equality check. * Use `BenchmarkRunner`; as per D61483962, this means that this problem now has a ground truth, which won't change its behavior since it doesn't have noise added. Reviewed By: Balandat Differential Revision: D61414680
esantorella
force-pushed
the
export-D61414680
branch
from
8e2a32a to
c4c1935
Compare
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Aug 26, 2024
…lidate PyTorchCNN problems (facebook#2688) Summary: Pull Request resolved: facebook#2688 Context: Consolidating on common abstractions will make it easier to add functionality to all classes. Also, this will make the code smaller and easier to navigate. This diff is substantially LOC-negative outside of tests. This PR: * Updates functionality in torchvision.py that replaces MNIST datasets with fakes when run in a test environment; the data sets are now realistic enough to be usable. * Merge the functionality in `pytorch_cnn.py` into `torchvision.py`; it was only ever used in order to support `torchvision.py`. * Remove `PyTorchCNNTorchvisionBenchmarkProblem` and its special serialization logic; it is replaced with `BenchmarkProblem` * Remove `PyTorchCNNTorchvisionRunner`, `PyTorchCNNBenchmarkProblem`, `PyTorchCNNMetric`, and `PyTorchCNNRunner` * Introduce `PyTorchCNNTorchvisionParamBasedProblem`. It does not need special serialization logic because it is a dataclass with datasets constructed in the `__post_init__`. When an instance is serialized, the data sets are not serialized; they are reconstructed when the instance is decoded. Using a dataclass here also allows for an automatic and more rigorous equality check. * Use `BenchmarkRunner`; as per D61483962, this means that this problem now has a ground truth, which won't change its behavior since it doesn't have noise added. Differential Revision: D61414680 Reviewed By: Balandat
|
This pull request was exported from Phabricator. Differential Revision: D61414680 |
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Aug 26, 2024
…lidate PyTorchCNN problems (facebook#2688) Summary: Pull Request resolved: facebook#2688 Context: Consolidating on common abstractions will make it easier to add functionality to all classes. Also, this will make the code smaller and easier to navigate. This diff is substantially LOC-negative outside of tests. This PR: * Updates functionality in torchvision.py that replaces MNIST datasets with fakes when run in a test environment; the data sets are now realistic enough to be usable. * Merge the functionality in `pytorch_cnn.py` into `torchvision.py`; it was only ever used in order to support `torchvision.py`. * Remove `PyTorchCNNTorchvisionBenchmarkProblem` and its special serialization logic; it is replaced with `BenchmarkProblem` * Remove `PyTorchCNNTorchvisionRunner`, `PyTorchCNNBenchmarkProblem`, `PyTorchCNNMetric`, and `PyTorchCNNRunner` * Introduce `PyTorchCNNTorchvisionParamBasedProblem`. It does not need special serialization logic because it is a dataclass with datasets constructed in the `__post_init__`. When an instance is serialized, the data sets are not serialized; they are reconstructed when the instance is decoded. Using a dataclass here also allows for an automatic and more rigorous equality check. * Use `BenchmarkRunner`; as per D61483962, this means that this problem now has a ground truth, which won't change its behavior since it doesn't have noise added. Reviewed By: Balandat Differential Revision: D61414680
esantorella
force-pushed
the
export-D61414680
branch
from
c4c1935 to
23587e4
Compare
|
This pull request was exported from Phabricator. Differential Revision: D61414680 |
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Aug 26, 2024
…lidate PyTorchCNN problems (facebook#2688) Summary: Pull Request resolved: facebook#2688 Context: Consolidating on common abstractions will make it easier to add functionality to all classes. Also, this will make the code smaller and easier to navigate. This diff is substantially LOC-negative outside of tests. This PR: * Updates functionality in torchvision.py that replaces MNIST datasets with fakes when run in a test environment; the data sets are now realistic enough to be usable. * Merge the functionality in `pytorch_cnn.py` into `torchvision.py`; it was only ever used in order to support `torchvision.py`. * Remove `PyTorchCNNTorchvisionBenchmarkProblem` and its special serialization logic; it is replaced with `BenchmarkProblem` * Remove `PyTorchCNNTorchvisionRunner`, `PyTorchCNNBenchmarkProblem`, `PyTorchCNNMetric`, and `PyTorchCNNRunner` * Introduce `PyTorchCNNTorchvisionParamBasedProblem`. It does not need special serialization logic because it is a dataclass with datasets constructed in the `__post_init__`. When an instance is serialized, the data sets are not serialized; they are reconstructed when the instance is decoded. Using a dataclass here also allows for an automatic and more rigorous equality check. * Use `BenchmarkRunner`; as per D61483962, this means that this problem now has a ground truth, which won't change its behavior since it doesn't have noise added. Reviewed By: Balandat Differential Revision: D61414680
esantorella
force-pushed
the
export-D61414680
branch
from
23587e4 to
8454732
Compare
…lidate PyTorchCNN problems (facebook#2688) Summary: Pull Request resolved: facebook#2688 Context: Consolidating on common abstractions will make it easier to add functionality to all classes. Also, this will make the code smaller and easier to navigate. This diff is substantially LOC-negative outside of tests. This PR: * Updates functionality in torchvision.py that replaces MNIST datasets with fakes when run in a test environment; the data sets are now realistic enough to be usable. * Merge the functionality in `pytorch_cnn.py` into `torchvision.py`; it was only ever used in order to support `torchvision.py`. * Remove `PyTorchCNNTorchvisionBenchmarkProblem` and its special serialization logic; it is replaced with `BenchmarkProblem` * Remove `PyTorchCNNTorchvisionRunner`, `PyTorchCNNBenchmarkProblem`, `PyTorchCNNMetric`, and `PyTorchCNNRunner` * Introduce `PyTorchCNNTorchvisionParamBasedProblem`. It does not need special serialization logic because it is a dataclass with datasets constructed in the `__post_init__`. When an instance is serialized, the data sets are not serialized; they are reconstructed when the instance is decoded. Using a dataclass here also allows for an automatic and more rigorous equality check. * Use `BenchmarkRunner`; as per D61483962, this means that this problem now has a ground truth, which won't change its behavior since it doesn't have noise added. Reviewed By: Balandat Differential Revision: D61414680
|
This pull request was exported from Phabricator. Differential Revision: D61414680 |
esantorella
force-pushed
the
export-D61414680
branch
from
8454732 to
1c042f8
Compare
|
This pull request has been merged in f1d2f88. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
Context: Consolidating on common abstractions will make it easier to add functionality to all classes. Also, this will make the code smaller and easier to navigate. This diff is substantially LOC-negative outside of tests.
This PR:
pytorch_cnn.pyintotorchvision.py; it was only ever used in order to supporttorchvision.py.PyTorchCNNTorchvisionBenchmarkProblemand its special serialization logic; it is replaced withBenchmarkProblemPyTorchCNNTorchvisionRunner,PyTorchCNNBenchmarkProblem,PyTorchCNNMetric, andPyTorchCNNRunnerPyTorchCNNTorchvisionParamBasedProblem. It does not need special serialization logic because it is a dataclass with datasets constructed in the__post_init__. When an instance is serialized, the data sets are not serialized; they are reconstructed when the instance is decoded. Using a dataclass here also allows for an automatic and more rigorous equality check.BenchmarkRunner; as per D61483962, this means that this problem now has a ground truth, which won't change its behavior since it doesn't have noise added.Differential Revision: D61414680