Train identity network from identity with 10% maximum random perturbations #86

rouson · 2023-09-20T03:50:07Z

No description provided.

This commit adds a unit test that verifies that the 'train' procedure preserves an initially exact network that uses a relu activation function to represent an mapping of inputs to identical outputs.

This commit adds a test in which the use of the Adam optimizer greatly speeds convergences of a randomly perturbed identity network with 10% magnitude perturbations. The resulting trained network maps inputs to nearly identical outputs.

rouson added 7 commits September 19, 2023 11:14

chore(app): whitespace edit

eb4d38e

feat(inference_engine_m): add relu activation func

a466101

feat(example): use relu to create linear mapping

e1e378d

feat(relu): make differentiable

c7fcb68

feat(trainable_engine):num_outputs type-bound proc

451d3db

test(train/relu): preserves identity mapping

e372991

This commit adds a unit test that verifies that the 'train' procedure preserves an initially exact network that uses a relu activation function to represent an mapping of inputs to identical outputs.

rouson merged commit 3886ff8 into main Sep 20, 2023
4 checks passed

rouson deleted the perturbed-identity branch September 20, 2023 03:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Train identity network from identity with 10% maximum random perturbations #86

Train identity network from identity with 10% maximum random perturbations #86

rouson commented Sep 20, 2023

Train identity network from identity with 10% maximum random perturbations #86

Train identity network from identity with 10% maximum random perturbations #86

Conversation

rouson commented Sep 20, 2023