Train deep networks #63

rouson · 2023-06-23T12:03:29Z

This PR

Incorporates the training algorithm contributed by @davytorres,
Adds related unit tests for training a two-layer network to represent a digital AND gate,
Adds a program in the example/ subdirectory based on the new unit tests.
To run the new example, execute

./setup.sh
./build/run-fpm.sh run --example train-and-gate

This eliminates a segmentation fault caused by a mistaken assumption of the allocate-on-assignment for what was actually a fixed-size left-hand-side array.

The current training algorithm has a hardwired architecture. The next step is to generalize the code.

This starts the process of generalizing the new deep network training algorithm by querying the passed-object dummy argument for the network properties (num_inputs, neurons_per_layer, num_outputs) instead of hardwiring the properties.

This commit 1. Continues the previous commit's pattern of using introspection on passed-object dummy argument in lieu of hardwired values. 2. Removes unnecessary real-kind conversions. 3. Minor house keeping to combine some declarations.

This commit moves a duplicated block of code in the two inference_engine_t constructor functions to one, private subroutine, which sets the activation_strategy_ component.

app/neural_network.f90

…ning ci: use newer version of gcc

rouson added 23 commits June 10, 2023 21:37

feat(expected_output_t): make constructor pure:

3687193

test: add deep and-gate network training test

d73688b

feat(example): add deep and-gate training demo

c0ffa16

fix(test/example): make actual_output allocatable

5740c94

This eliminates a segmentation fault caused by a mistaken assumption of the allocate-on-assignment for what was actually a fixed-size left-hand-side array.

feat(app): add alternative training algorithm

1cf6d22

feat(train):build trainable_engine_t from alt code

b970c7c

feat(train): switch to single precision

eb8127d

feat(app): generate all random numbers at start

845a257

feat(app): rm double-precision conversions

a897ca2

refac(app): rm unused file open

da8340d

feat(inputs_t): private component + constructor

06d8677

refac(inputs_t): use canonical nomenclature

8b8080d

fix(app): work around gfortran bug 100650

c6188bb

refac(app): prep for conversion to subroutine

d108a58

test(train_deep_network): draft unit test

2237aab

refac(app):finish prep to convert to subroutine

3e715cc

refac(app): convert to subroutine

baa5f32

feat(trainable_engine): deep net tests pass

dbcc202

The current training algorithm has a hardwired architecture. The next step is to generalize the code.

chore(app): rm standalone training code

18fbfb3

refac(train): query self for network properties

7aa0b90

This starts the process of generalizing the new deep network training algorithm by querying the passed-object dummy argument for the network properties (num_inputs, neurons_per_layer, num_outputs) instead of hardwiring the properties.

refac(train): introspection, rm kind converions

1dad9e7

This commit 1. Continues the previous commit's pattern of using introspection on passed-object dummy argument in lieu of hardwired values. 2. Removes unnecessary real-kind conversions. 3. Minor house keeping to combine some declarations.

test(train): reduce iterations to shorten runtime

fbc9520

doc(example): deep learning demo of new train algo

fb41b3d

rouson requested a review from davytorres June 23, 2023 12:09

rouson added 6 commits June 23, 2023 23:26

refac(train): use matmul in forward pass, backprop

b852b64

refac(train): def cost/delta in array statements

3a4eea8

refac(train): define gradient in array statements

a4cb842

refac(train): shorten subscripts; rm unused vars

63ccd8c

refac(train): rm unnecessary allocation

5893199

refac(train): uniform 2-space indentation

98d90f4

rouson added 7 commits June 25, 2023 12:48

refac(train):switch bias update to array statement

cf228d0

refac(train): associate input_output_pair array

b38026d

feat(train): generalize activation strategy

65b5f05

refac(inference_engine):activation strategy setter

d46753d

This commit moves a duplicated block of code in the two inference_engine_t constructor functions to one, private subroutine, which sets the activation_strategy_ component.

refac(train): array constructor sets layer widths

cda0d8b

feat(train): finish generalizing activation

9662ac4

refac(train): label loops; combine initializations

dbaf0b9

rouson force-pushed the merge-alternate-training branch from d30fda1 to dbaf0b9 Compare June 27, 2023 14:15

rouson commented Jun 27, 2023

View reviewed changes

app/neural_network.f90 Outdated Show resolved Hide resolved

everythingfunctional and others added 2 commits June 27, 2023 10:33

ci: use newer version of gcc

b80f209

Merge pull request #64 from everythingfunctional/merge-alternate-trai…

2aa0173

…ning ci: use newer version of gcc

rouson merged commit 234d57f into main Jun 28, 2023

rouson deleted the merge-alternate-training branch June 28, 2023 07:24

rouson removed the request for review from davytorres June 28, 2023 07:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Train deep networks #63

Train deep networks #63

rouson commented Jun 23, 2023 •

edited

Loading

Train deep networks #63

Train deep networks #63

Conversation

rouson commented Jun 23, 2023 • edited Loading

rouson commented Jun 23, 2023 •

edited

Loading