-
Notifications
You must be signed in to change notification settings - Fork 1
/
GettingStarted.html
220 lines (218 loc) · 17.4 KB
/
GettingStarted.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.4"/>
<title>OpenANN: Getting Started</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="search/search.js"></script>
<script type="text/javascript">
$(document).ready(function() { searchBox.OnSelectItem(0); });
</script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
<tbody>
<tr style="height: 56px;">
<td id="projectlogo"><img alt="Logo" src="openann-logo-small.png"/></td>
<td style="padding-left: 0.5em;">
<div id="projectname">OpenANN
 <span id="projectnumber">1.1.0</span>
</div>
<div id="projectbrief">An open source library for artificial neural networks.</div>
</td>
</tr>
</tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.4 -->
<script type="text/javascript">
var searchBox = new SearchBox("searchBox", "search",false,'Search');
</script>
<div id="navrow1" class="tabs">
<ul class="tablist">
<li><a href="index.html"><span>Main Page</span></a></li>
<li><a href="annotated.html"><span>Classes</span></a></li>
<li><a href="files.html"><span>Files</span></a></li>
<li>
<div id="MSearchBox" class="MSearchBoxInactive">
<span class="left">
<img id="MSearchSelect" src="search/mag_sel.png"
onmouseover="return searchBox.OnSearchSelectShow()"
onmouseout="return searchBox.OnSearchSelectHide()"
alt=""/>
<input type="text" id="MSearchField" value="Search" accesskey="S"
onfocus="searchBox.OnSearchFieldFocus(true)"
onblur="searchBox.OnSearchFieldFocus(false)"
onkeyup="searchBox.OnSearchFieldChange(event)"/>
</span><span class="right">
<a id="MSearchClose" href="javascript:searchBox.CloseResultsWindow()"><img id="MSearchCloseImg" border="0" src="search/close.png" alt=""/></a>
</span>
</div>
</li>
</ul>
</div>
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
onmouseover="return searchBox.OnSearchSelectShow()"
onmouseout="return searchBox.OnSearchSelectHide()"
onkeydown="return searchBox.OnSearchSelectKey(event)">
<a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(0)"><span class="SelectionMark"> </span>All</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(1)"><span class="SelectionMark"> </span>Classes</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(2)"><span class="SelectionMark"> </span>Namespaces</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(3)"><span class="SelectionMark"> </span>Files</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(4)"><span class="SelectionMark"> </span>Functions</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(5)"><span class="SelectionMark"> </span>Variables</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(6)"><span class="SelectionMark"> </span>Typedefs</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(7)"><span class="SelectionMark"> </span>Enumerations</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(8)"><span class="SelectionMark"> </span>Enumerator</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(9)"><span class="SelectionMark"> </span>Friends</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(10)"><span class="SelectionMark"> </span>Macros</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(11)"><span class="SelectionMark"> </span>Pages</a></div>
<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0"
name="MSearchResults" id="MSearchResults">
</iframe>
</div>
</div><!-- top -->
<div class="header">
<div class="headertitle">
<div class="title">Getting Started </div> </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><p>We will solve a very simple problem here to demonstrate the API of OpenANN.</p>
<h1><a class="anchor" id="XORDataSet"></a>
XOR Data Set</h1>
<p>The XOR problem cannot be solved by the perceptron (a neural network with just one neuron) and was the reason for the death of neural network research in the 70s until backpropagation was discovered.</p>
<p>The data set is simple: </p>
<table class="doxtable">
<tr>
<td><img class="formulaInl" alt="$ x_1 $" src="form_70.png"/></td><td><img class="formulaInl" alt="$ x_2 $" src="form_71.png"/></td><td><img class="formulaInl" alt="$ y_1 $" src="form_72.png"/> </td></tr>
<tr>
<td>0</td><td>1</td><td>1 </td></tr>
<tr>
<td>0</td><td>0</td><td>0 </td></tr>
<tr>
<td>1</td><td>1</td><td>0 </td></tr>
<tr>
<td>1</td><td>0</td><td>1 </td></tr>
</table>
<p>That means <img class="formulaInl" alt="$ y_1 $" src="form_72.png"/> is on whenever <img class="formulaInl" alt="$ x_1 \neq x_2 $" src="form_73.png"/>. The problem is that you cannot draw a line that separates the two classes 0 and 1. They are not linearly separable as you can see in the following picture. Therefore, we need at least one hidden layer to solve the problem. In the next sections you will find C++ code and Python code that solves this problem.</p>
<div class="image">
<img src="xor.png" alt="xor.png"/>
</div>
<h1><a class="anchor" id="Cpp"></a>
C++</h1>
<div class="fragment"><div class="line"><span class="preprocessor">#include <<a class="code" href="OpenANN.html">OpenANN/OpenANN</a>></span></div>
<div class="line"><span class="preprocessor">#include <<a class="code" href="Logger_8h.html">OpenANN/io/Logger.h</a>></span></div>
<div class="line"><span class="preprocessor">#include <<a class="code" href="DirectStorageDataSet_8h.html">OpenANN/io/DirectStorageDataSet.h</a>></span></div>
<div class="line"><span class="preprocessor">#include <<a class="code" href="Random_8h.html" title="Random number generator. ">OpenANN/util/Random.h</a>></span></div>
<div class="line"><span class="preprocessor">#include <Eigen/Core></span></div>
<div class="line"><span class="preprocessor">#include <iostream></span></div>
<div class="line"></div>
<div class="line"><span class="keyword">using namespace </span>OpenANN;</div>
<div class="line"></div>
<div class="line"><span class="keywordtype">int</span> <a class="code" href="dpb_8cpp.html#a3c04138a5bfe5d72780bb7e82a18e627">main</a>()</div>
<div class="line">{</div>
<div class="line"> <span class="comment">// Create dataset</span></div>
<div class="line"> <span class="keyword">const</span> <span class="keywordtype">int</span> D = 2; <span class="comment">// number of inputs</span></div>
<div class="line"> <span class="keyword">const</span> <span class="keywordtype">int</span> F = 1; <span class="comment">// number of outputs</span></div>
<div class="line"> <span class="keyword">const</span> <span class="keywordtype">int</span> N = 4; <span class="comment">// size of training set</span></div>
<div class="line"> Eigen::MatrixXd <a class="code" href="namespacesine.html#a480b21bfa9a8de67d85f8b3c5c471d3c">X</a>(N, D); <span class="comment">// inputs</span></div>
<div class="line"> Eigen::MatrixXd <a class="code" href="namespacesine.html#a5eec15d136977d47b133c067fb2d761a">T</a>(N, F); <span class="comment">// desired outputs (targets)</span></div>
<div class="line"> <span class="comment">// Each row represents an instance</span></div>
<div class="line"> X.row(0) << 0.0, 1.0;</div>
<div class="line"> T.row(0) << 1.0;</div>
<div class="line"> X.row(1) << 0.0, 0.0;</div>
<div class="line"> T.row(1) << 0.0;</div>
<div class="line"> X.row(2) << 1.0, 1.0;</div>
<div class="line"> T.row(2) << 0.0;</div>
<div class="line"> X.row(3) << 1.0, 0.0;</div>
<div class="line"> T.row(3) << 1.0;</div>
<div class="line"> <a class="code" href="classOpenANN_1_1DirectStorageDataSet.html" title="Stores the inputs and outputs of the data set directly in two matrices. ">DirectStorageDataSet</a> dataSet(&X, &T);</div>
<div class="line"></div>
<div class="line"> <span class="comment">// Make the result repeatable</span></div>
<div class="line"> <a class="code" href="classOpenANN_1_1RandomNumberGenerator.html" title="A utility class that simplifies the generation of random numbers. ">RandomNumberGenerator</a>().<a class="code" href="classOpenANN_1_1RandomNumberGenerator.html#a90f34332df2ac3ce80eadf1e66ad5711" title="Set the seed to ensure repeatability. ">seed</a>(0);</div>
<div class="line"></div>
<div class="line"> <span class="comment">// Create network</span></div>
<div class="line"> <a class="code" href="classOpenANN_1_1Net.html" title="Feedforward multilayer neural network. ">Net</a> <a class="code" href="agent_8cpp.html#a474ff98ad9b3a9eb55cfd3c3d4ec5e38">net</a>;</div>
<div class="line"> <span class="comment">// Add an input layer with D inputs, 1 hidden layer with 2 nodes and an</span></div>
<div class="line"> <span class="comment">// output layer with F outputs. Use logistic activation function in hidden</span></div>
<div class="line"> <span class="comment">// layer and output layer.</span></div>
<div class="line"> <a class="code" href="namespaceOpenANN.html#ad62475f272f705ba5a6072cc85b55b5e" title="Create a multilayer neural network. ">makeMLNN</a>(net, <a class="code" href="namespaceOpenANN.html#a90e479eb54bb177043ce063427022968ab78593ece2753677cebf6e7626324df0" title="Logistic sigmoid activation function. ">LOGISTIC</a>, <a class="code" href="namespaceOpenANN.html#a90e479eb54bb177043ce063427022968ab78593ece2753677cebf6e7626324df0" title="Logistic sigmoid activation function. ">LOGISTIC</a>, D, F, 1, 2);</div>
<div class="line"> <span class="comment">// Add training set</span></div>
<div class="line"> net.<a class="code" href="classOpenANN_1_1Learner.html#a24c588e03b847fc0479a681c77127cb4" title="Set training set. ">trainingSet</a>(dataSet);</div>
<div class="line"></div>
<div class="line"> <span class="comment">// Set stopping conditions</span></div>
<div class="line"> <a class="code" href="classOpenANN_1_1StoppingCriteria.html" title="Stopping criteria for optimization algorithms. ">StoppingCriteria</a> <a class="code" href="namespacesine.html#a5d8451dc96d4f2326024d9b8c7bad74f">stop</a>;</div>
<div class="line"> stop.<a class="code" href="classOpenANN_1_1StoppingCriteria.html#adb37c8f3a5ff2657f7c876c36d75ff9b" title="Minimal objective function difference between iterations. ">minimalValueDifferences</a> = 1e-10;</div>
<div class="line"> <span class="comment">// Train network, i.e. minimize sum of squared errors (SSE) with</span></div>
<div class="line"> <span class="comment">// Levenberg-Marquardt optimization algorithm until the stopping criteria</span></div>
<div class="line"> <span class="comment">// are satisfied.</span></div>
<div class="line"> <a class="code" href="namespaceOpenANN.html#a5fcdd987e4be42f5de6109e26c9691a5" title="Train a feedforward neural network supervised. ">train</a>(net, <span class="stringliteral">"LMA"</span>, <a class="code" href="namespaceOpenANN.html#a051970efc9e88e608fbe5cf52db8e87da725f9bfc7b6ef4358c13159ed28bbf3b" title="Mean squared error (regression, two classes) ">MSE</a>, stop);</div>
<div class="line"></div>
<div class="line"> <span class="comment">// Use network to predict labels of the training data</span></div>
<div class="line"> <span class="keywordflow">for</span>(<span class="keywordtype">int</span> n = 0; n < <a class="code" href="namespacexor.html#ae621cb60e5c26909dc63d15b8e2581f2">N</a>; n++)</div>
<div class="line"> {</div>
<div class="line"> Eigen::VectorXd <a class="code" href="namespacexor.html#ac10c3225dba7d99e545014d575bd3af4">y</a> = <a class="code" href="agent_8cpp.html#a474ff98ad9b3a9eb55cfd3c3d4ec5e38">net</a>(dataSet.getInstance(n));</div>
<div class="line"> std::cout << y << std::endl;</div>
<div class="line"> }</div>
<div class="line"></div>
<div class="line"> <span class="keywordflow">return</span> 0;</div>
<div class="line">}</div>
</div><!-- fragment --><p>Compile it with pkg-config and g++ (and really make sure that pkg-config is installed otherwise you might got misleading errors):</p>
<pre class="fragment">g++ main.cpp -o openann `pkg-config --cflags --libs openann`
</pre><h1><a class="anchor" id="Python"></a>
Python</h1>
<div class="fragment"><div class="line"><a name="l00001"></a><span class="lineno"> 1</span> from openann <span class="keyword">import</span> *</div>
<div class="line"><a name="l00002"></a><span class="lineno"> 2</span> <span class="keyword">import</span> numpy</div>
<div class="line"><a name="l00003"></a><span class="lineno"> 3</span> </div>
<div class="line"><a name="l00004"></a><span class="lineno"> 4</span> <span class="keywordflow">if</span> __name__ == <span class="stringliteral">"__main__"</span>:</div>
<div class="line"><a name="l00005"></a><span class="lineno"> 5</span>  <span class="comment"># Create dataset</span></div>
<div class="line"><a name="l00006"></a><span class="lineno"> 6</span>  X = numpy.array([[0, 1], [0, 0], [1, 1], [1, 0]])</div>
<div class="line"><a name="l00007"></a><span class="lineno"> 7</span>  Y = numpy.array([[1], [0], [0], [1]])</div>
<div class="line"><a name="l00008"></a><span class="lineno"> 8</span>  D = X.shape[1]</div>
<div class="line"><a name="l00009"></a><span class="lineno"> 9</span>  F = Y.shape[1]</div>
<div class="line"><a name="l00010"></a><span class="lineno"> 10</span>  N = X.shape[0]</div>
<div class="line"><a name="l00011"></a><span class="lineno"> 11</span>  dataset = DataSet(X, Y)</div>
<div class="line"><a name="l00012"></a><span class="lineno"> 12</span> </div>
<div class="line"><a name="l00013"></a><span class="lineno"> 13</span>  <span class="comment"># Make the result repeatable</span></div>
<div class="line"><a name="l00014"></a><span class="lineno"> 14</span>  RandomNumberGenerator().<a class="code" href="namespaceutil.html#a1ed149f5d214c2c3cf91d2ef9bedfb8e">seed</a>(0)</div>
<div class="line"><a name="l00015"></a><span class="lineno"> 15</span> </div>
<div class="line"><a name="l00016"></a><span class="lineno"> 16</span>  <span class="comment"># Create network</span></div>
<div class="line"><a name="l00017"></a><span class="lineno"> 17</span>  net = Net()</div>
<div class="line"><a name="l00018"></a><span class="lineno"> 18</span>  net.input_layer(D)</div>
<div class="line"><a name="l00019"></a><span class="lineno"> 19</span>  net.fully_connected_layer(3, Activation.LOGISTIC)</div>
<div class="line"><a name="l00020"></a><span class="lineno"> 20</span>  net.output_layer(F, Activation.LOGISTIC)</div>
<div class="line"><a name="l00021"></a><span class="lineno"> 21</span> </div>
<div class="line"><a name="l00022"></a><span class="lineno"> 22</span>  <span class="comment"># Train network</span></div>
<div class="line"><a name="l00023"></a><span class="lineno"> 23</span>  stop_dict = {<span class="stringliteral">"minimal_value_differences"</span> : 1e-10}</div>
<div class="line"><a name="l00024"></a><span class="lineno"> 24</span>  lma = LMA(stop_dict)</div>
<div class="line"><a name="l00025"></a><span class="lineno"> 25</span>  lma.optimize(net, dataset)</div>
<div class="line"><a name="l00026"></a><span class="lineno"> 26</span> </div>
<div class="line"><a name="l00027"></a><span class="lineno"> 27</span>  <span class="comment"># Use network</span></div>
<div class="line"><a name="l00028"></a><span class="lineno"> 28</span>  <span class="keywordflow">for</span> n <span class="keywordflow">in</span> range(N):</div>
<div class="line"><a name="l00029"></a><span class="lineno"> 29</span>  y = net.predict(X[n])</div>
<div class="line"><a name="l00030"></a><span class="lineno"> 30</span>  print(y)</div>
</div><!-- fragment --><h1><a class="anchor" id="MoreExamples"></a>
More Examples</h1>
<p>Classification</p>
<ul>
<li><a class="el" href="TwoSpirals.html">Two Spirals</a></li>
</ul>
<p>Regression</p>
<ul>
<li><a class="el" href="Sine.html">Sine</a></li>
</ul>
<p>Reinforcement Learning</p>
<ul>
<li><a class="el" href="PB.html">Pole Balancing</a></li>
</ul>
<p>We also have some <a class="el" href="Benchmarks.html">Benchmarks</a> that show how you can use ANNs and compare different architectures. </p>
</div></div><!-- contents -->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated on Wed Jul 9 2014 08:57:52 for OpenANN by  <a href="http://www.doxygen.org/index.html">
<img class="footer" src="doxygen.png" alt="doxygen"/>
</a> 1.8.4
</small></address>
</body>
</html>