[libfann] 96/242: Add fann_init_weights
Christian Kastner
chrisk-guest at moszumanska.debian.org
Sat Oct 4 21:10:24 UTC 2014
This is an automated email from the git hooks/post-receive script.
chrisk-guest pushed a commit to tag Version2_0_0
in repository libfann.
commit 2825d86fe5923ec332f0c59d98465c589985e0b4
Author: Evan Nemerson <evan at coeus-group.com>
Date: Fri Mar 5 23:43:42 2004 +0000
Add fann_init_weights
---
ChangeLog | 2 +-
doc/fann.xml | 82 +++++++++++++++++++++++++++++++++++++++++++++++++-----
src/fann.c | 60 +++++++++++++++++++++++++++++++++++++++
src/include/fann.h | 4 +++
4 files changed, 140 insertions(+), 8 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 4f125d3..f91d07e 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -9,7 +9,7 @@ libfann (1.1.0) stable; urgency=low
* Rearanged some code for easy access by the newly created fann php module
* Defined more activation functions and documented the old (in fann_activation.h)
* Customize error output
- * Added fann_set_error_log, fann_shuffle_train_data, fann_merge_train_data, and fann_duplicate_train_data
+ * Added fann_set_error_log, fann_shuffle_train_data, fann_merge_train_data, and fann_duplicate_train_data, fann_init_weights
* Added python bindings
* Created DocBook XML documentation, including API reference
diff --git a/doc/fann.xml b/doc/fann.xml
index 35a66ac..ff35106 100644
--- a/doc/fann.xml
+++ b/doc/fann.xml
@@ -275,8 +275,8 @@ int main()
<para>
The mean square error value is calculated while the ANN is being trained. Some functions are implemented, to use and manipulate this error value. The
- <link linkend="api.fann_get_error"><function>fann_get_error</function></link> function returns the error value and the
- <link linkend="api.fann_reset_error"><function>fann_reset_error</function></link> resets the error value. The following explains how the mean square error
+ <link linkend="api.fann_get_MSE"><function>fann_get_MSE</function></link> function returns the error value and the
+ <link linkend="api.fann_reset_MSE"><function>fann_reset_MSE</function></link> resets the error value. The following explains how the mean square error
value is calculated, to give an idea of the value's ability to reveal the quality of the training.
</para>
<para>
@@ -306,11 +306,11 @@ int main()
<![CDATA[
struct fann_train_data *data = fann_read_train_from_file(filename);
for(i = 1 ; i <= max_epochs ; i++) {
- fann_reset_error(ann);
+ fann_reset_MSE(ann);
for (j = 0 ; j != data->num_data ; j++) {
fann_train(ann, data->input[j], data->output[j]);
}
- if ( fann_get_error(ann) < desired_error ) {
+ if ( fann_get_MSE(ann) < desired_error ) {
break;
}
}
@@ -330,11 +330,11 @@ fann_destroy_train(data);
<programlisting>
<![CDATA[
struct fann_train_data *data = fann_read_train_from_file(filename);
-fann_reset_error(ann);
+fann_reset_MSE(ann);
for(i = 0 ; i != data->num_data ; i++ ) {
fann_test(ann, data->input[i], data->output[i]);
}
-printf("Mean Square Error: %f\n", fann_get_error(ann));
+printf("Mean Square Error: %f\n", fann_get_MSE(ann));
fann_destroy_train(data);
]]>
</programlisting>
@@ -821,6 +821,55 @@ fann_destroy(ann2);
<para>This function appears in FANN >= 1.0.0.</para>
</refsect1>
</refentry>
+ <refentry id="api.fann_init_weights">
+ <refnamediv>
+ <refname>fann_init_weights</refname>
+ <refpurpose>Initialize the weight of each connection.</refpurpose>
+ </refnamediv>
+ <refsect1>
+ <title>Description</title>
+ <methodsynopsis>
+ <type>struct fann *</type>
+ <methodname>ann</methodname>
+ <methodparam>
+ <type>fann_train_data</type>
+ <parameter>data</parameter>
+ </methodparam>
+ </methodsynopsis>
+ <para>
+ This function behaves similarly to <link linkend="api.fann_randomize_weights"><function>fann_randomize_weights</function></link>.
+ It will use the algorithm developed by Derrick Nguyen and Bernard Widrow
+ [<link linkend="bib.nguyen_1990" endterm="bib.nguyen_1990.abbrev" />] to set the weights in such a way as to speed up training.
+ </para>
+ <para>
+ The algorithm requires access to the range of the input data (ie, largest and smallest input), and therefore accepts a second
+ argument, <parameter>data</parameter>, which is the training data that will be used to train the network.
+ </para>
+ <para>
+ In tests using the XOR problem, use of this function decreased the number of iterations required to train the network by approximately
+ 25%, but increased the chances of failure to approximately 2.6%. It is therefore recommended to check the mean square error of the
+ network after training, to make sure it is below the desired error.
+ </para>
+ <example id="example.fann_randomize_weights">
+ <title id="example.fann_randomize_weights.title">Checking the network's mean square error to make sure training was successful.</title>
+ <programlisting>
+<![CDATA[
+while (1) {
+ fann_init_weights(ann, data);
+ fann_train_on_data(ann, data, max_iterations, iterations_between_reports, desired_error);
+ if ( fann_get_MSE(ann) <= desired_error )
+ break;
+}
+]]>
+ </programlisting>
+ </example>
+ <para>
+ In a test of using the XOR problem, this solution still achieved an approximately 15% speed increase. better results should be expected
+ for more complex problems- the authors of the algorithm claim to have decreased training time of one problem from 2 days to 4 hours.
+ </para>
+ <para>This function appears in FANN >= 1.1.0.</para>
+ </refsect1>
+ </refentry>
</section>
<section id="api.sec.io">
<title id="api.sec.io.title">Input/Output</title>
@@ -1028,7 +1077,7 @@ fann_destroy(ann2);
</para>
<para>
This function appears in FANN >= 1.1.0. (before this
- <link linkend="api.fann_get_error"><function>fann_reset_error</function></link> is used)
+ <link linkend="api.fann_reset_error"><function>fann_reset_error</function></link> is used)
</para>
</refsect1>
</refentry>
@@ -3896,6 +3945,25 @@ else
http://www.hamster.dk/~purple/robot/iBOT/report.pdf</ulink>
</releaseinfo>
</biblioentry>
+ <biblioentry id="bib.nguyen_1990">
+ <abbrev id="bib.nguyen_1990.abbrev">Nguyen and Widrow, 1990</abbrev>
+ <title id="bib.nguyen_1990.title">Reinforcement Learning</title>
+ <author>
+ <firstname>Derrick</firstname>
+ <surname>Nguyen</surname>
+ </author>
+ <author>
+ <firstname>Bernard</firstname>
+ <surname>Widrow</surname>
+ </author>
+ <pubdate>1990</pubdate>
+ <publishername>Proc. IJCNN</publishername>
+ <volumenum>3</volumenum>
+ <pagenums>21-26</pagenums>
+ <releaseinfo>
+ <ulink url="http://www.cs.montana.edu/~clemens/nguyen-widrow.pdf">http://www.cs.montana.edu/~clemens/nguyen-widrow.pdf</ulink>
+ </releaseinfo>
+ </biblioentry>
<biblioentry id="bib.OSDN_2003">
<abbrev id="bib.OSDN_2003.abbrev">OSDN, 2003</abbrev>
<pubdate>2003</pubdate>
diff --git a/src/fann.c b/src/fann.c
index f1d5306..c260b91 100644
--- a/src/fann.c
+++ b/src/fann.c
@@ -676,5 +676,65 @@ void fann_seed_rand()
srand(foo);
}
+/* Initialize the weights using Widrow + Nguyen's algorithm.
+*/
+void fann_init_weights(struct fann *ann, struct fann_train_data * train_data)
+{
+ fann_type smallest_inp, largest_inp;
+ unsigned int dat = 0, elem, num_neurons_in, num_neurons_out, num_connect;
+ struct fann_layer *layer_it;
+ struct fann_neuron *neuron_it, *last_neuron, *bias_neuron;
+#ifdef FIXEDFANN
+ unsigned int multiplier = ann->multiplier;
+#endif
+
+ for ( smallest_inp = largest_inp = train_data->input[0][0] ; dat < train_data->num_data ; dat++ ) {
+ for ( elem = 0 ; elem < train_data->num_input ; elem++ ) {
+ if ( train_data->input[dat][elem] < smallest_inp )
+ smallest_inp = train_data->input[dat][elem];
+ if ( train_data->input[dat][elem] > largest_inp )
+ largest_inp = train_data->input[dat][elem];
+ }
+ }
+
+ float scale_factor = powf((float)(0.7f * (float)(ann->total_neurons - (ann->num_input + ann->num_output))),
+ (float)(1.0f / (float)ann->num_input)) / (float)(largest_inp - smallest_inp);
+
+#ifdef DEBUG
+ printf("Initializing weights with scale factor %f\n", scale_factor);
+#endif
+ for ( layer_it = ann->first_layer+1; layer_it != ann->last_layer ; layer_it++) {
+#ifdef DEBUG
+ printf(" Layer: %x/%x (%d neurons)\n", layer_it, ann->last_layer, layer_it->last_neuron - layer_it->first_neuron);
+#endif
+ num_neurons_out = layer_it->last_neuron - layer_it->first_neuron - 1;
+ num_neurons_in = (layer_it-1)->last_neuron - (layer_it-1)->first_neuron - 1;
+ last_neuron = layer_it->last_neuron-1;
+ bias_neuron = (layer_it-1)->last_neuron-1;
+ for(neuron_it = layer_it->first_neuron ; neuron_it != last_neuron; neuron_it++) {
+#ifdef DEBUG
+ printf(" Neuron %x/%x (%d connections)\n", neuron_it, last_neuron, neuron_it->num_connections);
+#endif
+ for ( num_connect = 0 ; num_connect < neuron_it->num_connections ; num_connect++ ) {
+#ifdef DEBUG
+ printf(" Connection %d/%d (%x)\n", num_connect, neuron_it->num_connections, neuron_it->connected_neurons[num_connect]);
+#endif
+ if ( bias_neuron == neuron_it->connected_neurons[num_connect] ) {
+#ifdef FIXEDFANN
+ neuron_it->weights[num_connect] = (fann_type)fann_rand(-scale_factor, scale_factor * multiplier);
+#else
+ neuron_it->weights[num_connect] = (fann_type)fann_rand(-scale_factor, scale_factor);
+#endif
+ } else {
+#ifdef FIXEDFANN
+ neuron_it->weights[num_connect] = (fann_type)fann_rand(0, scale_factor * multiplier);
+#else
+ neuron_it->weights[num_connect] = (fann_type)fann_rand(0, scale_factor);
+#endif
+ }
+ }
+ }
+ }
+}
diff --git a/src/include/fann.h b/src/include/fann.h
index 10c8f22..e524a44 100644
--- a/src/include/fann.h
+++ b/src/include/fann.h
@@ -90,6 +90,10 @@ void fann_destroy(struct fann *ann);
*/
void fann_randomize_weights(struct fann *ann, fann_type min_weight, fann_type max_weight);
+/* Initialize the weights using Widrow + Nguyen's algorithm.
+*/
+void fann_init_weights(struct fann *ann, struct fann_train_data * train_data);
+
/* ----- Implemented in fann_io.c Saving and loading of ANNs ----- */
--
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/debian-science/packages/libfann.git
More information about the debian-science-commits
mailing list