Associative Neural Network Library
Video Recognition Systems |
Face Recognition in Video
Pseudo-inverse & other high performance
associative neural networks
1. What's in name? What it is ?
These neural networks are referred to as
pseudo-inverse networks - for using Moore-Penrose pseudo-inverse V+ in computing
- projection networks - for synaptic (weight) matrix C=VV+ being
Hopfield-like networks - for being binary and fully-connected in the stage of
recurrent networks - for evolving in time, based on external input and internal
attractor-based networks - for storing patterns as attractors (i.e. stable states of
dynamic systems - for allowing the dynamic systems theory to be applied
associative memory - for being able to memorize and recall and patterns
just as much as humans do
PINN memory - pseudo-inverse neural network - based associative
Eight reasons to like pseudo-inverse neural networks (PINN)
It evolved from the network of formal neurons as defined by Hebb in 1949 and has many
parallels with biological memory mechanisms.
It provides an analytical (close-form) solution, the approximation of which is a well-known delta learning rule.
The performance of the network can be very nicely analytically examined and
It can update the memory on fly in real time, storing patterns with a non-iterative learning rule.
- For a network of size N, it allows one to store (with error-correction!)
for M=70%N prototypes,
using the desaturation mechanism.
- At the same time when dealing with a continuous stream of data, thanks to the desaturation mechanism,
the network will never get saturated and will always
maintain the 20%N capacity, always being able to converge to 20%N main attractors.
All this makes the network very suitable for on-line memorization and
recognition, and also for
preprocessing binary patterns, as noise removing filter.
Starting from a fully connected configuration, it can be converted to a Small-World
configuration of the much smaller size.
2. Open Source codes and library:
>> Basic code for associativelt storing and retrieving patterns with fully-connected PINN memory:
This is a one-file code, which is entirely self-contained and ready to be compiled and run.
What it does:
- It defines the CPINN class, which is used to create an associative memory
- It takes a binary pattern, one at a time, and adds it to the memory - very fast
It computes the level of remaining memory capacity, which is used to evaluate the
network's recognition capability and to allow one to reconfigure the network, if
required, to ensure the best performance (analytically!)
It recognizes stored patterns: when presented a new pattern, the memory finds
the pattern the closest to it in associative sense and tells whether the retrieved pattern has been
previously seen or not.
>>Associative Neural Networks Library:
This is a C++ implementation of most known high-performance associative neural networks models, including those based on sparse
architectures and pseudo-inverse learning. For more details see:
High Performance Associative Neural Networks:
Overview and Library.
Alexey Dekhtyarenko and Dmitry Gorodnichy,
Submitted to the Canadian Conference on Artificial Intelligence (AI'06), Quebec
City, Canada, July 7-9, 2006
>> Applying PINN to Face Recognition in Video : FRiV.cpp
This is a C++ example that shows how to apply associative neural network to Face Recognition in Video, as presented in the publications below.
This code is provided for free for educational purposes only. If you use, please reference one of the these publications.
November 2008. - "Sorry, the FRiV is no longer available to public from this website. However, it is still available for educational purposes. Please contact Dmitry Gorodnichy, for more information."
- Dmitry O. Gorodnichy Video-based framework for face recognition in video.
Second Workshop on Face Processing in Video (FPiV'05) in Proceedings of Second Canadian Conference on Computer and Robot Vision (CRV'05), pp. 330-338, Victoria, BC, Canada, 9-11 May, 2005. ISBN 0-7695-2319-6. NRC 48216.
- Dmitry O. Gorodnichy, Associative neural networks as means for low-resolution video-based recognition.
International Joint Conference on Neural Networks (IJCNN'05), Montreal, Quebec, Canada, July 31-August 4, 2005. NRC 48217.
3. Other references
- D.O. Gorodnichy. "Designing High Performance Attractor-based Neural
Candidate of Science (Ph.D.) Dissertation in Phys. &
Math. In speciality 01.05.03 - Mathematical and software development for
computer systems. Institute of Mathematical Machines and Systems, Glushkov
Cybernetics Center of the National Academy of Sciences of Ukraine, Kiev, 10
September 1997. Written in Russian.
наук , НАН
Dissertation - pdf, Official summary