Pattern Recognition Using Neural and Functional Networks (Studies in Computational Intelligence)

Vasantha Kalyani David and Sundaramoorthy Rajasekaran Pattern Recognition Using Neural and Functional Networks Studies...

Author: Vasantha Kalyani David | S. Rajasekaran

85 downloads 641 Views 4MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!

Report copyright / DMCA form

DOWNLOAD PDF

Vasantha Kalyani David and Sundaramoorthy Rajasekaran Pattern Recognition Using Neural and Functional Networks

Studies in Computational Intelligence, Volume 160 Editor-in-Chief Prof. Janusz Kacprzyk Systems Research Institute Polish Academy of Sciences ul. Newelska 6 01-447 Warsaw Poland E-mail: [email protected] Further volumes of this series can be found on our homepage: springer.com Vol. 138. Bruno Apolloni, Witold Pedrycz, Simone Bassis and Dario Malchiodi The Puzzle of Granular Computing, 2008 ISBN 978-3-540-79863-7 Vol. 139. Jan Drugowitsch Design and Analysis of Learning Classifier Systems, 2008 ISBN 978-3-540-79865-1 Vol. 140. Nadia Magnenat-Thalmann, Lakhmi C. Jain and N. Ichalkaranje (Eds.) New Advances in Virtual Humans, 2008 ISBN 978-3-540-79867-5 Vol. 141. Christa Sommerer, Lakhmi C. Jain and Laurent Mignonneau (Eds.) The Art and Science of Interface and Interaction Design (Vol. 1), 2008 ISBN 978-3-540-79869-9 Vol. 142. George A. Tsihrintzis, Maria Virvou, Robert J. Howlett and Lakhmi C. Jain (Eds.) New Directions in Intelligent Interactive Multimedia, 2008 ISBN 978-3-540-68126-7

Vol. 149. Roger Lee (Ed.) Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, 2008 ISBN 978-3-540-70559-8 Vol. 150. Roger Lee (Ed.) Software Engineering Research, Management and Applications, 2008 ISBN 978-3-540-70774-5 Vol. 151. Tomasz G. Smolinski, Mariofanna G. Milanova and Aboul-Ella Hassanien (Eds.) Computational Intelligence in Biomedicine and Bioinformatics, 2008 ISBN 978-3-540-70776-9 Vol. 152. Jaroslaw Stepaniuk Rough – Granular Computing in Knowledge Discovery and Data Mining, 2008 ISBN 978-3-540-70800-1 Vol. 153. Carlos Cotta and Jano van Hemert (Eds.) Recent Advances in Evolutionary Computation for Combinatorial Optimization, 2008 ISBN 978-3-540-70806-3 Vol. 154. Oscar Castillo, Patricia Melin, Janusz Kacprzyk and Witold Pedrycz (Eds.) Soft Computing for Hybrid Intelligent Systems, 2008 ISBN 978-3-540-70811-7

Vol. 143. Uday K. Chakraborty (Ed.) Advances in Differential Evolution, 2008 ISBN 978-3-540-68827-3

Vol. 155. Hamid R. Tizhoosh and M. Ventresca (Eds.) Oppositional Concepts in Computational Intelligence, 2008 ISBN 978-3-540-70826-1

Vol. 144. Andreas Fink and Franz Rothlauf (Eds.) Advances in Computational Intelligence in Transport, Logistics, and Supply Chain Management, 2008 ISBN 978-3-540-69024-5

Vol. 156. Dawn E. Holmes and Lakhmi C. Jain (Eds.) Innovations in Bayesian Networks, 2008 ISBN 978-3-540-85065-6

Vol. 145. Mikhail Ju. Moshkov, Marcin Piliszczuk and Beata Zielosko Partial Covers, Reducts and Decision Rules in Rough Sets, 2008 ISBN 978-3-540-69027-6 Vol. 146. Fatos Xhafa and Ajith Abraham (Eds.) Metaheuristics for Scheduling in Distributed Computing Environments, 2008 ISBN 978-3-540-69260-7 Vol. 147. Oliver Kramer Self-Adaptive Heuristics for Evolutionary Computation, 2008 ISBN 978-3-540-69280-5 Vol. 148. Philipp Limbourg Dependability Modelling under Uncertainty, 2008 ISBN 978-3-540-69286-7

Vol. 157. Ying-ping Chen and Meng-Hiot Lim (Eds.) Linkage in Evolutionary Computation, 2008 ISBN 978-3-540-85067-0 Vol. 158. Marina Gavrilova (Ed.) Generalized Voronoi Diagram: A Geometry-Based Approach to Computational Intelligence, 2009 ISBN 978-3-540-85125-7 Vol. 159. Dimitri Plemenos and Georgios Miaoulis (Eds.) Generalized Voronoi Diagram: A Geometry-Based Approach to Computational Intelligence, 2009 ISBN 978-3-540-85125-7 Vol. 160. Vasantha Kalyani David and Sundaramoorthy Rajasekaran Pattern Recognition Using Neural and Functional Networks, 2009 ISBN 978-3-540-85129-5

Vasantha Kalyani David Sundaramoorthy Rajasekaran

Pattern Recognition Using Neural and Functional Networks

123

Professor Dr. Vasantha Kalyani David Department of Computer Science Avinashilingam University for Women Coimbatore - 641 043, Tamil Nadu India

Professor Dr. Sundaramoorthy Rajasekaran Infrastructure Engineering PSG College of Technology Coimbatore - 641 004, TamilNadu India Email: [email protected]

ISBN 978-3-540-85129-5

e-ISBN 978-3-540-85130-1

DOI 10.1007/978-3-540-85130-1 Studies in Computational Intelligence

ISSN 1860949X

Library of Congress Control Number: 2008931714 c 2009 Springer-Verlag Berlin Heidelberg This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typeset & Cover Design: Scientific Publishing Services Pvt. Ltd., Chennai, India. Printed in acid-free paper 987654321 springer.com

Authors

Vasantha Kalyani David and S. Rajasekaran About the Authors Vasantha Kalyani David, PhD., is a Selection Grade Lecturer in Computer Science in Avinashilingam University for Women, Coimbatore. Earlier a mathematician with a Master of Philosophy in Mathematics and later did research in Computer Science. Dr. Vasantha Kalyani David has published many papers in areas of Soft Computing, Her research interests include Neural Networks, Artiﬁcial Intelligence, Fuzzy Logic, Genetic Algorithms, Cellular Automata, Theoretical Computer Science and Automata Theory. S. Rajasekaran, DSc, (Civil Engineering) is Professor of Infrastructure Engineering., PSG College of Technology, Coimbatore. He has over 41 years of teaching and research experience and more than 300 research papers to his credit. His areas of special interest include Structural Engineering, Finite Element Analysis, and application of Soft Computing techniques in Structural Engineering.

Preface

Biologically inspired computing is diﬀerent from conventional computing. It has a diﬀerent feel; often the terminology does not sound like it’s talking about machines. The activities of this computing sound more human than mechanistic as people speak of machines that behave, react, self-organize, learn, generalize, remember and even to forget. Much of this technology tries to mimic nature’s approach in order to mimic some of nature’s capabilities. They have a rigorous, mathematical basis and neural networks for example have a statistically valid set on which the network is trained. Two outlines are suggested as the possible tracks for pattern recognition. They are neural networks and functional networks. Neural Networks (many interconnected elements operating in parallel) carry out tasks that are not only beyond the scope of conventional processing but also cannot be understood in the same terms. Imaging applications for neural networks seem to be a natural ﬁt. Neural networks love to do pattern recognition. A new approach to pattern recognition using microARTMAP together with wavelet transforms in the context of hand written characters, gestures and signatures have been dealt. The Kohonen Network, Back Propagation Networks and Competitive Hopﬁeld Neural Network have been considered for various applications. Functional networks, being a generalized form of Neural Networks where functions are learned rather than weights is compared with Multiple Regression Analysis for some applications and the results are seen to be coincident. New kinds of intelligence can be added to machines, and we will have the possibility of learning more about learning. Thus our imaginations and options are being stretched. These new machines will be fault-tolerant, intelligent and self-programming thus trying to make the machines smarter. So as to make those who use the techniques even smarter. Chapter 1 is a brief introduction to Neural and Functional networks in the context of Pattern recognition using these disciplines Chapter 2 gives a review of the architectures relevant to the investigation and the development of these technologies in the past few decades.

VIII

Preface

Chapter 3 begins with the look at the recognition of handwritten alphabets using the algorithm for ordered list of boundary pixels as well as the Kohonen Self-Organizing Map (SOM). Chapter 4 describes the architecture of the MicroARTMAP and its capability. Chapter 5 the MicroARTMAP is augmented with a moment based feature extractor and applied to character recognition in this chapter. Chapter 6 illustrates the use of wavelet transforms together with MicroARTMAP for character recognition. The microARTMAP gave solutions to problems in civil engineering like classiﬁcation of soil problem, ﬁnding the load from yield pattern of a plate and ﬁnding earthquake parameters from a given response spectrum. Chapter 7 MicroARTMAP and Back Propagation Network have been compared and presented for gesture recognition and signature veriﬁcation. Solving scheduling problem by means of a Competitive Hopﬁeld Neural Network are discussed in Chapter 8 Multiple Regression methods considered as a recognizer is compared with functional networks in solving certain problems as shown in Chapter 9. Conclusion and further applications are suggested in Chapter 10.

Contents

1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Recognition through Algorithm and Kohonen’s Self Organizing Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 MicroARTMAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Wavelet Transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5 Gesture Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.6 Competitive Hopﬁeld Neural Network . . . . . . . . . . . . . . . . . . . . . . . 1.7 Neural and Functional Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.8 Objectives and Scope of the Investigation . . . . . . . . . . . . . . . . . . . . 1.9 Organization of the Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.10 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 1 2 2 3 4 5 5 6 6 7

2

Review of Architectures Relevant to the Investigation . . . . . . 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Recognition through Self Organizing Map . . . . . . . . . . . . . . . . . . . 2.3 The μARTMAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Wavelet Transforms and MicroARTMAP . . . . . . . . . . . . . . . . . . . . 2.5 MicroARTMAP and Gesture Recognition . . . . . . . . . . . . . . . . . . . . 2.6 Competitive Hopﬁeld Neural Network . . . . . . . . . . . . . . . . . . . . . . . 2.7 Functional Networks and Multiple Regression Analysis . . . . . . . . 2.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

9 9 10 10 11 11 11 12 12

3

Recognition of English and Tamil Alphabets Using Kohonen’s Self-organizing Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Recognition of Handwritten Characters Using Ordered List of Image Pixels on Its Boundary . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 The Kohonen Feature Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Normalization of a Vector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 Training Law . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6 Neighbourhood Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

15 15 15 18 19 19 19

X

Contents

3.7 3.8 3.9 3.10 4

The Kohonen Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Representation of Characters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Weight Vector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

19 19 20 26

Adaptive Resonance Theory Networks . . . . . . . . . . . . . . . . . . . . . . 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 ART Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.1 Resonant State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.2 The STM and LTM Traces . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.3 The Structure of the ART Model . . . . . . . . . . . . . . . . . . . . . 4.2.4 Pattern-Matching Cycle in an ART Network . . . . . . . . . . . 4.2.5 The 2/3 Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.6 Gain Control Mechanism . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Fuzzy ART . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.1 Analogy between ART1 and Fuzzy ART . . . . . . . . . . . . . . 4.3.2 Fast-Learn, Slow-Recode and Complement Coding . . . . . . 4.3.3 Complement Coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.4 Weight Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.5 Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.6 Category Choice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.7 Resonance or Reset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.8 Learning Law . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.9 Normalization of Fuzzy ART Inputs . . . . . . . . . . . . . . . . . . 4.3.10 Geometric Interpretation of Fuzzy ART . . . . . . . . . . . . . . . 4.3.11 Fuzzy ART Category Boxes in Fuzzy Cubes . . . . . . . . . . . 4.3.12 Fuzzy ART Stable Category Learning . . . . . . . . . . . . . . . . . 4.4 Fuzzy ARTMAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.1 Fuzzy ARTMAP and MicroARTMAP . . . . . . . . . . . . . . . . . 4.4.2 MicroARTMAP Algorithm (Supervised Neural Network Architecture) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.3 Map Field Activation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.4 Match Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.5 Map Field Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.6 Deﬁning H . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.7 Training of μARTMAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.8 Inter ART Reset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.9 Oﬄine Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.10 μARTMAP Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.11 Fast Learning in μARTMAP . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.12 Reﬁning a Hyper Box . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.13 μARTMAP Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

27 27 28 28 28 30 31 31 32 33 33 34 34 36 36 36 37 37 38 39 39 42 43 43 44 45 45 46 46 47 47 48 48 48 49 49 49

Contents

5

6

XI

Applications of MicroARTMAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Recognition of Handwritten Alphabets by μARTMAP . . . . . . . . 5.3 Recognition of Handwritten Words by μARTMAP . . . . . . . . . . . . 5.4 Recognition of Handwritten Alphabets by μARTMAP Augmented with Moment-Based Feature Extractor . . . . . . . . . . . 5.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.2 Steps Involved in Obtaining Moment Invariants . . . . . . . . 5.5 Recognition of Handwritten Numbers by μARTMAP Using Hamming Distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.6 Recognition of Alphabets and Numbers Using μARTMAP with Only One Exemplar for Training . . . . . . . . . . . . . . . . . . . . . . . 5.7 Recognition of Alphabets by μARTMAP with Increased Sample Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.8 BIS Classiﬁcation of Soil . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.9 Plastiﬁcation of Clamped Isotropic Plate . . . . . . . . . . . . . . . . . . . . 5.10 Application to Earthquake Engineering . . . . . . . . . . . . . . . . . . . . . . 5.11 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

51 51 51 53

Wavelet Transforms and MicroARTMAP . . . . . . . . . . . . . . . . . . . 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 The Need for Transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Transforms Available . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5 Wavelet Transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.6 Continuous Wavelet Transforms (CWT) . . . . . . . . . . . . . . . . . . . . . 6.7 Discrete Wavelet Transforms (DWT) . . . . . . . . . . . . . . . . . . . . . . . . 6.8 Wavelet Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.9 Wavelet Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.10 Schematic Representation of the Working of a Wavelet . . . . . . . . 6.11 Handwritten Characters Recognition Using Wavelet Transforms and MicroARTMAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.12 Wavelet Transforms in Two Dimensions . . . . . . . . . . . . . . . . . . . . . 6.13 The Two-Dimensional DWT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.14 Recognition of Handwritten Alphabets Using Wavelet Packets and MicroARTMAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.14.1 The Decomposition Space Tree . . . . . . . . . . . . . . . . . . . . . . . 6.14.2 Analysis Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.14.3 Finding Optimal Decomposition . . . . . . . . . . . . . . . . . . . . . . 6.14.4 Eﬃcient Algorithm for Minimal Entropy Solutions . . . . . . 6.14.5 Denoising Using MATLAB for Handwritten Characters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.14.6 MicroARTMAP and Wavelet Packets . . . . . . . . . . . . . . . . . 6.15 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

73 73 73 73 74 74 75 75 77 77 78

53 53 55 59 60 61 61 64 65 71

82 82 84 85 85 85 86 87 87 88 91

XII

7

8

9

Contents

Gesture and Signature Recognition Using MicroARTMAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Gestures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3 Gesture Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4 Voice Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.5 Hand Writing Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.6 Hand Gestures in HCI (Human – Computer Interaction) . . . . . . 7.7 Gesture Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.7.1 Gesture Acquisition – Requirements . . . . . . . . . . . . . . . . . . 7.7.2 Gesture Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.7.3 Feature Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.7.4 Statistical Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.7.5 Block Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.8 Wavelet Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.8.1 DWT Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.9 Neural Network for Gesture Recognition . . . . . . . . . . . . . . . . . . . . . 7.9.1 Application- Robotics – Robotic Arm Model . . . . . . . . . . . 7.10 Interface Circuit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.11 Back Propagation Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.12 Statistical Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.13 Block Processing: For 16 Features . . . . . . . . . . . . . . . . . . . . . . . . . . 7.14 Wavelet Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.15 MicroARTMAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.16 Signature Recognition Using MicroARTMAP and Block Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.17 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

93 93 93 93 94 95 96 98 98 99 99 99 101 101 103 103 104 106 107 107 107 110 111 112 113

Solving Scheduling Problems with Competitive Hopﬁeld Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 The Energy Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4 Simulation Example Case (i) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5 Example Case (ii) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

115 115 115 117 117 119 122

Functional Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2 Functional Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3 Procedure to Work with Functional Networks . . . . . . . . . . . . . . . . 9.4 The Associativity Functional Network . . . . . . . . . . . . . . . . . . . . . . . 9.5 Multiple Regression Methods and Functional Networks . . . . . . . . 9.6 Rock Identiﬁcation by Functional Networks . . . . . . . . . . . . . . . . . . 9.7 Hot Extrusion of Steel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

123 123 123 124 125 127 128 131 134

Contents

XIII

10 Conclusions and Suggestions for Future Work . . . . . . . . . . . . . . 135 10.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 10.2 Suggestions for Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 A

MicroARTMAP MATLAB Implementation . . . . . . . . . . . . . . . . . 143

B

DWT on db1 Wavelets – number.m . . . . . . . . . . . . . . . . . . . . . . . . . 155

C

Inputs to ARTMAP for Signatures . . . . . . . . . . . . . . . . . . . . . . . . . 159

D

The Competitive Hopﬁeld Neural Network . . . . . . . . . . . . . . . . . 163

E

Moment Invariants for Handwritten Characters . . . . . . . . . . . . 169

F

Pattern.cpp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177

F

handpgm2.m . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

List of Figures

1.1

Architecture for Gestural Control of Memory and Display . . . . . .

4

3.1 3.2 3.3 3.4 3.5 3.6

Recognition of Alphabet P . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Recognition of Alphabet Close to R . . . . . . . . . . . . . . . . . . . . . . . . . . Nearest Recognition of U . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Alphabet A Taught to the System . . . . . . . . . . . . . . . . . . . . . . . . . . . Kohonen Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Semi-log Plot Between Average Distance and Cycle Number . . . .

16 16 17 17 18 25

4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8

ART Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A Pattern-Matching Cycle in an ART Network . . . . . . . . . . . . . . . . The Fuzzy ART . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Fuzzy ART Category Boxes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Fast Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Weight Vector WJ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Fuzzy ARTMAP Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . MicroARTMAP Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

29 30 35 40 40 42 43 44

Sample of Handwritten Characters . . . . . . . . . . . . . . . . . . . . . . . . . . The Character “Z” and Quarters . . . . . . . . . . . . . . . . . . . . . . . . . . . . Soil Classiﬁcation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Isotropic plate (Clamped) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Response spectra for M = 6, S = 0, H = 1, R = 30 . . . . . . . . . . . Response Spectra for M = 8, S = 0, H = 30, R = 50 . . . . . . . . . . . Response spectra for M = 7.5, S = 2, H = 8, R = 45 . . . . . . . . . . Black and white ﬁgure of Fig. 5.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . Figure (Fig. 5.8) divided into eight parts Number of black counts in each part is c1 = 4731, c2 = 4881, c3 = 4977, c4 = 4982, c5 = 4732, c6 = 4882, c7 = 4978, c8 = 4983 . . . . . . . 5.10 Comparison of MICROARTMAP with BPN . . . . . . . . . . . . . . . . . .

52 54 54 63 68 68 69 69

5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 5.9

70 71

XVI

List of Figures

6.1 6.2 6.3 6.4 6.5 6.6 6.7

Decomposition of a Signal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The DWT and CWT of a Signal . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tree Mode Analysis of a Signal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Shifting and Scaling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . DWT of a signal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wavelet transform applied to an image . . . . . . . . . . . . . . . . . . . . . . . Multiresolution scheme after one and two levels of decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.8 The Decomposition Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.9 The Best Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.10 Analysed and Denoised Signal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.11 2-D Wavelet Selection Upto Level 5 . . . . . . . . . . . . . . . . . . . . . . . . . .

75 76 79 80 81 84 84 86 87 88 89

7.1 7.2 7.3 7.4 7.5 7.6 7.7 7.8 7.9 7.10 7.11 7.12 7.13 7.14 7.15

Gestural Taxonomy for HCI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Overall Block Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Acquired Gestures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Gesture Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Block Processing - 16 Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Images After Wavelet Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . Neural Net block diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Robotic Arm Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Scope of the Movement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Interface Circuit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pin Conﬁguration of ULN 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . BPN - Statistical Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . BPN - Block Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . BPN - Wavelet Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . BPN - Wavelet Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

96 97 98 99 100 101 103 104 105 106 107 108 109 110 113

8.1 8.2 8.3

3-D Hopﬁeld Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 Simulation results Case (i) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 Simulation results obtained Case (ii) . . . . . . . . . . . . . . . . . . . . . . . . . 120

9.1 9.2 9.3

Functional Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 Associativity Functional network . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 Comparison of actual vs functional networks . . . . . . . . . . . . . . . . . . 133

List of Tables

3.1 3.2 3.3 3.3 3.4 5.1 5.2 5.3 5.4 5.5 5.6 5.7 6.1

Recognition run for English Alphabets (Two inputs given at the same time) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Recognition run for both English and Tamil Alphabets (Two inputs given together) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Recognition run for Tamil Alphabets (Two inputs given at the same time) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . (continued) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Comparison of the Recognition of Alphabets . . . . . . . . . . . . . . . . . . Moment-Based Invariants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Moment Invariants for Alphabets given to μARTMAP . . . . . . . . . Recognition of numerals with varying Hamming Distance . . . . . . . IS Classiﬁcation of Soil . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Moment Invariants for Yield Pattern for Clamped Isotropic Plate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M, S, H and R for Diﬀerent Response Spectra . . . . . . . . . . . . . . . . . Black Pixel Count in the Split Response Spectra . . . . . . . . . . . . . .

21 22 23 24 25 56 57 60 62 64 66 67

Statistical Measures for the Image given as Input to μARTMAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

90

7.1 7.2 7.3 7.4 7.5 7.6

Data Set - Block Processing With 16 Features . . . . . . . . . . . . . . . . BPN-Statistical Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . BPN-Block Processing with 16 features . . . . . . . . . . . . . . . . . . . . . . BPN-Wavelet Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . MicroARTMAP - Block Processing with 16 features . . . . . . . . . . . MicroARTMAP - Wavelet Approach . . . . . . . . . . . . . . . . . . . . . . . . .

102 108 109 110 111 111

8.1a 8.1b 8.1c 8.1d 8.2

Resource requested Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Timing Constraints Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Initial States for CHNN Case (i) . . . . . . . . . . . . . . . . . . . . . . . . . . . . Weighting Factor of CHNN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Initial states for CHNN Case (ii) . . . . . . . . . . . . . . . . . . . . . . . . . . . .

118 118 119 119 121

XVIII

9.1 9.2 9.3 9.4

List of Tables

Normalized Input and Output for Rock Classiﬁcation Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Undetermined Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Normalized Inputs and Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Undetermined Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

129 130 132 133

List of Notations

ε <, = , > ∀ |.| ∧ ∨ ⇒ ∫ ⊕ ∩ ||

-

belongs to strain either less than or equal to or greater than for all fuzzy count fuzzy intersection fuzzy union implies integral includes operator intersection operator modulus operator

|p|

- norm of p given by

n

τ σ Σ ∪ ∫∫ [0, 1] [0, 1]n a ac A A′ ARTa ARTb BLU C CWT

∑ pi

i =1

-

where p = (p1,p2, ...,pn)

shear strength normal stress summation union vector double integral closed interval between 0 and 1 fuzzy n-cube approximate signal fuzzy complement of a, i.e., a = 1-a vector normalized vector of A fuzzy ART module fuzzy ART module basic length unit Lead screw pitch / number of revolutions Cohesion intercept continuous wavelet transforms

XX

List of Notations

C1, C2, C3, C4, C5, C6

-

refer to weighting factors

+∞

C (a, b) =

∫ f (x) ψ a, b (x) dx

-∞

gives wavelet co-efficients +∞

C AN (k) =

∫ f (x) ψ N, k (x) dx

-∞

gives approximate co-efficients +∞

C DN (k) =

∫ f (x) ψ N , k (x) dx

-∞

d d DWT ej E E E E

gives detailed co-efficients detailed version Euclidean distance discrete wavelet transforms error in the jth data ej = fi(xij) + f2(x2j) – f3(x3j) Young’s modulus Number of times stepper motor has to be energized Energy function Euclidean norm of error function 2

n

E = 1 / 2∑ (O i − F(i )) i =1

E

-

sum of the squares of the error for all data n data

E=

∑ e Tj e j j=1

Ep Ep EA, EH, EV, ED

-

E(f)

-

plastic modulus entropy of parent node entropy of four offsprings. Approximation, horizontal, vertical and diagonal details additive cost function approximate neural function m

fi(x) = Fab F1 → F2 fn g(n) h hj hmax

-

∑ a ijφ ij (X ) j=1

map field mapping from F1 to F2 cyclic frequency high pass filter hamming distance contribution to H of set Aj upper bound on hj

List of Notations

h(n) H Hmax H(Gijk) H(B/A) mpq, p, q = 0, 1, 2, . .. , n M M M M, R, S, H

-

m N Netijk n Pi R R

-

Ris t T Tj Tn Vijk

-

Wnew Wold Wxyzijk Wϕ, WψH, WψV and WψD x(n)

-

XXI

low pass filter conditional entropy upper bound on H heavside function conditional entropy moment transformation mass earthquake magnitude minimum step angle total number of machines Magnitude, Site condition, Focal depth, Epicentral distance from response spectrum strain rate sensitivity index total number of processes total input to the neuron (i, j, k) strain hardening exponent total execution time required by a process required angle augmented function R = E + < a > [φ0] {λ} - < λ > {α} R = < a > [A] {a} + < a > [φ0] {λ} - < λ > {α} process i requires resource s thickness of the plate maximum time quantum of a process choice function natural vibration period state variable representing whether or not a job i is executed on a machine j at a certain time k updated weight vector old weight vector synaptic interconnection strength quarter size output subimages original signal

+∞

x(n)* h(n) =

∑ x(k) * h(n - k)

k = -∞

yhigh (k) ylow (k) α β δ(a,b) φ

convolution operation output of high pass filter outputs of low pass filter choice parameter material thermal constant learning rate parameter Kronecker delta function shape functions (1, x, x2, . . . , xn) or (1, sin(x), cos(x), sin(2x), cos(2x), sin(3x), cos(3x)………) angle of internal friction

XXII

List of Notations

φ1, φ2, φ3, φ4, φ5, φ6, φ7 ηpq ϕ (x, y) ϕj, m, n (x, y) μpq μARTMAP ν θijk ρ σ0 ψ(x) ψ(u) ψa, b (x) ψD (x, y) ψH (x, y) ψV (x, y) ψij, m, n (x, y) ωn

∂( ) ∂( )

2/3 rule

-

invariant functions normalized central moments two dimensional scaling function scaled basis function central moments microARTMAP Poisson ratio bias input vigilance parameters yield stress wavelet function Fourier transform mother wavelet generates basis functions variations along diagonals variations along columns (horizontal) variations along rows (vertical) translated basis function circular frequency

-

partial derivative

-

two out of possible three inputs are active

1 Introduction

1.1 Introduction For the past several years, pattern recognition has been used in a number of areas such as medicine, weather forecasting, automated industrial inspection, geology and agriculture. Pattern recognition is concerned with the automatic detection or classification of objects or events. Most pattern recognition tasks are first done by human beings and automated later. Automating the classification of objects using the features as those used by people can be a very difficult task. The features used by machines are not precisely the same as those used by human beings. Features that would be impossible or difficult for humans to estimate are useful in automated systems. For example some systems classify objects in satellite images using the wavelengths of light that are invisible to human beings. We live in a world where events never repeat exactly. Even though events are never exactly the same, they are also not completely different. There is some sort of continuity, similarity and predictability that allows one to generalize correctly from past experience to future events. Neural networks is based on the idea that one can reproduce the power of the human brain by artificial means. The problems where artificial neural networks are very promising include signal processing, speech recognition, visual perception, control and robotics. The neural networks help to solve these problems with natural mechanisms of generalization. Suppose one represents an object in a network as a pattern of activation of several units. If a unit or two responds incorrectly, the overall pattern anyhow remains the same and the neural network responds correctly to the stimuli. When neural networks operate, similar inputs naturally produce similar outputs. Most real-world perceptual problems have this structure of input-output continuity. Neural networks are model free estimators. Neural networks do classification by similarity and hence are not suitable in situations where we cannot trust similarity. Neural networks recognize patterns we cannot even define. This property is called recognition without definition. This characterizes much of intelligent behaviour and enables systems to generalize. Neural networks store pattern or function information with distributed encoding. They superimpose pattern information on the many synaptic connections between neurons. This enables the neural networks to complete partial patterns and ‘clean V.K. David and S. Rajasekaran: Pattern Recog. Using Neural & Funct. Net., SCI 160, pp. 1–7. c Springer-Verlag Berlin Heidelberg 2009 springerlink.com

2

Introduction

up’ noisy patterns. Distributed encoding produces interference between stored patterns. Similar patterns may clump together and new patterns may crowd out older learned patterns. Older patterns may distort newer patterns. On the whole, the neural network behaves as an adaptive function estimator. Functional Networks on the other hand are extensions of neural networks and are useful in problems where both domain knowledge and data are available. Pattern recognition using the above disciplines have been studied.

1.2 Recognition through Algorithm and Kohonen’s Self Organizing Map Recognition of alphabets through the algorithm for ordered list of boundary pixels was carried out, but the algorithm proved inconsistent in its treatment of corner-connected regions. This inconsistency would not matter in image where corner-neighbouring regions do not appear. The Kohonen’s self organizing map [59], [60], [62] was used to recognize alphabetic characters. By its self-organizing properties and winner-take-all strategy, the Kohonen’s feature map was able to recognize English as well as Tamil alphabets and their garbled versions.

1.3 MicroARTMAP [35] A new neural architecture called μARTMAP or (MicroARTMAP) [35] is considered as a solution to the category proliferation problem which are present in Fuzzy ARTMAP – based architecture capable of performing fast, stable learning in supervised settings. This reduces the number of committed categories, while preserving generalization performance, without changing the geometry of category representation. A compact set of IF – THEN rules are easily extracted. This favours the use of neural networks in problems where comprehensibility of decisions are required, or where it is important to gain insight into the problem through the data. μARTMAP intelligently position hyper boxes in the input space and optimize their size to achieve category reduction. Two different learning stages are considered. In the first stage an inter – ART reset mechanism is fired if selected ARTa category has an entropic prediction. The ARTa vigilance is not raised. The total prediction entropy is evaluated in the second stage, and if need be some patterns are presented again with increased ARTa vigilance values. The μARTMAP allows some training error, avoiding committing categories with small relevance for generalization and also permits placing hyper boxes inside other hyper boxes to describe efficiently populated exceptions. These are problems where many patterns associated to one class label are surrounded by many others associated with a different one. Experimental results obtained show that an inter – ART reset mechanism is necessary for treating correctly the populated exceptions. In the first learning stage in the μARTMAP the vigilance in ARTa is not raised after inter ART reset and so this does not cause category proliferation, while during the second stage of learning, accuracy is guaranteed. The μARTMAP allows a small error on the training set, it finds more compact rule sets when there is an overlap between concept

Wavelet Transforms

3

classes and therefore has no exact solution. Thus this result generalizes μARTMAP as being more robust to noise compared to FUZZY ARTMAP. μARTMAP may be modified to control category growth on each input feature independently. μARTMAP is favoured as compared to other rule pruning or extraction methods. The μARTMAP network has a compact set of rules and also preserves the on-line features. By accepting a small training error, the μARTMAP is able to generate a compact rule set. These rules reflect more reliability on the underlying distribution of the data and hence are more efficient. “Wavelet Theory” is the result of multidisciplinary effort that brought together mathematicians, physicists and engineers . . . . . this connection has created a flow of ideas that goes well beyond the construction of new bases or transforms. - Stephane Mallat.

1.4 Wavelet Transforms The Fourier transforms has been the mainstay of transform – based image processing since 1950s. The word wavelet is due to Morlet and Grossman in the early 1980’s. Wavelet transform makes it easier to compress, transmit and analyze many images [25], [26]. The basis function of Fourier transform are sinusoids, whereas wavelet transforms are based on small waves called wavelets of varying frequency and limited duration. Temporal information is not lost in the wavelet transformation process. In 1987 wavelets were first shown to be the foundation of a powerful new approach to signal processing and analysis called multiresolution theory. Multiresolution theory incorporates and unifies techniques from a variety of disciplines, including subband coding from signal processing, quadratic mirror filtering from digital speech recognition and pyramidal image processing. Multiresolution theory is concerned with the representation and analysis of signals (or images) at more than one resolution, thus enabling one to note those features that might go undetected at one resolution but may be easy to spot at another one. The simplest of wavelets is the Haar wavelet. In discrete form Haar wavelets are related to a mathematical operation called the Haar transform. They are used to compress audio signals and for removing noise. A large collection of wavelet transforms were discovered by Daubechies [30]. The Daubechies Wavelet transforms are defined in the same way as the Haar Wavelet transform – by computing running averages and differences via scalar products with scaling signals and wavelets. For the Daubechies wavelet transforms, the scaling signals and wavelets produce averages and differences using just a few more values from the signal as when compared to the Haar wavelet transform. The simplest of Daubechies transforms is Daub4 wavelet transform. This is defined the same way as the Haar wavelet transform. The DaubJ transforms for J = 6, 8, . . . . . . 20 and Coif I transforms for I = 6, 12, 18, 24, 30 are all quite similar to the Daub4 transform. CoifI wavelets are designed for maintaining a close match between the trend values and original signal values. There are many other wavelet transforms such as Spline wavelet transforms and various types

4

Introduction

of Biorthogonal wavelet transforms. The DaubJ transforms for J = 6, 8, . . . . . 20 are just treated as a simple generalization of Daub4 transform. The most obvious difference between them is the length of the supports of their scaling signals and wavelets. Wavelets provide a powerful and remarkably flexible set of tools for handling fundamental problems in science and engineering. A wide range of problems that are being solved by wavelets include audio denoising, signal compression, object detection, fingerprint compression, Image denoising, Image enhancement, Image recognition, diagnosing heart trouble and speech recognition. Wavelet Transforms have been used to recognize hand written alphabets.

1.5 Gesture Recognition Gesture recognition [66] is an interface with computers using gestures of the human body. A camera reads the movements of the human body and communicates the data to a computer that uses the gestures as input to control devices or applications. This will help the physically impaired to interact with computers using sign language. The technology will eliminate input devices and allow gestures as inputs. Gesture recognition does not require the user to wear any special equipment or attach any devices to the body. The gestures of the body are read by a camera ‘instead of sensors attached to a device’. In addition to hand and body movement, gesture recognition can be used to read facial and speech expressions (ie. lip reading) and eye movements. A number of systems have been designed to use gestural input devices to control computer memory and display. These systems perceive gestures through variety of methods and devices. While all the systems presented identify gestures, only some systems transform gestures into appropriate system specific commands.

Display

Human created

Gesture input device

CPU

Memory

Fig. 1.1. Architecture for Gestural Control of Memory and Display

Murakami’s system inputs the gesture data via a data glove. Instead of geometric modeling a neural network identifies gestures based on a finger alphabet containing 42 symbols. Several hours were required to train the neural network. Darell’s monocular vision processing system accurately identifies a wide variance of yes / no hand gestures [102]. Gestures are represented by a view-based approach and stored patterns are matched to perceived gestures using dynamic fine warping. Through the use of specialized hardware and a parallel architecture, processing time is reduced.

Neural and Functional Networks [22]

5

Baudel’s system [101] uses symbolic gestures to control a Machintosh hypertext program. Baudel’s system identifies natural hand gestures and hence is more intuitive to use than a standard mouse or stylus control system. The ALIVE II system [82] identifies full body gestures opposed to hand gestures, through basic image processing techniques. Here the user can use gestures to point at or grab object in the virtual environment.

1.6 Competitive Hopfield Neural Network The Hopfield neural network is commonly applied to obtain an optimal solution in various scheduling applications. A competitive learning rule provides a highly effective solution that reduces the network complexity. This feature is applied to the Hopfield neural network to derive a new technique called the Competitive Hopfield Neural Network technique (CHNN) [93]. A competitive learning rule reduces the time consumed in obtaining the coefficients and also provides an effective and sound solution. The CHNN has been applied to various fields such as image clustering processes, medical image segmentation and color image segmentation and also for polygonal approximation. The CHNN was used in the study to resolve a multiprocessor scheduling problem with time constraints (execution time and deadline), no process migration and limited resources. The CHNN ensured an appropriate approach to solving scheduling problems, when imposed on the energy function proposed in [93]. The CHNN was considered on a multiprocess in a multiprocessor system and attempted to obtain a set of schedules.

1.7 Neural and Functional Networks [22] Artificial neural networks have been recognized as a powerful tool to learn and reproduce systems in various fields of application. Neural Networks are inspired by brain behaviour and consist of one or several layers of neurons connected by links. Artificial neurons are concatenated or assembled to build a neural network with the aim of reproducing a multi dimensional function giving a set of output values as a function of the corresponding set of input values. The neural network has the ability to learn from data. The learning process is achieved by changing the networks’ architecture (links) and the connection weights according to the given information. When selecting the topology of the network and its associated weights, the required information can be derived either from data, or from domain knowledge, or from different combinations of the two. Earlier research tended to be on the data end of the spectrum, but subsequent research gradually has introduced the domain knowledge, either as general information about the required function or as specific domain knowledge such as that derived from dimensional theory, to supplement information available in the data. Several of the extensive application of artificial standard neural networks in several domains, are treated as black boxes. This may be an advantage if the user does not care about the functional structure of the function being modeled but will be a great disadvantage if considered otherwise.

6

Introduction

Functional networks [22] is an alternate to artificial neural networks that can bring together information from both ends of the spectrum. They can use domain knowledge to determine the structure of the network and the data to determine the unknown functions, which can be assumed to belong to a given family to be estimated during the learning process. Functional networks require domain knowledge for deriving the functional equations and make assumptions about the form the unknown functions should take. The class of problems where functional networks are most convenient is the class where the two sources of knowledge are available namely domain and data. The main extensions of functional networks as a powerful alternative to standard neural networks are based on (1) the multivariate and multiargument character of neural functions, (2) the possibility of using different neural functions, (3) the possibility of connecting neuron inputs and (4) the fact that neural functions are learned instead of weights when different neuron outputs are connected to the same intermediate or output unit. Functional equations appear that substantially reduce the degrees of freedom of the initial neural functions thus leading to important simplifications. Some applications namely rock identification and hot extrusion of steel using functional networks have been investigated.

1.8 Objectives and Scope of the Investigation This book deals with the following architectures and how they are implemented and their application to pattern recognition problems. 1. to investigate the application of Kohonen Self Organising Map (SOM) in recognition of both Tamil and English alphabets. 2. to apply MicroARTMAP for recognizing hand written characters and numbers. 3. to apply Hu’s method of moments for extracting certain features which are invariant to rotation, scaling and translation transformations. 4. to carry out character recognition using MicroARTMAP with the extracted features. 5. to investigate the application of wavelet transforms in data compression and to recognize characters using MicroARTMAP with these compressed coefficients. 6. to study the application of MicroARTMAP for pattern recognition in civil engineering problems. 7. to make comparison of the efficiency of MicroARTMAP with BPN in gesture recognition. 8. to solve scheduling problem with Competitive Hopfield Neural Networks.(CHNN). 9. to apply Functional Networks and Multiple Regression methods to rock identification problem and hot extrusion of steel.

1.9 Organization of the Book Chapter 2 discusses the architectures chosen relevant to the investigation. Recognition of handwritten alphabets using Kohonen Self Organizing Map has been described in Chapter 3.

Summary

7

Chapter 4 describes the architecture of the MicroARTMAP and its capability. The MicroARTMAP is augmented with a moment based feature extractor and applied to character recognition in Chapter 5. Chapter 6 illustrates the use of wavelet transforms together with MicroARTMAP for character recognition. Gesture Recognition and signature verification using MicroARTMAP and Back Propagation Network have been compared and presented in Chapter 7. Solving scheduling problem by means of a Competitive Hopfield Neural Network is discussed in Chapter 8. Multiple Regression methods considered as a recognizer is compared with functional networks in solving certain problems as shown in Chapter 9. Conclusion and further applications are suggested in Chapter 10.

1.10 Summary In this chapter the various neural network architectures and other domains taken for study have been introduced briefly. They are the Kohonen Self Organizing Map, MicroARTMAP, Wavelet Transforms, Competitive Hopfield Neural Networks, Multiple Regression and Functional Networks. The survey of the literature relevant to the study has been referred to in the places appropriately. The scope and objectives of the study are mentioned clearly.

2 Review of Architectures Relevant to the Investigation

2.1 Introduction Humans recognize the faces of friends in a crowd, characters and words on the printed page, the voices of acquaintances, favourite melodies, the scent of fresh lime, textures of tree bark, patterns of weave in cloth, the shape of leaves and the contextual meaning in word phrases. The senses preprocess signals such as sound or light waves that have been modulated. The preprocessed modulated signals are then mapped into a decision that equates to recognition of the object. Such processing discriminates subtle differences in the modulation of the signals to perform pattern recognition. When it is determined that an object from a population P belongs to a known subpopulation S, then pattern recognition is done. Recognizing a single object is identification, whereas classification is the process of grouping objects together into classes (subpopulations) according to their perceived similarities. Pattern recognition includes both classification and recognition [4], [9]. Reasoning is a process of applying general rules, equations or relationships to an initial collection of data to deduce a result. Learning is done by a system when it records its experience into internal system changes that cause its behaviour to be changed. Classification is a form of learning whereas recognition is a form of reasoning. Automated pattern recognition systems use computers that execute programs of instructions to implement mathematical algorithms. Applications exist in every field like optical character recognition, recognition of printed characters, recognition of spoken words, recognition of biochemicals from samples of blood, hair and tissue and recognition of patterns in physiological signals like ECG (Electrocardiogram) and EMG (Electromyogram). On attempting to understand the human computational power, two new fields emerged, fuzzy logic and artificial neural networks [55], [88]. Though developed independently, merging them gets us closer to understanding the human computational power. Approximate human reasoning in knowledge - based systems resulted in fuzzy logic [111]. The quest for mimicking the biological sensory systems in pattern recognition resulted in neural networks [78], [9]. Though pattern recognition is still at the heart of neural networks applications, they have also been proven useful in many other applications such as V.K. David and S. Rajasekaran: Pattern Recog. Using Neural & Funct. Net., SCI 160, pp. 9–13. c Springer-Verlag Berlin Heidelberg 2009 springerlink.com

10

Review of Architectures Relevant to the Investigation

optimization, computations and decision making. Neural networks have the advantage of learning, adaptation and fault tolerance [106]. The crucial steps in using functional networks to solve a problem requires the understanding of the problem, the topology of the initial functional network to be selected based on the knowledge of the problem, simplification using functional equations, obtaining the data required to train the functional network and estimating the functions to be learned. Both linear method and non linear methods are used in learning the functions. Both neural networks and functional networks have been used for pattern recognition and results are fruitful.

2.2 Recognition through Self Organizing Map The Kohonen’s feature map or the Self-Organizing Map (SOM) is used to categorize alphabetic characters. The self-organizing map is an unsupervised network [59], [60], [62] and hence the network is not fed with the desired outputs. It is able to infer relationship and learn more by its self-organizing properties. The strategy used in SOM is winner-take-all strategy. The process adopted tends to support the competitive learning thereby singling out a winner neuron for a given pattern. The SOM (Self Organizing Map) recognized both Tamil and English alphabets in our study. [103]

2.3 The μARTMAP [35] Artificial neural networks [35] have been applied successfully to a wide variety of realworld problems and are capable of performing far better than some of the common symbolic learning algorithms. They are applied in tasks which need a human supervisor to confirm and have confidence in the way the network makes its predictions or detection of salient features hidden in the data which went previously unnoticed. Neural networks are used for knowledge refinement provided their concepts were easily interpretable. Though the Back Propagation type neural networks and Adaptive Resonance Theory networks were advanced in nature, the IF – THEN rules were derived more readily from a Fuzzy ARTMAP [10] architecture only. The fuzzy ARTMAP inherently represents acquired knowledge in the form of IFTHEN rules and large or noisy data sets cause them to generate too many rules [17]. This is known as category proliferation. A new architecture called μARTMAP (MicroARTMAP) has been proposed by Sanchez et al. in [35] which combines probabilistic information in order to reduce the number of categories by optimizing their sizes and the use of an inter-ART reset mechanism which allows the correct treatment of exceptions. Under a probabilistic setting, the μARTMAP seeks a partition of the input space that optimizes the mutual information with the output. The μARTMAP is similar to fuzzy ARTMAP in several ways but is more robust to noise than the Fuzzy ARTMAP and less degrading as dimensionality increases. This network has been applied to the recognition of handwritten characters and performs comparably to fuzzy ARTMAP while generating a much more compact rule set.

Competitive Hopfield Neural Network

11

2.4 Wavelet Transforms and MicroARTMAP Wavelet transforms are used to evaluate and process signals. The field includes work by Ingrid Daubechies, Y. Meyer, Ronald Coifman and Victor Wickerhauser [25], [26]. Wavelet transforms are capable of providing the time and frequency information simultaneously, thus giving a time-frequency representation of a given signal. The wavelet coefficients that have to be calculated at every possible scale involves a lot of data being generated. The dyadic scales and positions are chosen and they are obtained from the discrete wavelet transform (DWT). The Mallat algorithm [70] yields a fast wavelet transform, which allows wavelet coefficients to emerge quickly. The wavelet db1 is used to obtain the coefficients from the handwritten character which are further given as compressed inputs to a MicroARTMAP to recognize handwritten characters.

2.5 MicroARTMAP and Gesture Recognition The primary goal of gesture recognition [66] is to create a system which can identify specific human gestures and use them for device control or to convey information. Vision based user friendly interfaces like hand gestures are used for man-machine interaction. The hand gesture image is transformed to the wavelet domain and features are extracted. These features are fed as inputs to MicroARTMAP which are trained to recognize them. Signatures were recognized using MicroARTMAP and Block Processing [6].

2.6 Competitive Hopfield Neural Network Various schemes have been developed for solving scheduling problems. Generally linear programming was used to determine the cost function based on a specific scheduling problem. Willems and Rooda [110] translated the job shop scheduling problem into a linear format and then mapped it into an appropriate neural network structure to obtain a solution. Hopfield and Tank [48] led the way in using the neural network to solve optimization problems. The basic operation of the Hopfield neural network co-operatively decides neuron output state information based on the state input information from a community of neurons. The neurons apply this information to drive the network to achieve convergence. Each neuron exchanges information with other neurons in the network. They allow the energy function to reach an optimum solution with less computation. The energy function [49], [93] used in the Hopfield Neural Network is an appropriate Lyapunov function. A job schedule problem of a multiprocess on a multi processor that includes timing as well as resource constraints has been investigated via a competitive Hopfield neural network (CHNN). The energy function as proposed in [93] is used to illustrate the timing and resource constraint and is according to the intrinsic constraints of the scheduling problem. The energy functions are translated to the CHNN algorithm. The energy function converges as seen through the simulation examples. The problem is focussed on resource utilization. Each job would require different resources in a specific

12

Review of Architectures Relevant to the Investigation

order at different times. The energy function can be modified by adding additional terms to satisfy additional requirements. ⎛ ⎞2 N M T T N M T M C2 C3 ⎝ E= Vijk Vij1 k1 + Vijk − Pi ⎠ 2 i=1 j=1 2 i=1 j=1 j =1 k=1

+

1 j=j1

k1 =1

k=1

N M T C5 Vijk G2ijk H(Gijk ) 2 i=1 j=1 k=1

+

N M T M F N C6 Vijk Ris Vi1 j1 k Ri1 s 2 i =1 j =1 s=1 i=1 j=1 k=1

1 j=j1

(2.1)

1 j=j1

The synaptic interconnection strength, Wxyzijk and bias input θijk are given as Wxyzijk = −C2 δ(x, i)(1 − δ(y, j)) − C3 δ(x, j) −C6 (1 − δ(x, i))(1 − δ(y, j)) δ(z, k)

Rxs Ris

(2.2)

s

and θxyz = −C3 Pi +

C5 2 G H(G) 2

(2.3)

1 if a = b 0 if a = b The term θxyz in the energy equation 2.1 has been modified from [93] by removing the term Ckt when the program was terminated for E = 0.

where δ(a, b) =

2.7 Functional Networks and Multiple Regression Analysis Functional networks were introduced by Castillo [21], [22] as a powerful alternative to neural networks. Functional networks use domain knowledge in addition to data knowledge. They conveniently deal with the class where both the sources of knowledge are available. Regression approximately maps a set of given example input points into a given set of output points and this is equivalent to recognition. Functional networks control a system by recognizing the state of the system and hence are tools for non-linear regression analysis. The results obtained for two applications by both multiple regression method as well as functional network method agree well.

2.8 Summary Neural networks can be used for problems that cannot be solved with a known formula and for problems with incomplete or noisy data. They have the capacity to recognize

Summary

13

patterns in the data presented to them and are thus useful in many types of pattern recognition problems. Functional networks is a generalization of neural networks bringing together knowledge and data. They are more efficient and powerful taking less computer time when compared with neural networks. The functional network uses two types of learning. They are structural learning and parametric learning. In some functional networks, the learning method leads to a global minimum in a single step, thus reducing the learning time and also the problems associated with local and global optima.

3 Recognition of English and Tamil Alphabets Using Kohonen’s Self-organizing Map

3.1 Introduction The Kohonen feature map or the Self-Organizing Map (SOM) is used to recognize or categorize alphabetic characters [47], [59], [60], [61], [62]. The SOM is an unsupervised network and hence no expected outputs are presented to the neural network. By its self-organizing properties, it is able to infer relationships and learn more, as more inputs are presented to it. The network is able to recognize all the English and certain Tamil alphabets and also garbled versions of the same.

3.2 Recognition of Handwritten Characters Using Ordered List of Image Pixels on Its Boundary Handwritten characters (alphabets, numbers and signs) are recognized using the algorithm for computing an ordered list of the boundary pixels [9]. The handwritten characters are compared with the stored pattern of the characters and recognized when it matches to about 70 per cent and above. When the percentage is less while matching, the nearest matching alphabet is given along with the percentage of match. Provision is made to teach the handwritten pattern of Tamil and English alphabets in the program in visual basic with Aishwarya Tamil font and the newly taught handwritten character is stored in memory. Every drawn character is first passed and the software changes the image into lower resolution. The user may teach the character into the system. This will be encoded and stored in a database for recognition. During recognition process the best match for the drawn character will be retrieved. Figs. 3.1, 3.2, 3.3 and 3.4 show the various characters recognized by the program. About 70% match was obtained by the program for the example patterns that were considered. Algorithm (Ordered List of Boundary Pixels) [9] 1. Scan the image (left to right, top to bottom) until a pixel in the desired region is entered. 2. If the current pixel is in the object, turn left (compared to the direction from which the current pixel was entered) and step one pixel in that direction. V.K. David and S. Rajasekaran: Pattern Recog. Using Neural & Funct. Net., SCI 160, pp. 15–26. c Springer-Verlag Berlin Heidelberg 2009 springerlink.com

16

Recognition of English and Tamil Alphabets Using Kohonen’s Self-organizing Map

Fig. 3.1. Recognition of Alphabet P

Fig. 3.2. Recognition of Alphabet Close to R

Recognition of Handwritten Characters Using Ordered List of Image Pixels

Fig. 3.3. Nearest Recognition of U

Fig. 3.4. Alphabet A Taught to the System

17

18

Recognition of English and Tamil Alphabets Using Kohonen’s Self-organizing Map

3. Else if the current pixel is outside the object turn right and step one pixel in that direction. 4. Repeat steps 2 and 3 until the first pixel is entered a second time from the same direction, then stop. Each time a region pixel is entered, its location is placed on an ordered list, with the restriction that a pixel is never placed on the list twice in succession and the starting pixel is not listed the first time it is entered. Inconsistency of the Algorithm This algorithm is inconsistent in its treatment of corner-connected regions. The one pixel region at the upper right is as if it were a part of the main object, while the region on the lower right is not. But this inconsistency would not matter in images where corner-neighbouring regions do not appear [9].

3.3 The Kohonen Feature Map The Kohonen SOM (Fig. 3.5) is a case of using winner-take-all strategy and is found to recognize alphabets. Inputs xj with weights wij are fed into each of the neurons in the Kohonen layer (from the input layer). Each neuron determines its output according to the general weighted sum formula. Output = wij xj (3.1)

Input layer

Kohonen layer W

1 0 0 0 0 0

Fig. 3.5. Kohonen Network

Representation of Characters

19

The weights and inputs are usually normalized. The winning neuron is the one with the largest output and this has a final output of 1.

3.4 Normalization of a Vector If A = ax + by + cz is a vector, then A’the normalized vector is obtained by multiplying 1 each component of A by √a2 +b . Both the weight vector and the input vector are 2 +c2 normalized during the operation of the SOM. The reason for this is that the training law (3.2) uses subtraction of the weight vector from the input vector.

3.5 Training Law The training law used is Wnew = Wold + alpha ∗ (Input − Wold )

(3.2)

alpha is a gain constant between 0 and 1. The old weight vector and the input vector are normalized to unit length.

3.6 Neighbourhood Size The neurons that are within the distance specified by the neighbourhood size participate in training and the weight vector changes. Those outside this distance do not participate in the learning. The neighbourhood size is started as an initial value and is decreased as the input pattern cycles continue. The process tends to support the winner-take-all strategy by eventually singling out a winner neuron for a given pattern.

3.7 The Kohonen Network The Kohonen network [3], [14], [18], [103], [107], has two layers, an input layer and a Kohonen output layer. The input layer size is determined by the user and must match the size of each row (pattern) in the input data file. The lateral inhibition and excitation is modeled by looking at the maximum output for the output neurons and making that output belong to a winner neuron. Other outputs are set to zero. Training or weight update is performed on all outputs that are within a neighbourhood size distance from the winner neuron. The criterion for ending the program pattern.cpp is the average winner distance (see Appendix – F). This is the Euclidean distance between the input vector and the winner’s weight vector. This is the square root of the sum of the squares of the differences between individual vector components between the two vectors.

3.8 Representation of Characters Each character (English, Tamil alphabet or mathematical sign) is represented by a 5 × 7 grid of pixels. The graphical printing characters of the IBM extended ASCII character is done using grayscale output for each pixel. The blackened boxes represent value one and empty boxes are represented by a zero.

20

Recognition of English and Tamil Alphabets Using Kohonen’s Self-organizing Map

The alphabet A

Tamil alphabet Pa

00100 01010 10001 10001 11111 10001 10001

10001 10001 10001 10001 10001 10001 11111

The letter A is put in the input file. The input is in serialized rows so that all entries appear on one line. Letter A 0 0 1 0 0 0 1 0 1 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 1 1 0 0 0 1 Letter Pa 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 Many characters (including Tamil alphabets) are presented to the Kohonen map and the output response is noted. The Kohonen map goes through the cycles and learns the input patterns. The weight vectors develop a pattern. The weight vectors tend to become aligned with the input vectors. So after a while the weight vector for the input will resemble the input pattern that is being categorized.

3.9 Weight Vector The weight vector is quantized into four bins. Each of these is represented by a different ASCII graphic character so that the grayscale values for each weight vector can be seen. Quantizing the Weight Vector <= 0 0 < weight <= 0.25 0.25 < weight <= 0.50 0.50 < weight <= 0.75 Weight > 0.75

White rectangle (space) Light-dotted rectangle Medium-dotted rectangle Dark-dotted rectangle Black rectangle

The ASCII Values for Rectangle Graphic Characters White rectangle Light-dotted rectangle Medium-dotted rectangle Dark-dotted rectangle Black rectangle

255 176 177 178 219

The input file is created for the needed characters either English or Tamil. A run of the program with the inputs for various alphabets given two at a time are tabulated.

Weight Vector

21

Table 3.1. Recognition run for English Alphabets (Two inputs given at the same time)

cycle

pattern

win index

neigh_size

avg_dist_per_patter

0 0 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 10 10 11 11 12 12 13 13 14 14 15 15 16 16 17 17 18 18 19 19 20 20

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41

36 76 36 76 36 76 36 76 36 76 36 76 36 76 36 76 36 76 36 76 36 76 36 76 36 76 36 76 36 76 36 76 36 76 36 76 36 76 36 76 36 76

5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5

100.000000 100.000000 1.110680 1.110680 0.777476 0.777476 0.544233 0.544233 0.380963 0.380963 0.266674 0.266674 0.186672 0.186672 0.130670 0.130670 0.091469 0.091469 0.064028 0.064028 0.044820 0.044820 0.031374 0.031374 0.021962 0.021962 0.015373 0.015373 0.010761 0.010761 0.007533 0.007533 0.005273 0.005273 0.003691 0.003691 0.002584 0.002584 0.001809 0.001809 0.001266 0.001266

22

Recognition of English and Tamil Alphabets Using Kohonen’s Self-organizing Map

Table 3.2. Recognition run for both English and Tamil Alphabets (Two inputs given together)

cycle

pattern

win index

neigh_size

avg_dist_per_patter

0 0 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 10 10 11 11 12 12 13 13 14 14 15 15 16 16 17 17 18 18 19 19 20 20

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41

60 83 60 83 60 83 60 83 60 83 60 83 60 83 60 83 60 83 60 83 60 83 60 83 60 83 60 83 60 83 60 83 60 83 60 83 60 83 60 83 60 83

5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5

100.000000 100.000000 1.076986 1.076986 0.753890 0.753890 0.527723 0.527723 0.369406 0.369406 0.258584 0.258584 0.181009 0.181009 0.126706 0.126706 0.088694 0.088694 0.062086 0.062086 0.043460 0.043460 0.030422 0.030422 0.021296 0.021296 0.014907 0.014907 0.010435 0.010435 0.007304 0.007304 0.005113 0.005113 0.003579 0.003579 0.002505 0.002505 0.001754 0.001754 0.001228 0.001228

Weight Vector

23

Table 3.3. Recognition run for Tamil Alphabets (Two inputs given at the same time)

cycle

pattern

win index

neigh_size

avg_dist_per_patter

0 0 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 10 10 11 11 12 12 13 13 14 14 15 15 16 16 17 17 18 18 19 19 20 20

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41

15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 19 11 19

5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5

100.000000 100.000000 0.927960 0.927960 0.489715 0.489715 0.301244 0.301244 0.233631 0.233631 0.213378 0.213378 0.207795 0.207795 0.206240 0.206240 0.205773 0.205773 0.205616 0.205616 0.205557 0.205557 0.205532 0.205532 0.205521 0.205521 0.205515 0.205515 0.205513 0.205513 0.205512 0.205512 0.205511 0.205511 0.205511 0.205511 0.205511 0.205511 0.205511 0.205511 0.205510 0.205510

24

Recognition of English and Tamil Alphabets Using Kohonen’s Self-organizing Map Table 3.3. (continued)

cycle

pattern

win index

neigh_size

21 21 22 22 23 23 24 24 25 25 26 26 27 27 28 28 29 29 30 30 31 31 32 32 33 33 34 34

42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69

11 19 13 19 10 19 11 19 13 19 10 20 10 19 10 19 10 10 10 19 10 19 10 19 10 19 10 19

5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5

avg_dist_per_patter 0.143857 0.143857 0.100700 0.100700 0.070490 0.070490 0.049343 0.049343 0.034540 0.034540 0.024178 0.024178 0.016925 0.016925 0.011847 0.011847 0.008293 0.008293 0.005805 0.005805 0.004064 0.004064 0.002845 0.002845 0.001991 0.001991 0.001394 0.001394

The tolerance for the distance was set to be 0.001 for the program pattern.cpp and the program was able to converge to this value. All the inputs given were successfully classified into different winning output neurons. The starting neighbourhood size is taken as 5, it remains the same and the Kohonen map converges to a solution. At times the neighbourhood size reduces to 0 and then Kohonen map generalizes arriving at the middle value for all input vectors. Choosing a small neighbourhood size of say 2, provides less generalization (converges for unique winner). Only the neurons within a certain distance (neighbourhood) from the winner neuron are allowed to participate in training for a given input pattern. As training progresses, the weight vector for the winner neuron resembles the input character map. It is found that the vector resembles the input as training progresses but it is not an exact replication. This alphabet classifier system is able to process noisy data. As more characters were added into the input file, the network did not find any difficulty in classifying them. When the inputs are

Weight Vector

25

α

Table 3.4. Comparison of the Recognition of Alphabets

Pa La Ma La Ya Pa # NBD - Neighbourhood Input given in pairs Winner index – neuron which emerges as winner. α - chosen by trial and error Pattern – given alphabets. Nbd size – Neighbourhood size (chosen by trial and error). Period – number of cycles to execute before decreasing the value α. Cycle – each pass through the data (starting value 500).

Recognition Run Average Distance

10 1 0.1 0.01 0.001 1

6

11 Cycle No

16

Fig. 3.6. Semi-log Plot Between Average Distance and Cycle Number

26

Recognition of English and Tamil Alphabets Using Kohonen’s Self-organizing Map

given in different order, the Kohonen network is able to train faster. Garbled versions of characters are also distinguished by the Kohonen network. Tables 3.1, 3.2, 3.3 and 3.4 illustrate the results. Fig. 3.6 shows the graph depicting average distance versus cycle number plotted on a semi log plot. Generally a subset of the characters are trained. The number of neurons needed to recognize the complete alphabet when noisy input is given can be determined. Both the input vectors and the weight vectors are displayed to note the convergence and the similarities between the two vectors.

3.10 Summary Recognition of hand written characters has been carried out using the algorithm for ordered list of boundary pixels as well as the Kohonen Self-Organizing Map (SOM). The algorithm was found to be inconsistent. Using the Kohonen feature map, the input vectors and the weight vectors were displayed to show convergence and to note similarity between the two vectors. As training progressed, the weight vector for the winning neuron resembled the input character map. The system was found to change with changing conditions and inputs and learnt constantly. Being an unsupervised neural network, the SOM was used as a classifier system. The weight vectors were aligned in the same direction as the input vectors due to the Kohonen training law. For the characters that look alike in English such as P and B the classifier grouped them into the same class. The weight vector though resembled the input vector was not found to be an exact replication of the input vector. The network was able to recognize English alphabets N, T, U, V, P, X, Y, W and certain Tamil alphabets like, Sa, Pa, Ma, Ya, Ta, La, Va, Ra, La (in Ullae), though all the characters were given a trial.

4 Adaptive Resonance Theory Networks

4.1 Introduction

The Adaptive Resonance Theory (ART) model is one of the triumphs of modern neural network theory developed to overcome the stability-plasticity dilemma [1], [2], [37], [38], [47]. When someone has to add more to what they have already learnt, there is no need for complete retraining as there is no chance of one forgetting all that he has already learnt. But such is not the case in electronic networks as a catastrophic forgetting may occur if one tries to add more patterns to a stable trained network. There is a tradeoff between the stability and ability to learn new data. This is known as the stability plasticity dilemma. The ART network produces a stable network that can learn new data. The main idea behind this is to spare some of the output units for new patterns. When the input and a stored pattern are sufficiently similar, they are said to resonate. When there is not sufficient similarity, a new class of patterns is formed utilizing the unused output units. There is no response from the network when all output units are used. Several versions of ART networks have been implemented : ART1, ART2, ART3 and ARTMAP [15], [19], [76]. ART1 can stably learn to categorize binary input patterns presented in an arbitrary order. ART2 does the same with either analog or binary input patterns whereas ART3 can carry out parallel search, or hypothesis testing, or distributed recognition codes in a multilevel network hierarchy. The ARTMAP in turn can rapidly self-organize stable categorical mappings between n-dimensional inputs and m-dimensional output vectors. The fuzzy ART model [16], [17], [32], [41], [119] approximates surfaces in a fuzzy cube of high dimension. Each pattern is considered as a discrete fuzzy set. This ART system uses the degree of subsethood to control how well fuzzy patterns match. The system covers the decision surface with hyper boxes just as a feed forward fuzzy system covers a function’s graph with rule patches. The growth in the dimension of the space results in an explosion. The fuzzy ART model uses subsethood to control the fineness of the hyper box cover in a fixed fuzzy cube. This result blends the core concepts of fuzzy cubes and feedback neural networks in a novel and powerful architecture. V.K. David and S. Rajasekaran: Pattern Recog. Using Neural & Funct. Net., SCI 160, pp. 27–49. c Springer-Verlag Berlin Heidelberg 2009 springerlink.com

28

Adaptive Resonance Theory Networks

4.2 ART Network Grossberg [14], [18] and his associates have worked for nearly 20 years on theories to explain human cognition in terms of neural network operations. Their research has culminated in a model called ART. In a competitive learning scheme, there is no guarantee that the clusters formed will be stable unless the learning rate gradually approaches zero with iteration. When this happens, the network loses its plasticity. The ART1 network overcomes this dilemma. In ART, a weight vector (prototype of a category) is adapted only when the input is sufficiently similar to that of the prototype, that is when the input and a prototype resonate. When an input is not sufficiently similar to any prototype, a new category is formed using the input as the prototype. This condition is checked using a vigilance parameter. The ART network gets its name from the particular way in which learning and recall interplay in the network. In physics, resonance occurs when a small-amplitude vibration of the proper frequency causes a large-amplitude vibration in an electrical or mechanical system. In ART network, information in the form of processing element outputs, reverberates back and forth between layers. If the proper pattern develops, a stable oscillation ensues, which is the neural network equivalent of resonance. During resonant period, learning or adaptation can occur. Before the network has achieved a resonant state, no learning takes place, because the time required for changes in the processing element weights is much longer than the time that it takes the network to achieve resonance. 4.2.1

Resonant State

A resonant state can be attained in one of the two ways. If the network has learned previously to recognize an input vector, then when that input vector is presented, a resonant state will be achieved quickly. During resonance, the adaptation process will reinforce the memory of the stored pattern. If the input vector is not immediately recognized, the network will rapidly search through its stored patterns looking for a match. If there is no match to be found, the network will enter a resonant state whereupon the new pattern will be stored for the first time. Thus the network responds quickly to previously learned data and yet remains able to learn when novel data are presented. 4.2.2

The STM and LTM Traces

The ART model comprises of two subsystems namely the attention subsystem and the orienting subsystem. Familiar or previously learned patterns are processed within the attentional subsystem. This subsystem establishes precise internal representations to familiar events. By itself this subsystem is unable to simultaneously maintain stable representations of familiar categories and create new categories for the unfamiliar patterns. The second subsystem fills this inadequacy by resetting the attentional subsystem when unfamiliar events occur. This orienting subsystem is essential for determining whether a new pattern is familiar and has a recognition code or unfamiliar and in need for a new recognition code. Patterns of activity that develop over the nodes in the two

ART Network

29

layers of the attentional subsystem are called short-term memory (STM) traces because they exist only in association with a single application of an input vector. The weights associated with the bottom up and top-down connections between F1 and F2 are called long-term memory (LTM) traces because they encode information that remains a part of the network for an extended period.

G2 +

+

Orienting subsystem A

Attentional subsystem F2 Layer

Gain control

+ +

+ -

+

-

+

+

G2 Gain control

F1 Layer

+

Input Vector

• • • • • • • • • •

Reset signal

F1 - layer of neurons called input layer or comparison layer. A node for each layer as a gain control unit. F2 – layer of neurons called output layer or recognition layer A node as a reset unit Bottom-up connections from F1 layer to F2 layer Top-down connections from F2 layer to F1 layer Inhibitory connection from F2 layer to gain control Excitatory connection from gain control to a layer Inhibitory connection from F1 layer to reset node Excitatory connection from reset node to F2 layer. Fig. 4.1. ART Network

30

Adaptive Resonance Theory Networks

4.2.3

The Structure of the ART Model

The core structure of the ART model shown in Fig. 4.1 consists of two stages or layers plus some additional contact connections [37]. The major subsystems of the ART system are the attentional subsystem and the orienting subsystem . F1 and F2 represent two layers of nodes in the attentional subsystem. Nodes on each layer are fully interconnected to the nodes on the other layer. The interconnects among the nodes on each layer are not shown in the Figure 4.1, but the other connections are indicated by arrows. A plus sign indicates an excitatory connection and a minus sign indicates an inhibitory connection. The ART system does pattern-matching operation during which the network tries to determine whether an input pattern is among the patterns previously stored in the network or not.

F2

=Y

F2 0

0

=Y 0 =U

1

G

G _ 1

0

1

0

F1

A

=S

0

1

0

V= 1 F1 1

+

=X

1

_

=I

0 0

0

1

0

=I

0

F2

= Y*

G

_

_

1

A

F1

+

1

0

1

(c) Reset

0

=I

+

(b) Pattern – matching attempt

=Y

G

A

=S*

0

= X*

1

(a) Presentation of input pattern F2

0 0

0

1

0

F1

=S =X

1

0

1

0

(d) Final recognition

Fig. 4.2. A Pattern-Matching Cycle in an ART Network

=I

A +

ART Network

4.2.4

31

Pattern-Matching Cycle in an ART Network

In Fig 4.2(a), a input pattern I, is presented to the units on F1 . One vector component goes to each node. A pattern of activation X, is produced across F1 . The same input pattern excites both the orienting subsystem, A and the gain control G. The output pattern S, results in an inhibitory signal that is also sent to A. The network is structured such that this inhibitory signal exactly cancels the excitatory effect of the signal I, so that A remains inactive. G supplies an excitatory signal to F1 . The same signal is applied to each node on the layer and is known as a nonspecific signal. The appearance of X on F1 results in an output pattern S, which is sent through connections to F2 . Each F2 unit receives the entire output vector S from F1 . The net input values are calculated in the F2 units by summing the product of the input values and the connection weights. In response to inputs from F1 , a pattern of activity Y, develops across the nodes of F2 , which is a competitive layer that performs a contrast enhancement on the input signal. In Fig. 4.2(b) the pattern of activity Y results in an output pattern U from F2 . This output pattern is sent as an inhibitory signal to the gain control system. This gain control is configured such that if it receives any inhibitory signal from F2 , it ceases activity. U also becomes a second input pattern for F1 units. U is transformed to pattern V, by LTM traces on the top-down connections from F2 to F1 . However patterns of activity that develop over the nodes in the two layers of the attentional subsystem are the shortterm memory (STM) traces. 4.2.5

The 2/3 Rule

There are three possible sources of input to F1 , but only two are used at a time. The units on F1 and F2 are constructed so that they can become active only if two out of the possible three sources of input are active. This feature is called the 2/3 rule. Due to the 2/3 rule, only those F1 nodes receiving signals from both I and V will remain active. So the pattern that remains on F1 is I ∩ V. In Fig. 4.2(b) patterns mismatch and a new activity pattern X∗ develops on F1 . As the new output pattern S∗ , is different from the original pattern S, the inhibitory signal to A no longer cancels the excitation coming from I. In Fig. 4.2(c), A becomes active in response to the mismatch of patterns on F1 . A sends a non-specific reset signal to all nodes on F2 . These nodes respond according to their present state. If they are inactive, they do not respond. If they are active, they become inactive and they stay that way for an extended period of time. This sustained inhibition is necessary to prevent the same node from winning the competition during the next matching cycle. Since Y no longer appears, the top-down output and the inhibitory signal to the gain control also disappear. In Fig. 4.2(d) the original pattern X, is reinstated on F1 , and a new cycle of pattern matching begins. Now a new pattern Y∗ , appears on F2 . The nodes participating in the original pattern Y, remain inactive due to the long term effects of the reset signal from A. This cycle of pattern matching will continue until a match is found, or until F2 runs out of previously stored patterns. If no match is found, the network will assign some

32

Adaptive Resonance Theory Networks

uncommitted node or nodes on F2 and will begin to learn the new pattern. Learning takes place through the modification of the weights, or the LTM traces. This learning process does not start or stop, but continues even while the pattern matching process takes place. Whenever signals are sent over connections, the weights associated with those connections are subject to modification. The mismatches do not result in loss of knowledge or the learning of incorrect associations. This is due to the fact that the time required for significant changes to occur in the weights is very long with respect to the time required for a complete matching cycle. The connections participating in mismatches are not active long enough to affect the associated weights seriously. When a match does occur, there is no reset signal and the network settles down into a resonant state. During this stable state, connections remain active for sufficiently long time so that the weights are strengthened. This resonant state can arise only when a pattern match occurs, or during the enlistment of new units on F2 in order to store a previously unknown pattern. 4.2.6

Gain Control Mechanism

The gain control and the 2/3 rule are important features of the ART system. Let this ART network be the only one in a hierarchy of networks in a much larger system. The F2 layer might be receiving inputs from a layer above it as well as from the F1 layer below. In case the F2 layer was stimulated by an upper layer, it could produce a topdown output and send signals back to the F1 layer. It is possible that this top-down signal would arrive at F1 before an input signal I arrived at F1 from below. A premature signal from F1 could be the result of an expectation arising from a higher level in the hierarchy. F2 is actually indicating what it expects the next input to be before the pattern actually arrives at F1 . The appearance of an early top-down signal from F2 results in a small dilemma. Suppose F1 produced an output in response to any single input vector, no matter what the source, then the expectation signal arriving from F2 would elicit an output from F1 and the pattern matching cycle would ensue without ever having an input vector to F1 from below. G nonspecifically arouses every F1 unit and if G exists, any signal coming from F2 results in an inhibition of G. With the 2/3 rule in effect, inhibition of G means that a topdown signal from F2 cannot, but itself, elicit an output from F1 . The F1 units become preconditioned by the top-down pattern. F1 units are said to have received a subliminal stimulation from F2 . If now the expected input pattern is received on F1 from below, this preconditioning results in an immediate resonance in the network. Even if the input pattern is not the expected one, F1 will still provide an output, since it is receiving inputs from two out of the three possible sources namely I, G and F2 . If there is no expectation signal from F2 , the F1 remains completely quiescent until it receives an input vector from below. Then, since G is not inhibited, F1 units are again receiving inputs from two sources and F1 will send an output up to F2 to begin the matching cycle. G and the 2/3 rule combine to permit the F1 layer to distinguish between an expectation signal from above, and an input signal from below. When an expectation signal to

Fuzzy ART

33

F1 is from above, F1 ’s response is subliminal when an input signal to F1 is from below, F1 ’s response is supraliminal that is it generates a nonzero output. Algorithm Apply an input vector I to F1 and calculate the F1 activities. Calculate the output vector for F1 . Propagate S forward to F2 and calculate activities in F2 . Only the winning F2 node has a nonzero output Propagate the output from F2 back to F1 . Calculate the net inputs from F2 to F1 units. 6. Calculate the new activities. 7. Determine the new output values (as in step 2). 8. Determine the degree of match between the input pattern and the top down template using |S| |I| . 1. 2. 3. 4. 5.

9. If

|S| |I|

< ρ make VJ as inactive, zero the outputs of F1 and return to step 1 using the

original input pattern. If |S| |I| ≥ ρ continue. 10. Update bottom-up weights on VJ only 11. Update the top-down weights coming from VJ only to all F1 units. 12. Remove the input pattern. Restore all inactive F2 units. Return to step 1 with a new input pattern.

4.3 Fuzzy ART Fuzzy ART inherits the design features of other ART models [10], [16], [17]. When the intersection operator (∩) of ART is replaced by fuzzy intersection or point wise minimum operator (∧), the ART translates into fuzzy ART operations of category choice, matching, searching and learning. There is a close formal homology between the fuzzy ART and a neural network. Carpenter et al. [16] describe a neural network realization of fuzzy ART. The computations of fuzzy ART are the same as those of the ART neural network for the special case of binary input vectors and fast learning. 4.3.1

Analogy between ART1 and Fuzzy ART

ART 1 (Binary)

Fuzzy ART (Analogous) Category Choice

Tj(I) =

|I∩wj | Į+ | w j |

Tj(I) =

|I ∧ wj | Į+ | w j |

34

Adaptive Resonance Theory Networks

Match Criterion |I∩wj | |I|

|I∧wj |

≥ρ

|I|

≥ρ

Fast Learning wj(new) = I ∩ wj(old)

wj(new) = I ∧ wj(old)

∩ = Logical AND Intersection

∧ = Fuzzy AND Minimum

4.3.2

Fast-Learn, Slow-Recode and Complement Coding

Many applications of ART1 use fast learning where adaptive weights fully converge to equilibrium values in response to each input pattern. A system adapts quickly to inputs that occur rarely and require immediate accurate performance because of fast learning. Fast learning destabilizes the memories of feed forward error-based models like back propagation as it zeroes the error on each input trial. Fast commitment retains the advantage of fast learning – the ability to respond to important inputs that occur only rarely. Slow recoding prevents features in a category’s prototype from being wrongly deleted in response to noisy or partial inputs. Some applications benefit from a fast-commit slow recode option that combines fast initial learning with a slower rate of forgetting. Only a statistically persistent change in a feature’s relevance to an established category can delete it from the prototype of the category. 4.3.3

Complement Coding

Input patterns are normalized by complement coding, a preprocessing step that solves potential fuzzy ART category proliferation problem [16], [56], [57], [58], [63], [64]. Complement coding in neurobiological terms uses both on-cells and off cells to represent an input pattern and preserves individual feature amplitudes while normalizing the total on-cell / off-cell activity. The features that are critically present in category exemplars are encoded by the on-cell portion of a prototype. The off-cell portion encodes features that are critically absent. The features that are sometimes present and sometimes absent in both on-cell and off-cell portions of a prototype are encoded as uninformative by the small weights. The fuzzy intersection (∧) and the fuzzy union (∨) play complementary roles [111], [116]. Thus complement coding allows a geometric interpretation of fuzzy ART recognition categories as box-shaped regions of the input fuzzy space, where Fuzzy Intersections and Unions interactively define the corners of each box. The Fuzzy ART model (Fig. 4.3) approximates surfaces in a fuzzy cube of high dimension. Each pattern is a discrete fuzzy set. The ART system uses the degree of subsethood to control how well fuzzy patterns match. The system covers the decision

Fuzzy ART

35

Choice function | I ∧ wj | Tj = α+ | w j | Choice (J)

Reset

j

F2 wji

xi

yi f|x|

F1

Match xi = Ii ∧ wji

wji i

Ii

ρ|I|

F0

i Complement coding

a

ac

a In the fuzzy ART algorithm wj equals both the bottom-up weight vector and the top-down weight vector Fig. 4.3. The Fuzzy ART

36

Adaptive Resonance Theory Networks

surface with hyper boxes much as a feed forward fuzzy system covers a function’s graph with rule patches. All such graph covers lead to hyper box or rule explosion as the dimension of the space grows. The fuzzy ART model uses subsethood to control the fineness or grid size of the hyper box cover in a fixed fuzzy cube. The result blends the core concepts of fuzzy cubes and feedback neural networks in a novel and powerful architecture. Fuzzy ART system (Fig. 4.3) has three fields of nodes F0 , F1 and F2 . Field F0 represents current input vector and field F1 receives both bottom-up input from F0 and top-down input from field F2 . The field F2 represents the active code or category. Vector I = (I1 , I2 . . . . Im ) denotes F0 activity with each component Ii in the interval [0, 1] for i = 1, 2, . . . M. I is a point or fit vector in fuzzy n-cube I ∈ [0, 1]n . Vector x = (x1 , x1 , . . . . . . xm ) denotes F1 activity and y = (y1 , y2 , . . . . YN ) denotes F2 activity. The number of nodes in each field is arbitrary. 4.3.4

Weight Vectors

With each F2 category node j (j = 1, . . . . N) is associated a vector wj = (Wj1 , Wj2 , . . . WjM ) of adaptive weights or long-term memory traces (LTM). Wj1(0) = wj2(0) . . . WjM(0) = 1

(4.1)

Each category node is then uncommitted. After a category codes its first input it becomes committed. Each component wji can decrease but never increase during learning, thereby each weight vector wj (t) converges to a limit. The fuzzy ART weight or prototype weight – wj subsumes both the bottom-up and top-down weight vectors of ART1. 4.3.5

Parameters

The fuzzy ART dynamics is fixed by choosing three parameters [13]. A choice parameter α > 0, a learning rate parameter β ∈ [0, 1], and a vigilance parameter ρ ∈ [0, 1]. 4.3.6

Category Choice

For each input I and F2 node j, the choice function Tj is defined as Tj (I) =

| I ∧ wj | α+ | wj |

(4.2)

where the fuzzy intersection is defined as in [113], [117]. (p Λ q)i ≡ min(pi qi )

(4.3)

Fuzzy ART

37

and where the norm | . | is the fuzzy count c(p) given by | p |=

M

| pi |

(4.4)

l=1

The system makes a category choice when atmost one F2 node can become active at a given time. The index J denotes the chosen category. (4.5)

Tj = max{Tj : j = 1, . . . N}

If more than one Tj is maximal then the system chooses the category with the smallest j index. Thus nodes become committed in order j = 1, 2, 3 . . . . When the system chooses the Jth category then yJ = 1 and yj = 0 for j = J. In a choice system the F1 activity vector x obeys the equation ⎧ ⎨I if F2 is inactive (4.6) x= ⎩ I ∧ w if Jth F node node is chosen. J 2 4.3.7

Resonance or Reset

J| If the subsethood match function |I∧w of the chosen category meets the vigilance |I| criterion as follows then resonance occurs.

| I ∧ wJ | ≥ρ |I|

(4.7)

This implies that when the Jth category becomes active the resonance occurs if | x |=| I ∧ wJ |≥ ρ | I |

(4.8)

| I ∧ wJ | <ρ |I|

(4.9)

| x |=| I ∧ wJ |< ρ | I |

(4.10)

mismatch reset occurs if

and thus if The system then sets the value of the choice function. TJ to 0 for the duration of the input presentation to prevent the persistent selection of the same category during search. The index J represents the active category. The search process continues until J| the chosen J satisfies the matching criterion |I∧w ≥ρ |I| 4.3.8

Learning Law

The weight vector wJ learns once the search ends according to the equation [76], [95] wJ (new) = β(I ∧ wJ (old) ) + (1 − β)wJ (old) when β = 1, fast learning occurs.

(4.11)

38

Adaptive Resonance Theory Networks

This learning law (4.11) equals the learning law of nested generalized exemplars [16] in the fast-learn limit with complement coding. This learning law was introduced by Moore [76] with fast commitment and slow recoding to explore many of the generalized ART1 models. When J is an uncommitted node, to facilitate efficient coding of noisy input sets, it helps to set the learning rate parameter β = 1. After the system already commits the category, β < 1 is considered for slower adaptation. Many of the models of ART1 are similar to Fuzzy ART but none uses complement coding. Moore [76] describes a category proliferation problem that can occur in some analog ART systems when many random inputs erode the norm of the weight vectors. This problem is solved by complement coding. 4.3.9

Normalization of Fuzzy ART Inputs

Normalization of fuzzy ART inputs is done by complement coding and this prevents category proliferation. The system normalizes the F0 Î F1 inputs if for some α > 0 Ii =| I |= α for all inputs I (4.12) i

Generally to normalize each vector a, we divide it by the fit count I=

a |a|

(4.13)

Complement coding not only normalizes the input but it also preserves amplitude information. Complement coding both represents the on-response and the off-response to an input vector a. The on-response is represented by a and the off response by ac where ac = 1 − a (4.14) The complement coded F0 Î F1 input I is the 2M dimensional fit vector. I = (a, ac ) = (a1 , . . . . . . aM , ac1 , ac2 . . . .acM )

(4.15)

I is a point in the double-sized fuzzy cube [0, 1]2M . I has the same fuzzy structure as does its terms a and ac . The system automatically normalizes a complement coded input because | I | = | ((a, ac )) | m m ai + (M − ai) = i=1

(4.16)

i=1

=M The fuzzy ART initial condition is now replaced due to complement coding and the condition is wj1 (0) = wj2 (0) . . . . = wj2M (0) = 1 (4.17)

Fuzzy ART

39

The foundation of the computational properties of fuzzy ART is due to the close link between fuzzy subsethood and ART choice / search / learning. When the choice parameter α = 0+ , the choice function Tj measures the degree to which wj is a fuzzy subset of I. Tj = S(wj , I) = Degree(wj ⊂ I) The system will select a category J for which wJ is a fuzzy subset of I first if such a category exists. Resonance depends on the degree to which I is a fuzzy subset of wJ by the equations | I ∧ wJ | ≥ ρ and |I| | I ∧ wJ | <ρ |I| when J is such a fuzzy subset choice then the match function value is | wJ | | I ∧ wJ | = |I| |I|

(4.18)

Choosing J to maximize | WJ | among fuzzy subset choices by (4.2), maximizes the chance of resonance in (4.7). If reset occurs for the node that maximizes | WJ | then reset will also occur for all other subset choices. 4.3.10

Geometric Interpretation of Fuzzy ART

Each category is represented as a box in a fuzzy M- cube [0,1]M where M is the number of terms of input fit vector a. Consider an input set that consists of 2 dimensional fit vector a. The complement coding gives I = (a, ac ) = (a1 , a2 , 1 − a1 , 1 − a2 )

(4.19)

The each category j has the geometric shape of a rectangle Rj in the fuzzy cube. A complement coded weight vector wj takes the form. Wj = (uj , vjc )

(4.20)

where uj and vj are 2 dimensional vectors. Vector uj defines one corner of a rectangle Rj and vj defines the opposite corner (note Fig. 4.4). 4.3.11

Fuzzy ART Category Boxes in Fuzzy Cubes

Complement coding form with M = 2. Each weight vector wj has the geometry of a rectangle Rj with corners (uj , vj ) (Fig. 4.4). The size of Rj is (4.21) | Rj |≡| vj − uj | which equals the height plus the width of Rj .

40

Adaptive Resonance Theory Networks

1

vj Rj

uj 0

1 Fig. 4.4. Fuzzy ART Category Boxes

1

avvj

Rj

Rj ⊕ a a 0

a∧ ∧uj 1 Fig. 4.5. Fast Learning

Fuzzy ART

41

During fast learning RJ expands to RJ ⊕ a (Fig. 4.5) which is the smallest rectangle that includes RJ and a if | RJ ⊕ a | 2(1 − ρ) Consider a fast-learn fuzzy ART system with β = 1 in equation (4.11). wJ (new) = I = (a, ac ) where J is an uncommitted node The corners of RJ (new) are a and (ac )c = a Hence RJ (new) is just the point a in the fuzzy cube. Learning increases the size of RJ . This size grows as the size of wJ shrinks during learning. Vigilance ρ picks the maximum size of RJ with | RJ | 2 ≤ 2(1 − ρ). During each fast-learning trial RJ expands to RJ ⊕ a or the minimum rectangle containing RJ and a (Fig. 4.5). The corners of RJ ⊕ a are a ∧ uJ and a ∨ vJ where ∧ denotes fuzzy intersection and ∨ denotes fuzzy union, where (p ∨ q)i = max(pi , qi ). (4.22) By equation (4.21) the size of RJ ⊕ a is | RJ ⊕ a |=| (a ∨ vJ ) − (a ∧ uJ ) |

(4.23)

Before RJ can expand to include a reset chooses another category if | RJ ⊕ a | is too large. With fast learning RJ is the smallest rectangle that encloses all vectors a that have chosen category j without reset. If the dimension of a is M then the box Rj contains the two opposing vertices Λj a and Vj a where the ith component of each vector is (Λj a)i = min{ai : category j has coded a}

(4.24)

(Vj a)i = max{ai : category j has coded a} (Fig. 4.6)

(4.25)

and The size of Rj is

| Rj |=| Vj a − Λj a |

(4.26)

wj = (Λj a, (Vj a)c )

(4.27)

and the weight vector wj is as in (4.20) and (4.21). Thus | wj |= (Λj a)i + [1 − (Vj a)i] = M− | Vj a − Λj a | i

(4.28)

i

and so the size of the box Rj is | Rj |= M− | wj |

(4.29)

| wj |≥ ρM

(4.30)

| Rj |≤ (1 − ρ)M

(4.31)

By (4.8), (4.11) and (4.16) and by (4.29) and (4.30)

42

Adaptive Resonance Theory Networks

a2 1

Wjc4

•

•

•

•

•

• ∧ja X

Wj2

Vja • X

Rj

• •

0 0

Wj1

Wjc3

1

a1

Fig. 4.6. Weight Vector WJ

Inequality (4.31) shows that high vigilance (ρ ∼ = 1) leads to small Rj while low vigilance (ρ ∼ = 0) permits large Rj . If j is uncommitted node then | wj |= 2M by (4.17). So formally | Rj |≡ − M by (4.29). Fuzzy ART fast learning and complement coding gives the jth category rectangle Rj that includes all those fit vectors a in the fuzzy unit square that have activated category j without reset. The weight vector wj equals (Λj a, (Vj a)c ) as seen in Fig. 4.6. 4.3.12

Fuzzy ART Stable Category Learning

A Fuzzy ART system with complement coding, fast learning, and constant vigilance forms categories that converge to limits in response to any sequence of analog or binary input vectors. Category boxes grow in each dimension but never shrink. The size of the box Rj = M− | wj | . where wj is the corresponding weight vector. The term M (1 − ρ) bounds the size | Rj | from above. In the conservative limit one-pass learning obtains that no reset or additional learning occurs on subsequent presentations of any input. If 0 ≤ ρ < 1, the number of categories is bounded even if the number of exemplars in the training set is not bounded. Similar properties hold for the fast-learn slow-recode case except that the system may need repeated presentations of each input before stabilization occurs even in the conservative limit.

Fuzzy ARTMAP

43

4.4 Fuzzy ARTMAP This uses boxes in fuzzy cubes to represent category weights in a supervised learning paradigm. Stability and match tracking allow fuzzy ARTMAP to construct automatically as many categories as it needs to learn any consistent training set to 100 per cent accuracy. It relies on multilayer structures to effect its learning strategies. 4.4.1

Fuzzy ARTMAP and MicroARTMAP

The fuzzy ARTMAP inherently represents acquired knowledge in the form of IF-THEN rules. Large or noisy data sets cause them to generate to many rules [56]. This is known as category proliferation. Category proliferation in fuzzy ARTMAP has been handled in different ways. It can be overcome by rule extraction process after training has been completed. Modifications of the architecture or the training algorithm can be done. Distributed coding to avoid commitment of unnecessary categories can be done. The ART choice and match functions can be made to be discriminant functions. Hence the new architecture proposed as ARTMAP fits in exactly for category reduction in fuzzy ARTMAP. The architecture is given in Fig. 4.7.

wjab

Fuzzy ARTa

F20

Map Field Fab

ρab

ya

wja

Fuzzy ARTb

xab

Wkb

reset

F1a

xa

F0a

Ia = A = (a, ac)

reset inter ART

ρa

yb

F1b

F0b

a

reset xb

Ib = B = (b, bc)

b Fig. 4.7. Fuzzy ARTMAP Architecture

F2b

ρb

44

Adaptive Resonance Theory Networks

4.4.2

MicroARTMAP Algorithm (Supervised Neural Network Architecture)

μARTMAP (Fig. 4.8) is a modification of fuzzy art map and uses two fuzzy ART modules ARTa and ARTb that link together via an inter-ART (reset mechanism) module Fab called a map field. The reset mechanism does not cause category proliferation [80]. The map field forms predictive association between categories and realizes the ARTMAP match tracking rule. Match tracking increases ARTa vigilance parameter ρa in response to a predictive error or mismatch at ARTb . Match tracking reorganizes category structure so that subsequent presentations of the input do not repeat the error. The fuzzy ARTa complement coding preprocessor transforms the Ma – D fuzzy unit vector into the 2 Ma – D fit vector A = (a, ac ) at the ARTa field of Fa0 . The input vector to the ARTa field of Fa1 is A. The input to Fb1 is also the 2 Mb – D fit vector B = (b, bc ). When ARTb disconfirms a prediction of ARTa then map field inhibition induces the match tracking process. Match tracking raises the ARTa vigilance ρa to just above Fa1 Î Fa0 match a | a a ratio |X |A| . This triggers an ART search that leads to activation of either an ART category a that correctly predicts b or to a which is a previously uncommitted ART category node. The inputs to the two fuzzy ART modules are first complement coded. Ia = (a, ac ) for ARTa Ib = (b, bc ) for ARTb

wjab

Fuzzy ARTa

F2a

Map Field Fab

ρab

ya

wja

Fuzzy ARTb

xab

Wkb

reset

F1a

xa

F0a

Ia = A = (a, ac)

reset inter ART

ρa

yb

F1b

F0b

a

reset xb

Ib = B = (b, bc)

b Fig. 4.8. MicroARTMAP Architecture

F2b

ρb

Fuzzy ARTMAP

45

Here a is the stimulus and b is the response. For ARTa xa = (xa1 , xa1 . . . xa2 Ma ) denotes Fa1 output vector. a a ya = (y1a , . . . .yN a ) denotes F2 outputvector and

wja

a a th = (wj1 , . . . .wj2M ARTa weight vector a ) is the j

For ARTb xb = (xb1 , . . . .xb2Mb ) and b yb = (y1b , . . . .yN b ) are

b b th output vectors of fields Fb1 and Fb2 respectively and wkb = (wj1 , . . . wj2m b ) is the k ARTb weight vector. For the map field, xab = (xab , . . . xab Nb ). ab ab Denotes Fab output vector and wjab = (wj1 , . . . wjN b ) denotes the weight vector for th ab the j node to F . The system resets xa , ya , xb , yb , xab to 0 between input presentations.

4.4.3

Map Field Activation

The map field Fab receives input from either or both of the ARTa and ARTb category fields. A chosen Fa2 node J sends input to the map field Fab via the weights wjab . Active Fb2 node K sends input to Fab via one-to-one pathways between Fb2 and Fab . If both ARTa and ARTb are active, Fab remains active only if ARTa predicts the same category as ARTb . The Fab output vector xab obeys the following. ⎧ b y ∧ wjab if Jth Fa2 node is active and Fb2 is active ⎪ ⎪ ⎪ ⎨ wab if Jth Fa node is active and Fb is inactive 2 2 j xab = b a b ⎪ y if F is inactive and F is active ⎪ 2 2 ⎪ ⎩ 0 if Fa2 is inactive and Fb2 is inactive

(4.32)

xab = 0 if yb fails to confirm the map field prediction made by wjab . Such a mismatch even triggers an ARTa search for a better category. 4.4.4

Match Tracking

At the start of an input presentation to ARTa , the vigilance parameter ρa is set to its baseline vigilance parameter ρa . The map field vigilance parameter ρab governs matching between categories in ARTa and ARTb .

46

Adaptive Resonance Theory Networks

If a predictive error occurs, then the match tracking raises ARTa vigilance just enough to trigger a search for a new Fa2 coding node. ARTMAP detects predictive error when | xab |< ρab | yb | Now match tracking raises ρa such that ρa >

| Ia ∧ wJa | | Ia |

A search for a new Fa2 coding node is triggered. This process is performed until an ARTa category is selected that correctly predicts ARTb class, or a new category is committed in ARTb . 4.4.5

Map Field Learning

ab Weights wjk in Fa2 Î Fab paths at first satisfy ab wjk (0) = 1, j = 1, . . . ., Na

k = 1, . . . ., Nb During resonance with ARTa category J active wjab approaches the map field vector xab . The Jth category in ARTa always predicts the same category K in ARTb with fast ab learning and that association is permanent (Thus wjk = 1 for all time). The MicroARTMAP is a modification of fuzzy ARTMAP that includes an inter-ART reset mechanism. This reset mechanism does not raise ARTa vigilance and thus does not cause category proliferation, while the predictive accuracy is guaranteed by an off-line learning stage [52], [80]. ab Weights vjk are introduced with an off-line map field. These store the probability a b of F2 Î F2 relations when the inter-ART reset is disabled in the prediction mode. These weights allow the system to evaluate the predictive entropy of the training set. A vigilance parameter is associated to each category node in ARTa . 4.4.6

Defining H

Given partitions of the input space A into Na sets Aj and output space B into Nb sets Bk , the conditional entropy. H(B/A) denoted by H is given by N a

H=−

j=1

N a

pj

pjk log2 pjk

(4.33)

k=1

Where pj is the probability of occurrence of class Aj and pjk is the conditional probability of Bk assuming Aj N a

hj = −pj

pjk log2 pjk be the contribution to H of set Aj .

k=1

Minimizing the conditional entropy is equivalent to the maximization of the mutual information.

Fuzzy ARTMAP

4.4.7

47

Training of μARTMAP

All the weights are initialized to 1 as all categories are uncommitted. ab = 0, wjk

j = 1, . . . .Na k = 1, . . . .Nb

The starting vigilance is set to 0 to minimize the number of categories. 1. Present the input - output pairs (a, b) 2. When a pattern ‘a’ is presented to ARTa , a category J is selected according to the |I∧w | choice function. Tj = α+|wjj | 3. If it is a newly committed category then ρJ = ρa 4. The reset condition is evaluated as |I∧w | ρJ = |I| j ≥ ρ where ρ ∈ [0, 1] if J node is active. 5. If this condition is not satisfied, this node will be inhibited and new search triggered. 6. Pattern b is presented to ARTb selecting the Kth category 7. Map field activity is given by ⎧ b y + wjab if Jth Fa2 node is active and Fb2 is active ⎪ ⎪ ⎪ ⎨ wab if Jth Fa node is active and Fb is inactive 2 2 j xab = (4.34) b a b ⎪ y if F is inactive and F is active ⎪ 2 2 ⎪ ⎩ 0 if Fa2 is inactive and Fb2 is inactive 4.4.8

Inter ART Reset

After xab the map field activity has been calculated, replacing pj and pjk by xab k if j = J | xab | xab pj = Na | xab | + | wiab |

pjk =

i=1,i=J

pjk =

| |

ab wjk wjab

| |

other wise

pj = | xab | +

wjab Na

i=1,i=J

| wiab |

The contribution to the total entropy of the Jth unit if allowed to learn this pattern, hj is calculated. If hj > hmax , then the category is too entropic and the Jth node in ARTa is inhibited by setting Tj (Ia ) = 0, for the rest of this pattern presentation, its vigilance parameter is not valid. Other categories will be chosen in ARTa until the entropy contribution criterion is met.

48

Adaptive Resonance Theory Networks

When previously uncommitted category is selected say Ji then pJik = 1 while pjk = 0 for k = K and hJ = 0. Then the weights in ARTa and ARTb are updated & also in the map field by wJab = xab . 4.4.9

Offline Evaluation

ab The off-line map field is initialized by Vjk = 0, j = 1, . . . ..Na , k = 1, . . . .Nb after all patterns have been processed and the data are presented again to update these weights. A test on the training data is done and results are stored in weights Vjab . The entropy H is computed and compared to Hmax . If H > Hmax , then mapping defined by μARTMAP is too entropic. The ARTa node J that has maximal contribution to the total entropy. J = arg max hj , j = 1, . . . Na is searched and removed (ie wJa = 1, wJa = 0). The baseline vigilance is set to

ρa =

| wJ | RJ = 1 − a + ρ Ma M

This is done so that RJ ≤ M(1 − ρa ) (the category size is bounded) All the patterns that previously selected the Jth ARTa category are presented and the rest are not presented. This process continues till H < Hmax . 4.4.10

μARTMAP Prediction

th

The J ARTa node is selected so that TJ (I) value is highest and then the node correb sponding to the Kth J ART category is predicted. ab , k = 1, . . . ..Nb }. KJ = arg maxh {wjk

KJ has the most frequent association to node J. 4.4.11

Fast Learning in μARTMAP a

When ρ = 0, hmax = 0, fast learning is assumed [104], [109]. All patterns associated with a given class label will be inside the same ARTa hyper box, which is arbitrarily large. The off-line evaluation will measure the probabilistic overlapping of the created hyper boxes. This is related to the number of patterns that select a different category when inter-ART reset is enabled and when it is disabled. This happens as the inter-ART reset does not raise ARTa vigilance. Overlapping of patterns will be large if H > Hmax . Learning is stopped when H = 0 that is when there is no overlapping in the input space.

Summary

4.4.12

49

Refining a Hyper Box

The hyper box to be refined is deleted and all patterns that previously selected that particular hyper box are presented again. Smaller hyper boxes are forced to cover the same region. Through this batch learning process, large hyper boxes are placed in regions where all patterns have the same class label, while small hyper boxes are placed in the boundaries between classes. Populated exceptions are handled with one large hyper box which is a general rule and a specific rule is represented by one smaller hyper box. 4.4.13

μARTMAP Rules

The μARTMAP rules are extracted from the weights in the form [11], [12], [63]. IF a is cj THEN output is LK (priority Pi )

(4.35)

this means that pattern a selects the jth category and LK is the predicted label. The most general rule that is the category with largest hyper box is evaluated. If the just rule is impure, then μARTMAP dynamically finds some category that augments the mutual information between input and output partitions. When entropy has been reduced sufficiently μARTMAP training algorithms stop [65]. The rules of μARTMAP are constructed to be as general as possible. These general rules are refined with increased specifications.

4.5 Summary The Fuzzy ARTMAP unites two fuzzy ART networks to solve problems of supervised learning and prediction. A minimax learning rule controls ARTMAP category structure. It both minimizes predictive error and maximizes code compression. ARTMAP automatically constructs a minimal number of recognition categories to meet accuracy criteria. An ARTMAP voting strategy improves prediction by training the system several times using different orderings of the input set. Voting assigns confidence estimates to competing predictions given small, noisy or incomplete training sets. The MicroARTMAP is a modification of Fuzzy ARTMAP that includes an inter – ART reset mechanism. This reset mechanism does not cause category proliferation, while the predictive accuracy is guaranteed by an off-line learning stage.

5 Applications of MicroARTMAP

5.1 Introduction The microARTMAP was operated in two modes, both in the training mode and in the testing mode. It was trained to recognize hand written alphabets. The microARTMAP was also augmented with a moment-based feature extractor to recognize hand written characters and was found to perform well for slight variations with increased sample size. The microARTMAP gives promising solutions to practical problems in civil engineering like classification of soil problem, finding the load from yield pattern of a plate and finding earthquake parameters from a given response spectrum.

5.2 Recognition of Handwritten Alphabets by μARTMAP μARTMAP has been tested to recognize hand written upper case letters (in English and Tamil) given in a 7 x 5 matrix form. The input space has 30 patterns in 7 x 5 matrix with each pattern having 10 noisy data. Noise was applied randomly. Handwritten characters were taught first and then noisy data entered. Some of the handwritten characters are given in Fig. 5.1. A total of 300 x 35 input and 300 x 5 outputs are given to the network and the network is trained using program handpgm1.m (see Appendix – A). Data = 30 ( 26 English + 4 Tamil alphabets ) Training is done in 27 steps SIZE (INPUT) = 300 x 35 SIZE (OUT) = 300 x 5 The details of the recognition of alphabets is given below:

Input Alphabet A (exact pattern) O (noisy pattern) R (noisy) Pa (noisy)

Output Step Recognition 00001

23rd

Correct

01111

th

Correct

24

th

Correct

14

th

Correct

00100 11101

15

V.K. David and S. Rajasekaran: Pattern Recog. Using Neural & Funct. Net., SCI 160, pp. 51–71. c Springer-Verlag Berlin Heidelberg 2009 springerlink.com

52

Applications of MicroARTMAP

Fig. 5.1. Sample of Handwritten Characters

Recognition of Handwritten Alphabets by μARTMAP Augmented

53

5.3 Recognition of Handwritten Words by μARTMAP Each handwritten alphabet (26 in English and 4 in Tamil) is given in a 7 x 5 matrix form as input. The input space has 30 patterns totally. There are 10 noisy data for each pattern and the number of samples are 30 x 10 = 300 in 7 x 5 matrix form. The program handpgm2.m (see Appendix – G) has been written to train and to recognize any word input to the network. After training the network, any word given as input (though noisy) is easily recognized by the network. Data = 30 ( 26 English + 4 Tamil alphabets ) Training is done in 27 steps SIZE (INPUT) = 300 x 35 SIZE (OUT) = 300x 5 Total time for training and recognition of word (WORD =0.5313 sec). The word WORD consists of four characters W, O, R, D. The details of the recognition of the word (WORD) is given below:

Input Alphabet W (exact pattern) O (noisy pattern) R (noisy) D (noisy)

Output Step Recognition 10111

23rd

Correct

01111

th

Correct

00100 00100

15 4

th

Correct

4

th

Correct

5.4 Recognition of Handwritten Alphabets by μARTMAP Augmented with Moment-Based Feature Extractor 5.4.1

Introduction

Optical character recognition of hand printed characters is widespread nowadays. Images of pages in text are converted to ASCII character files for editing and combining with other files. Generally these systems operate on typed fonts. The problem of recognition becomes more difficult for hand-printed alphanumeric characters. There are many different styles of hand printing for any given person, and within a given style there are also many variations. Any given person prints rather differently at different times, depending on pen or pencil, the width of the lines, the slight rotation of the paper, the type of paper or digitized pad, and the mood and stress level of the person. For example, there may be a deep loop in a character at one time and not at another time. Moreover the line endings may have unintentional serifs. There are similarities between certain characters. A “2” may appear to be a “Z” or vice versa (Fig. 5.2). An “a” may appear as “9”, or a “h” may appear as “n”. If the recognition features are invariant under rotation then a “d” and “p” will have the same features. Hence some features that are not rotation-invariant would distinguish these characters. Each character is thus treated as a texture and the texture features are

54

Applications of MicroARTMAP

Quarter 1

Quarter 2

c.m. Quarter 3

Quarter 4

Soil Classification

Fig. 5.2. The Character “Z” and Quarters

0.8 0.6 0.4 0.2 0 0

5

10

15

Data No SLNN (Eq)

MICRO ARTMAP

Fig. 5.3. Soil Classification

20

Recognition of Handwritten Alphabets by μARTMAP Augmented

55

extracted. Moment invariants are used as the features for recognition of handwritten alphabets [39], [51]. Images are generally subjected to various geometric disturbances or pattern perturbations. Hence it is very essential that features that are invariant to orientations be used for purposes of recognition or classification. For 2D images, moments have been used to achieve Rotation (R), Dilation (D) and Translation (T) invariants and are shown in the Table 5.1. Herein we use Hu’s moments [51]. The normalized moments η are defined as ((p+q)/2+1)

ηpq = μpq /(μ00

(5.1)

where p + q <= 3, . . . ., and μ is evaluated at (xs , ys ) instead of (x, y). Hu’s moment invariants are given in terms of η as defined in Eq. 5.1. The μ moments have physical meaning. μ00 denotes the mass of the image, μ01 , μ10 represent the first moments of masses about y and x axes respectively. If x and y axes pass through mass centroid μ01 , and μ10 are zero. μ20 , μ02 and μ11 represent horizontal and vertical and diagonal centralness of the mass of the image, μ30 and μ03 represent horizontal and vertical imbalance of the image and μ21 and μ12 represent vertical and horizontal divergence respectively. Hu’s seven moment invariants with respect to R,S,T (rotation, scaling and translation) transformations are obtained in terms of these ten moments. From equation (5.1), constraining p, q such that p + q <= 3 and using the tools of invariant algebra, a set of seven RST invariant features as shown in Table 5.2 can be derived [39], [87]. 5.4.2

Steps Involved in Obtaining Moment Invariants

(i) The first step is to capture the characters in blocks of pixels. (ii) The pixels in the block are thresholded so as to be white (0) and black (1) which is the inversion of the usual binary pixels. (iii) The centre of mass (c.m.) of the black pixels that form the character on a white background is found. The c.m. is a two dimensional vector ( x, y ) as given in the Table 5.1. (iv) ( x, y ) in the pixel block is used as the point about which moments are computed namely μpq . (v) The second order moment μ11 , μ20 and μ02 provide variational moments about the c.m. (vi) The third order moments μ30 , μ03 , μ21 , μ12 provide the measure of skewness of the distributions of black pixel values about c.m. (vii) These features are normalized by dividing each by the appropriate powers of the c.m. (viii) The character block is partitioned into quarter sections above and below and to the left and to the right of the c.m. and they provide a rotation variant set of features that can distinguish between a “d” and a “p”. Those patterns that are similar are classified as belonging to the same class. The images that are perturbed (rotated, scaled or translated) versions of the given nominal

56

Applications of MicroARTMAP Table 5.1. Moment-Based Invariants

φ1

=

η20 + η02,

φ2

=

2 (η20 - η02)2 + 4η11 ,

φ3

=

(η30 - 3η12)2 + (3η21 - η03)2,

φ4

=

(η30 + η12)2 + (η21 + η03)2,

φ5

=

(η30 - 3η12) (η30 + η12) [(η30 + η12)2 - 3(η21 + η03)2] + 3 (η21 - η03) (η21 + η03) + [3(η30 - η12)2 - (η21 + η03)2],

φ6

=

(η20 - η02) [(η30 + η12)2 - (η21 + η03)2] + 4η11 (η30 + η12) (η21 + η03),

φ7

=

(3η21 - η03) (η30 + η12) [(η30 + η12)2 - 3(η21 + η03)2] - (η30 - 3η12) (η21 + η03) [3(η30 + η12)2 - (η21 + η03)2],

μ00 =

M (Mass),

μ10 =

0,

μ01 =

0,

μ20 =

2 ∑ ∑ f(x i , y j ) (x i +

μ02 =

1 ), 12 j=1 i =1 n n 1 2 ∑ ∑ f(x i , y j ) (y j + ) , 12 j=1 i =1 n

n

n

n

μ03 =

3 ∑ ∑ f(x i , y j ) (y j ) ,

μ21 =

2 ∑ ∑ f(x i , y j ) (x i y j ) ,

μ12 =

2 ∑ ∑ f(x i , y j ) (x i y j ) ,

μ11 =

n ∑ ∑ f(x i , y j ) (x i y j ) ,

j=1 i =1

n

n

j=1 i =1 n

n

j=1 i =1 n

n

j=1 i =1

n

n

μ30 =

3 ∑ ∑ f(x i , y j ) (x i ) ,

mpq =

p q ∫ ∫ x y f(x, y) dxdy ,

mpq =

p q ∫ ∫ (x - x) (y - y) f(x, y) dxdy ,

x =

j=1 i =1

m10 m , y = 01 , x s = m 00 m 00

μ 20 , ys = m 00

μ 02 . m 00

Recognition of Handwritten Alphabets by μARTMAP Augmented

57

Table 5.2. Moment Invariants for Alphabets given to μARTMAP A = [ 0.51654 0.15068

0.034829

0.17461

0.018480

0.067780

0.028790;

0.46086 0.12242

0.026619

0.12471

0.96889E-02 0.043607

-0.12143E-16;

0.47823 0.13061

0.027640

0.13883

0.011980

0.050226

-0.71265E-03;

0.52128 0.15567

0.040689

0.18011

0.020236

0.071057

-0.13573E-02];

B = [ 0.50273 0.18440

0.067334

0.17317

0.026059

0.074183

-0.28948E-02

0.47844 0.16473

0.055689

0.14866

0.018808

0.060172

0.00000

0.44624 0.14686

0.048145

0.12159

0.012960

0.046458

-0.42772E-03

0.49340 0.18460

0.070760

0.16567

0.024933

0.071007

0.46357E-02];

C = [ 0.89198 0.69288

0.57509

0.10280E+01

0.11149E+01 0.85092

- 0.13677

0.73162 0.46798

0.31636

0.56895

0.33975

0.38619

0.68783E-02

0.73113 0.46806

0.31670

0.56767

0.33907

0.38586

- 0.014218

0.74328 0.46617

0.30882

0.58817

0.35212

0.39946

- 0.040686];

D = [ 0.63835 0.28736

0.12662

0.35085

0.10284

0.18764

- 0.011993

0.55444 0.21877

0.084470

0.23064

0.044752

0.10758

-0.12884E-02

0.55949 0.21933

0.083267

0.23584

0.045995

0.11016

-0.36919E-02

0.62967 0.28709

0.13147

0.33937

0.099291

0.18144

0.017772];

F = [ 0.58925 0.25688

0.14029

0.27524

0.075056

0.14176

-0.047279

0.47841 0.16286

0.066910

0.14682

0.019810

0.059709

-0.013032

0.48003 0.16505

0.069729

0.14831

0.020752

0.060890

-0.013439

0.49143 0.16920

0.072468

0.15773

0.023202

0.065738

-0.013819];

G = [ 0.68643 0.38530

0.22716

0.45912

0.21001

0.28381

-0.025579

0.59429 0.29012

0.14734

0.29745

0.087413

0.15964

0.010266

0.59201 0.28990

0.14758

0.29474

0.086338

0.15816

0.33665E-02

0.59950 0.28909

0.14408

0.30316

0.088770

0.16250

-0.39003E-02 ];

H = [ 0.56337 0.17981

0.040355

0.22657

0.031431

0.096147

0.62226E-03

0.50091 0.14130

0.026786

0.15916

0.015237

0.059813

0.47250E-04

0.49703 0.13905

0.025998

0.15548

0.014511

0.057957

-0.12143E-16

0.50091 0.14130

0.026786

0.15916

0.015237

0.059813

-0.47250E-04];

58

Applications of MicroARTMAP

pattern are all classified as belonging to a class. The Feature Extractor extracts feature vectors from the pattern before presenting them as “pre-processed” inputs to the MicroARTMAP. The Feature Extractor actually enables the MicroARTMAP to have maximum recognition of the hand written alphabets [86], [87]. The character z and quarters are shown in Fig. 5.2. Weights are initialized to 1 before training. The inputs are normalized by complement coding. Total number of samples 40 x 10 = 400. (26 English alphabets + 4 Tamil alphabets + 10 numbers). For each input handwritten alphabet, moment based feature extractor gives seven characteristics of which the first six are used as the inputs to the microARTMAP (as the seventh one has negligible value). These characteristics are obtained using the program, HPGMomout.m (see Appendix – E). For example if alphabet B has seven characteristics as follows: φ1 φ2 φ3 φ4 φ5 φ6 φ7

0.47844 0.16473 0.055689 0.14866 0.018808 0.060172 -0.28948E-02 ; 0.48329 0.16441 0.054477 0.15207 0.019304 0.061551

0.00000 ;

0.49262 0.16460 0.051448 0.15941 0.020118 0.064518 -0.42772E-03 ; 0.49418 0.16467 0.051349 0.16068 0.020242 0.06503

0.46357E-02;

the seventh characteristic φ7 is neglected and the input to B is given as

B=[0.47844 0.16473 0.055689 0.14866 0.018808 0.060172; 0.48329 0.16441 0.054477 0.15207 0.019304 0.061551; 0.49262 0.16460 0.051448 0.15941 0.020118 0.064518; 0.49418 0.16467 0.051349 0.16068 0.020242 0.06503; 0.48796 0.16446 0.053943 0.15569 0.019676 0.063005; 0.47561 0.16269 0.055137 0.14588 0.018239 0.058735]; For the letter B, the Hu’s moment invariants are calculated using the procedure explained previously. The letter B can be written in many ways. Hu’s moments are given for B written in four ways. Out of seven invariants φ7 is comparatively small and hence need not be taken for pattern recognition. In Table 5.2 Hu’s moments for A, B, C, D, F, G, H written in four various ways are given. Similarly input to A is given as

A=[0.49175 0.13667 0.02946 0.15069 0.013739 0.055703; 0.48652 0.13733 0.029446 0.14694 0.013640 0.054421; 0.50320 0.14848 0.037433 0.16274 0.017352 0.062808; 0.49017 0.14221 0.036228 0.15100 0.014939 0.056930; 0.50320 0.14848 0.037433 0.16274 0.017352 0.062808; 0.47823 0.13061 0.027640 0.13883 0.011980 0.050226];

Recognition of Handwritten Numbers by μARTMAP Using Hamming Distance

59

The following seven parameters are used

input module:

BETA : learning rate (w,v weights)

[0,1]

RHO : vigilance factor

[0,1]

output module: BETA : learning rate (w,v weights)

[0,1]

RHO : vigilance factor control or error: h_max : maximum contribution to entropy

[0,1] [0,infty]

H_max : maximum entropy

[0,infty]

RHOstep: step for incrementing RHO

[0,1]

No of training pairs used for each alphabet = 6 (with noise). The recognition is 100%. Training with correct data takes 30 steps and is shown below. Alphabet Input Steps Elapsed time % of noise

Parameters

A

Noisy

30

0.1100

3%

[1 0 1 1 0 .08 .002]

B

Noisy

30

0.1100

3%

[1 0 1 1 0 .08 .002]

B

Noisy

0

0.0500

3%

[ 1 0 1 1 0 .08 .002]

A

Noisy

0

0.1700

3%

[1 0 0 1 1 0 1 .002]

All the 300 alphabets both noisy as well as noisy free have been recognized with slight perturbations plus or minus 5 per cent from actual value. Some of the inputs to microARTMAP are given in Table 5.2 and the parameter can be referred in Appendix – A.

5.5 Recognition of Handwritten Numbers by μARTMAP Using Hamming Distance Binary inputs are used and the network is trained. The total number of exemplars used is 5 and the sample taken is 100. Hamming distance h = number of mismatched components. h = d2 /4 (where d is the Euclidean distance). After training, input is given as noisy data with a Hamming distance ranging from 1 to 9. The recognition is 100% for slight variations. The readings for input of handwritten numbers with a Hamming distance from 1 to 9 is given in Table 5.3 and network is trained using the program number.m. (Refer Appendix – B).

60

Applications of MicroARTMAP Table 5.3. Recognition of numerals with varying Hamming Distance

Number

Hamming Distance

Time

Recognition Correct

0

8

0.3410

‘’

0

3

0.3510

‘’

1

6

0.1800

‘’

1

9

0.1800

‘’

2

4

0.3110

‘’

2

7

0.1800

‘’

3

5

0.7100

‘’

3

6

0.6600

‘’

4

1

0.4400

‘’

4

8

0.4900

‘’

5

4

0.50

‘’

5

7

0.44

‘’

6

2

0.3300

‘’

6

9

0.500

‘’

7

3

0.4600

‘’

7

4

0.4900

‘’

8

6

0.500

‘’

8

7

0.3300

‘’

9

8

0.4400

‘’

9

7

0.8800

‘’

5.6 Recognition of Alphabets and Numbers Using μARTMAP with Only One Exemplar for Training Using Binary Inputs The recognition is 100% for slight variations. Using Moment Invariant Recognition of noisy data is 93%.

BIS Classification of Soil

61

5.7 Recognition of Alphabets by μARTMAP with Increased Sample Size Size of sample was increased as follows. Sample size = 6000 for English and Tamil handwritten alphabets. The network was trained with 6000 samples for English and Tamil alphabets using the program sample 6000.m. The recognition was 100%. It is noticed that learning steps were reduced considerably. When the network was trained using 8000 samples using the program newsample.m, the learning steps were reduced and time taken was also reduced. For example for the alphabet N the learning steps reduced from 132 steps to 32 steps. The time taken reduced from 1.0310 to 0.2700 seconds. The results for all alphabets learning steps were reduced as well as time factor.

5.8 BIS Classification of Soil The two objectives of soil exploration and classification are to find the suitability of the soil for the construction of different structures like dams, embankments sub grade and wearing surfaces, and secondly, the effect on the fertility of soil when it is irrigated. Soils seldom exist in nature separately as sand, gravel or any other single component, but are usually found as a mixture with varying proportions of particles of different sizes. When the soil consists of various constituents in different proportions, the mixture is given the name of constituent that appears to have a significant influence on its behaviour. Thus sandy clay has most of the properties of clay, but contains a significant amount of sand. The behaviour of soil mass under load depends upon many factors: the properties of the various constituents present in the mass, the degree of density saturation and the environmental conditions. There are many classification systems of soils, from among which we consider the unified soil classification system [85]. In practice, soil classification is determined by comparing data with the existing experimental results. Hence MicroARTMAP is applied to this problem, as existing techniques used by foundation engineers are not accurate and satisfactory. It is necessary for the foundation engineer to classify the site soils for use as a foundation for several reasons. The purpose of soil classification is to arrange various types of soils into groups according to their engineering or agricultural properties and various other characteristics. However, from engineering point of view, the classification may be done with the objective of finding the suitability of the soil for construction of dams, highways or foundations, etc. For general engineering purposes, soils may be classified by the following system; 1. Particle size classification 2. Textural classification 3. The United Soil Classification System (USCS) 4. Highway Research Board (HRB) classification 5. Indian Standards (IS) classification system. The United Soil Classification System (USCS) as shown in Table 5.4 is much used in foundation work. A version of this system has been standardized by ASTM (American Society for Testing and Materials) as D 2487 (in volume 04.08: Soil and Rock: Dimension Stone; Geosynthetics). The standardized version is similar to the original USCS as given by Casagrande [20]

62

Applications of MicroARTMAP Table 5.4. IS Classification of Soil

Color of % Gravel/18 Soil

% sand /82

%Grain Liquid Plastic Particles/84 Limit/50 Limit/34

IS Classification Actual Microart

0.1

0.000

0.975

0.238

0.61

0.647

0.1

0.1

0.2

0.111

0.682

0.5

0.508

0.529

0.1

0.1

0.1

0.000

0.341

0.857

0.728

0.764

0.2

0.2

0.1

0.000

0.329

0.869

0.711

0.735

0.2

0.2

0.2

0.000

0.524

0.678

0.576

0.676

0.3

0.3

0.2

0.000

0.512

0.690

0.576

0.647

0.3

0.3

0.2

0.000

0.548

0.654

0.593

0.676

0.3

0.3

0.2

0.166

0.67

0.5

0.525

0.558

0.1

0.1

0.2

0.000

0.585

0.619

0.61

0.823

0.6

0.6

0.2

0.222

0.682

0.476

0.508

0.529

0.1

0.1

0.1

0.000

0.341

0.857

0.711

0.735

0.2

0.2

0.2

0.000

0.536

0.666

0.576

0.647

0.3

0.3

0.5

0.000

0.597

0.607

0.61

0.823

0.6

0.6

0.2

0.000

0.512

0.690

0.593

0.676

0.3

0.3

0.1

0.000

0.926

0.285

0.627

0.676

0.1

0.1

0.2

0.222

0.658

0.5

0.525

0.529

0.1

0.1

0.1

0.000

0.951

0.261

0.61

0.676

0.1

0.1

0.1

0.000

0.341

0.857

0.728

0.735

0.2

0.2

but with specified percentages of sand or gravel passing specific sieves being used to give the “visual description” of the soil. USCS used Casagrande system and classified the soil using the symbols. GP-poorly graded gravel; GW- well graded gravel; SMsilty sand; SP-poorly graded sand; CL-inorganic clay of low to medium plasticity; CHorganic clays of medium to high plasticity. The Indian standard classification system, ISCS (BIS: Bureau of Indian Standards, 1498) first developed in 1959, was revised in 1970. The revised system is essentially based on USCS with the modification that the

BIS Classification of Soil

63

fine grained soils have been subdivided into three groups (low, medium and high plasticity) [83]. Many experts suggest a visual description supplement to any other method. Rajasekaran et.al [85] have used both back propagation and Sequential Learning Neural Network for soil classification. Fig. 5.9 shows the comparison of values obtained from MicroARTMAP and the ones obtained from using SLNN (Sequential Learning Neural Network proposed by Zhang and Morris [118]). s.c = 210/(1 − e−p ) where (5.2) p = (−0.164 I1 − 2.12 I2 − 3.52 I3 − 0.715 I4 − 10.95 I5 + 8.061 I6 − 3.237) where Ii are the normalized inputs and s.c is the soil classification rounded off value. They used 6 input neurons + 1 bias neuron and one hidden neuron and one output neuron. The learning rate and momentum factor values are chosen as 0.5 and 0.0000001 respectively. SLNN required 10,000 epochs for training and the computer time in Pentium IV computer is 5 minutes whereas the time taken by MicroARTMAP is few seconds only. The BIS classifies the soil based on properties such as color of soil, percentage of gravel, percentage of sand, percentage of fine-grained particles, liquid limit and plastic limit. For input to the network, the following rule is used for the color of the soil; .1 brown; 0.2 - brownish grey; 0.3 - grayish brown; 0.5 - reddish yellow; 0.7 - yellowish red. The classification of the soil is given as 0.1 - clayey soil; 0.2 - clay with medium compressibility; 0.3 - clay of low compressibility; 0.6 - silt with medium compressibility. Table 5.4 shows the six normalized data to the MicroARTMAP viz., colour of soil, percentage of gravel, percentage of grain particles and plastic limit, liquid limit, percentage of sand particles got from IS design lab. It is possible to classify the soil using MicroARTMAP.

=

=

=

plastic Fig. 5.4. Isotropic plate (Clamped)

64

Applications of MicroARTMAP

5.9 Plastification of Clamped Isotropic Plate For many generations, engineers have based the analysis and design of structures on a linear theory of elasticity assuming the material to be isotropic. But, it is common that the elastic analysis is unduly conservative because it fails to take advantage of the ability of many materials to carry stresses above the yield stress. The problem is pertained to the recognition of patterns for the prediction of load from yield patterns of Elastoplastic, clamped plates. Finite element displacement method for elasto-plastic analysis Table 5.5. Moment Invariants for Yield Pattern for Clamped Isotropic Plate

M1

M2

M3

M4

M5

P/P(y)

0.93

0.26

0.321

0.938

0.483

1.4

0.568

0.0874

0.041

0.205

0.016

1.6

0.275

0.627

0.004

0.018

0.0001

1.8

0.228

0.015

0.00395

0.0137

0

2.0

0.935

0.267

0.321

0.938

0.483

1.4

0.596

0.0904

0.041

0.205

0.016

1.6

0.258

0.0217

0.004

0.018

0.0001

1.8

0.228

0.015

0.00375

0.0137

0

2.0

0.938

0.268

0.321

0.938

0.483

1.4

0.586

0.0674

0.041

0.205

0.016

1.6

0.285

0.047

0.0048

0.018

0.0001

1.8

0.228

0.015

0.00395

0.0137

0

2.0

0.939

0.269

0.321

0.938

0.483

1.4

0.576

0.0784

0.041

0.205

0.0116

1.6

0.275

0.0217

0.0214

0.067

0.0031

1.8

0.228

0.015

0.00395

0.0137

0

2.0

0.937

0.269

0.321

0.938

0.483

1.4

0.568

0.0874

0.041

0.205

0.016

1.6

(M1 , M2 , . . . , M5 are the moment invariants normalized value).

Application to Earthquake Engineering

65

of bilinear strain hardening orthotropic plates and shells, assuming elastic unloading was developed by Benjamin Whang [108]. The two basic approaches by Whang in elasto plastic analysis are the initial stiffness approach and tangent stiffness approach, in conjunction with the Huber-Mises yield criterion and the Prandtl Reuss flow rule in accordance with the strain hardening yield function. The formation of plastic zones with respect to the loading is to be represented in the patterns. The MicroARTMAP architecture is trained with the patterns representing the plastic zones and their corresponding loading and tested for its inference capability. Once the yield pattern is known the corresponding load may be calculated. The uniformly loaded isotropic plates with clamped edges are assumed to have the thickness as t = 1.0, the Young’s modulus and the Poisson’s ratio as E = 30000, ν = .3 and σ0 is the yield stress = 30, Ep = 300 being the plastic modulus. Considering the doubly symmetric nature of the patterns, only a quarter of the image is presented to MicroARTMAP for training as shown in Fig. 5.4. A slight (±5 per cent) disturbance from the actual value is made in the original patterns and these patterns are used for inference out of the training. Moment invariants are calculated for all the patterns used as data for training the architecture and are tabulated in Table 5.5. It is seen that MicroARTMAP is able to recognize patterns if the obtained values differ from actual values by ±5% and it is not possible to identify patterns if the noise is too high.

5.10 Application to Earthquake Engineering In the last century, India and other countries have had the world’s greatest earthquakes. More than 60% of the area in India are considered prone to damaging earthquakes. The north-eastern region of the country as well as entire Himalayan belt is susceptible to great earthquakes of magnitude more than 8. Besides these, even peninsular India is prone to damaging earthquakes. The Earthquake of 2001 at Bhuj, Gujarat has been the most damaging earthquake in the last five decades. The M7.9 quake has caused a huge loss of life and property. Housing and safety are the basic human needs. Structures adequately designed for usual loads need not necessarily be safe for earthquake forces. In case of design to earthquake loading, it is not practical and economically viable to design the structures to remain within elastic limits. Cracking of concrete and yielding of steel, which would be considered unacceptable for usual types of loading are depended on to dissipate the seismic energy without collapse for ground shaking that may take place infrequently. It is also necessary to generate artificial accelerograms and study the behaviour of structures due to various types of earthquakes simulated. The problem is pertained to the recognition of patterns for the prediction of Magnitude (M), Site condition (S), Focal depth (H) and Epicentral distance (R) once response spectrum is known. The actual and normalized values of M, R, S and H are given in Table 5.6. Three different spectrums are given in Fig. 5.5, Fig. 5.6, Fig. 5.7, for different M, R, S and H values. The image of the given response spectrum is split into eight - regions using program written in MATLAB. In each region the ratio of the total ON pixels to the total OFF pixels is found. These 8 features are considered for all. (see Fig. 5.8 and Fig. 5.9). These 29 data as shown in Table. 5.7. and corresponding Magnitude,

66

Applications of MicroARTMAP Table 5.6. M, S, H and R for Different Response Spectra S.NO

M (actual)

M (normal)

S (actual)

S (normal)

H (actual)

H (normal)

R (actual)

R (normal)

1

6

0.166667

0

0.166667

1

0.03125

30

0.083333

2

6

0.166667

0

0.166667

1

0.03125

50

0.916667

3

6

0.166667

0

0.166667

30

0.9375

30

0.083333

4

6

0.166667

0

0.166667

30

0.9375

50

0.916667

5

6

0.166667

2

0.833333

1

0.03125

30

0.083333

6

6

0.166667

2

0.833333

1

0.03125

50

0.916667

7

6

0.166667

2

0.833333

30

0.9375

30

0.083333

8

6

0.166667

2

0.833333

30

0.9375

50

0.916667

9

8

0.833333

0

0.166667

1

0.03125

30

0.083333

10

8

0.833333

0

0.166667

1

0.03125

50

0.916667

11

8

0.833333

0

0.166667

30

0.9375

30

0.083333

12

8

0.833333

0

0.166667

30

0.9375

50

0.916667

13

8

0.833333

2

0.833333

1

0.03125

30

0.083333

14

8

0.833333

2

0.833333

1

0.03125

50

0.916667

15

8

0.833333

2

0.833333

30

0.9375

30

0.083333

16

8

0.833333

2

0.833333

30

0.9375

50

0.916667

17

6

0.166667

1

0.5

15.5

0.484375

40

0.5

18

8

0.833333

1

0.5

15.5

0.484375

40

0.5

19

7

0.5

0

0.166667

15.5

0.484375

40

0.5

20

7

0.5

2

0.833333

15.5

0.484375

40

0.5

21

7

0.5

1

0.5

1

0.03125

40

0.5

22

7

0.5

1

0.5

30

0.9375

40

0.5

23

7

0.5

1

0.5

15.5

0.484375

30

0.083333

24

7

0.5

1

0.5

15.5

0.484375

50

0.916667

25

7

0.5

1

0.5

15.5

0.484375

40

0.5

26

7.5

0.666667

0

0.166667

22

0.6875

45

0.708333

27

7.5

0.666667

1

0.5

22

0.6875

35

0.291667

28

7.5

0.666667

2

0.833333

8

0.25

45

0.708333

29

6.5

0.333333

0

0.166667

8

0.25

35

0.291667

Minimum

6

0

1

30

Maximum

8

2

30

50

Min (taken)

5.5

-0.5

0

28

Max (taken)

8.5

2.5

32

52

Application to Earthquake Engineering

67

Table 5.7. Black Pixel Count in the Split Response Spectra

C1

C2

C3

C4

C5

C6

C7

C8

0.6871

0.7419

0.9080

0.9365

0.6886

0.7435

0.9100

0.9385

0.4214

0.7452

0.7420

0.7827

0.4229

0.7468

0.7440

0.7846

0.8214

0.7613

0.8540

0.8038

0.8229

0.7629

0.8560

0.8058

0.5286

0.7694

0.8200

0.8481

0.5300

0.7710

0.8220

0.8500

0.7114

0.7839

0.9640

0.9615

0.7129

0.7855

0.9660

0.9635

0.3100

0.9274

0.8640

0.8538

0.3114

0.9290

0.8660

0.8558

0.4871

0.7597

0.7520

0.8692

0.4886

0.7613

0.7540

0.8712

0.1243

0.7403

0.7680

0.7462

0.1257

0.7419

0.7700

0.7481

0.6357

0.6000

0.4800

0.6077

0.6371

0.6016

0.4820

0.6096

0.4529

0.9677

0.9700

0.7673

0.4543

0.9694

0.9720

0.7692

0.8386

0.8065

0.6300

0.5712

0.8400

0.8081

0.6320

0.5731

0.9971

0.9677

0.9620

0.7038

0.9986

0.9694

0.9640

0.7058

0.9786

0.8645

0.8760

0.9385

0.9800

0.8661

0.8780

0.9404

0.8443

0.7194

0.5880

0.6212

0.8457

0.7210

0.5900

0.6231

0.7214

0.7661

0.6160

0.5577

0.7229

0.7677

0.6180

0.5596

0.6500

0.5387

0.4620

0.5135

0.6514

0.5403

0.4640

0.5154

0.5500

0.5371

0.5740

0.6346

0.5514

0.5387

0.5760

0.6365

0.5157

0.7258

0.6120

0.5423

0.5171

0.7274

0.6140

0.5442

0.5414

0.7258

0.5380

0.5096

0.5429

0.7274

0.5400

0.5115

0.8243

0.6855

0.6740

0.6442

0.8257

0.6871

0.6760

0.6462

0.4557

0.8065

0.6600

0.6808

0.4571

0.8081

0.6620

0.6827

0.7257

0.6226

0.5480

0.6038

0.7271

0.6242

0.5500

0.6058

0.8500

0.6097

0.5860

0.7635

0.8514

0.6113

0.5880

0.7654

0.7186

0.7435

0.6040

0.6096

0.7200

0.7452

0.6060

0.6115

0.1186

0.4758

0.3240

0.2096

0.1200

0.4774

0.3260

0.2115

0.8600

0.9677

0.8660

0.8058

0.8614

0.9694

0.8680

0.8077

0.9943

0.8048

0.7800

0.7904

0.9957

0.8065

0.7820

0.7923

0.4271

0.1435

0.0360

0.1538

0.4286

0.1452

0.0380

0.1558

0.2629

0.5645

0.3720

0.3365

0.2643

0.5661

0.3740

0.3385

68

Applications of MicroARTMAP

C1

4 0 .0 0

Acceleration

3 0 .0 0

2 0 .0 0

1 0 .0 0

0 .0 0 0 .0 0

0 .5 0

1 .0 0

1 .5 0

2 .0 0

T im e

Fig. 5.5. Response spectra for M = 6, S = 0, H = 1, R = 30

C 12

2 0 0 .0 0

Acceleration

1 5 0 .0 0

1 0 0 .0 0

5 0 .0 0

0 .0 0 0 .0 0

0 .5 0

1 .0 0

1 .5 0

T im e

Fig. 5.6. Response Spectra for M = 8, S = 0, H = 30, R = 50

2 .0 0

Application to Earthquake Engineering

69

Acceleration

C28 100 80 60 40 20 0 0

0.5

1

1.5

2

Time Fig. 5.7. Response spectra for M = 7.5, S = 2, H = 8, R = 45

Fig. 5.8. Black and white figure of Fig. 5.5

Site condition, Focal depth and Epicentral distance is given as input to MicroARTMAP. Training is successfully carried out for 29 different patterns. The range of the training patterns is limited to 6.0-8.0 for M ; 30-50(m) for R ; 0, 1, 2 for S (classification by UBC) and 1-30 for H (UBC 1994 - Uniform Building Code). The complete history r(t) of any response quantity can be determined from the accelerogram available during the earthquakes using numerical procedure. However, for

70

Applications of MicroARTMAP

Fig. 5.9. Figure (Fig. 5.8) divided into eight parts Number of black counts in each part is c1 = 4731, c2 = 4881, c3 = 4977, c4 = 4982, c5 = 4732, c6 = 4882, c7 = 4978, c8 = 4983

the design of new structures or the evaluation of existing structures, it is generally sufficient to know only the peak value of response defined by the maximum absolute value. A plot of the peak value of a response quantity as a function of the natural vibration period Tn of the system or a related parameter such as circular frequency ωn or cyclic frequency fn is called response spectrum for that quantity. This is plotted for a single degree of freedom system for a particular damping ratio and several such plots for different values of damping are included to cover the range of damping values encountered in any structure. If the black counts in each sub-region differs from the actual values with ±5% then it is called slight disturbance. Then MicroARTMAP will be able to classify the patterns. If the variation is more than 5% it will not be able to classify accurately. With 5% disturbance, BPN also will not be able to classify patterns correctly. Fig. 5.10 shows the comparison of MicroARTMAP with BPN.

Summary

0.9

0.8

0.8

0.7

0.7 S oil C o n dit io n ( S )

M ag n it u d e ( M )

0.9

0.6 0.5 0.4 0.3

0.4 0.3

5

10

15 Data No

20

25

30

Microartmap BPN

0.8 0.6 0.4 0.2

0

5

10

15 Data No

20

25

Microartmap BPN

0.1

Epicen tral Distanc e (R)

Focal Depth (H)

0.5

0

0

1

0

0.6

0.2

Microartmap BPN

0.2 0.1

71

30

0

5

10

15 Data No

20

25

30

Microartmap BPN

1 0.8 0.6 0.4 0.2 0 0

5

10

15 Data No

20

25

30

Fig. 5.10. Comparison of MICROARTMAP with BPN

When the same problem is presented to BPN (available in tool box of MATLAB), it requires 2800 epochs for convergence. Fig 5.10 shows the comparison of M, S, H and R obtained by MicroARTMAP and BPN. MicroARTMAP does not take much time for training as BPN does. A slight disturbance is made in the original patterns and these patterns are used for inference out of the training. It is seen that MicroARTMAP is able to recognize patterns if they are slightly disturbed and it is not possible to identify patterns if the noise is too high. It is possible to classify the Magnitude, Site condition, Focal depth and Epicentral distance of any given response spectrum using MicroARTMAP.

5.11 Summary The MicroARTMAP recognizes handwritten alphabets. Three examples from engineering viz. : soil classification, identification of load from yield patterns of the plate and prediction of earthquake parameters from response spectrum are considered and it is seen that MicroARTMAP can be applied for classification problems.

6 Wavelet Transforms and MicroARTMAP

6.1 Introduction Wavelet transform is a new way of evaluating and processing signals. Wavelet transform can be defined as “a tool that cuts up data functions or operates into different frequency components and then studies each component with a resolution matched to its scale”. Wavelet literally means “small wave”. The field includes work by Ingrid Daubechies (1996), Y. Meyer (1993), Stephane Mallat (1998), Ronald Coifman and Victor Wickerhauser [26], [27], [28], [29], [30], [70].

6.2 The Need for Transforms Mathematical transformations are applied to signals to obtain further information from a signal that is not readily available in the raw signal. A number of transformations can be applied and the Fourier transform [7] is the most popular one. Mostly signals are TIME-DOMAIN signals in their raw format. The time domain signal is considered as a raw signal and a signal that has been transformed by any available mathematical transformation as a processed signal. When the signal is plotted, one of the axis is time (independent variable) and the other is the amplitude (dependent variable). When the time-domain signals are plotted, a time amplitude representation of the signal is obtained. This representation is not the best one. In most signal processing related applications, the most distinguished information is hidden in the frequency content of the signal. Frequency denotes the number of cycles in unit of time.

6.3 Fourier Transform Frequency is measured in ‘Hertz’. Frequency spectrum separates frequency components and thereby make them measurable. If the Fourier Transform of a signal in time domain is taken, the frequency-amplitude representation of that signal is obtained [7]. The plot tells how much of each frequency exists in that signal. The frequency axis starts from V.K. David and S. Rajasekaran: Pattern Recog. Using Neural & Funct. Net., SCI 160, pp. 73–91. c Springer-Verlag Berlin Heidelberg 2009 springerlink.com

74

Wavelet Transforms and MicroARTMAP

zero and goes up to infinity. If the Fourier Transform of the electric current in our house is considered, there will be one spike at 50 Hz and nothing elsewhere as the signal has only 50 Hz frequency component.

6.4 Transforms Available Though, Fourier Transforms, being the popular one applied, many other transforms are also available like Hilbert Transform, Short-Time Fourier Transform, Wigner distributions and the Radon Transforms. In a time domain signal no frequency information is available and in a Fourier Transformed signal no time information is available. Thus Fourier Transform is not suitable for non-stationary signals. To overcome resolution related problems, the wavelet transforms are used [68], [95].

6.5 Wavelet Transforms A wavelet transform [6], [8], [25], [53] is capable of providing the time and frequency information simultaneously, hence giving a time-frequency representation of a given signal. In wavelet transforms, the time-domain signal is passed through various high pass and low pass filters and the transform in turn filters out either high frequency or low frequency portions of the signal. This procedure is repeated and every time some portion of the signal corresponding to some frequency is being removed from the signal. In the first stage, a signal with frequencies upto say 1000 Hz is split up into two parts by passing the signal from a high pass and a low pass filter (filters satisfy the admissibility condition). Two different versions of the same signal is obtained. The portion corresponding to 0 – 500 Hz (low pass portion) and 500 – 1000 Hz (high pass portion). Either portion (usually low pass portion) or both is taken and the process is repeated. This operation is called decomposition (Fig. 6.1). On the assumption that low pass portion is taken, we have three sets of data each corresponding to the same signal at frequencies 0 – 250 Hz, 250 – 500 Hz, 500 – 1000 Hz. Again the low pass portion is taken and passed through low and high pass filters and four sets of signals corresponding to 0 – 125 Hz, 125 – 250 Hz, 250 – 500 Hz and 500 – 1000 Hz are obtained. The signal is continuously decomposed upto a certain pre-defined level. A bunch of signals actually representing the same signal but all corresponding to different frequency bands is got. When all of them are put together and plotted on a 3-D graph, with time in one axis, frequency in the second axis and amplitude in the third axis, the frequencies that exist at different times are obtained. It is not exactly known as to what frequency exists at what time instance but it is known as to what frequency band exists at what time interval. Wavelets comprise a family of basis functions that are capable of describing signals in a localized time and frequency format. The sets of basis functions have compact support (they have all their energy localized to a finite space in time). In wavelet analysis the basic wavelet function called the mother wavelet or generating wavelet or prototype wavelet is shifted and scaled to generate a family of orthonormal basis function.

Discrete Wavelet Transforms (DWT)

75

Fig. 6.1. Decomposition of a Signal

6.6 Continuous Wavelet Transforms (CWT) Continuous wavelet transform was developed as an alternative approach to the short time Fourier transforms to overcome the resolution problem. The signal is multiplied with a function (that is the wavelet) and the transform is computed separately for different segments of the time domain signal [69], [74], [75].

6.7 Discrete Wavelet Transforms (DWT) The main idea in Discrete Wavelet Transforms (DWT) is the same as in the CWT. A time-scale representation of a digital signal is obtained using digital filtering techniques.

76

Wavelet Transforms and MicroARTMAP

Fig. 6.2. The DWT and CWT of a Signal

CWT is a correlation between a wavelet at different scales and the signal with the scale (or the frequency) being used as a measure of similarity. CWT is computed by changing the scale of the analysis window, shifting the window in time, multiplying by the signal and integrating over all times. In the discrete case, filters of different cut off frequencies are used to analyze the signal at different scales. The signal is passed through a series of high pass filters to analyze the high frequencies and it is passed through a series of low pass filters to analyze the low frequencies. By the filtering operations, the resolution of the signal is changed and the scale is changed by upsampling and downsampling (subsampling) operations. Subsampling a signal corresponds to reducing the sampling rate, or removing some of the samples of the signal. Subsampling by two refers to dropping every other sample of the signal. Subsampling by a factor ‘n’ reduces the number of samples in the signal ‘n’ times. If the sampling rate of a signal is increased by adding new samples to the signal, then it is called upsampling. Upsampling by two refers to adding a new sample, usually a zero or an interpolated value, between every two samples of the signal. Upsampling a signal by a factor of ‘n’ increases the number of samples in the signal by a factor of ‘n’. DWT coefficients are usually sampled from the CWT on a dyadic grid (Fig. 6.2) in which X axis represents frequency and Y axis represents the amplitude.

Wavelet Analysis

77

6.8 Wavelet Functions Two important properties namely oscillation and short duration are associated with a wavelet function. ψ (x) is a wavelet function if and only if its Fourier transforms ψ (u) satisfies the following condition +∞ ψ (u)du = 0 (6.1) −∞

Thus the wavelet is an oscillating function with zero mean value. It is required that the wavelet be concentrated in a limited interval (the wavelet has compact support).

6.9 Wavelet Analysis The wavelet analysis starts by selecting a basic wavelet function that can be a function of space x or time t. The basic wavelet function called the mother wavelet is dilated by ‘a’ and translated in space by ‘b’ to generate a set of basis functions as follows: x−b 1 ψ a,b (x) = √ ψ (6.2) a a This function is centered at ‘b’ and the spread is proportional to ‘a’. Wavelet transform correlates the function f(x) with ψ a,b (x). Continuous wavelet transform (CWT) is the sum over all the time of the signal multiplied by a scaled and shifted version of the mother wavelet. +∞ C(a, b) = f(x)ψ a,b (x)dx (6.3) −∞

In CWT, a large number of wavelet coefficients C (a, b) are generated during analysis. CWT is highly redundant and it is not necessary to use the full domain to reconstruct. Instead of continuous dilation and translation, discrete values of the parameters are used. Dilation is defined as a = 2j and the translation parameter as b = k 2j . This sampling of co ordinates is called as dyadic sampling as consecutive values of discrete scales differ by a factor of 2. Using a dyadic scale, discrete wavelet transform can be defined as +∞ j j Cj,k = 2− 2 f(x)ψ (2− 2 x − k)dx

−∞

+∞

= −∞

f(x)ψ j,k (x)dx

(6.4)

When the wavelet transform is only available for a small scale of a < a0 , and has to be recovered, the information has to be complemented for a > a0 . In order to obtain this information, a scaling function is introduced. +∞ D(a0 , b) = f(x)φa0 ,b (x)dx (6.5) −∞

78

Wavelet Transforms and MicroARTMAP

Scaling function does not exist for all wavelets. If the dyadic scale for ‘a’ and ‘b’ is applied at a level N, one set of coefficients called the detailed coefficients are got as +∞ C DN (k) = f(x)ψ N,k (x)dx (6.6) −∞

When the scaling function is used, a set of coefficients called approximate coefficients are got as +∞ C AN (k) = f(x)φN,k (x)dx (6.7) −∞

The discrete version of the reconstructed signal is given by ∞ ∞ N C DN (k)ψ j,k (x) + C AN (k)φj,k (x) f(x) = j=−∞

k=−∞

(6.8)

k=−∞

Thus the above equation shows that the original function can be expressed as the sum of its approximation at Nth level plus the sum of all the details upto that level. Dj (x) (6.9) f(x) = AN (x) + j
6.10 Schematic Representation of the Working of a Wavelet The working of a wavelet can be schematically represented in tree mode as in Fig. 6.3. Procedure The signal is first passed through a half band digital low pass filter with impulse response h[n]. Filtering a signal corresponds to the mathematical operation of convolution of the signal with the impulse response of the filter. The convolution operation is defined in discrete time as x[n]∗ h[n] =

+∞

x[k]∗ h[n − k]

(6.10)

k=−∞

where x[n] is an input signal. A half band low pass filter removes all frequencies that are above half of the highest frequency in the signal. The unit of frequency is of particular importance and in discrete signals the frequency is expressed as radians. The sampling frequency of the signal is equal to 2P radians in terms of radial frequency. The highest frequency component that exists in a signal will be P radians, if the signal is sampled at Nyquist’s rate (which is twice the maximum frequency that exists in the signal) which is P radians in the discrete frequency domain. After passing the signal through a half band low pass filter, half of the samples can be eliminated according to Nyquist’s rule and the signal now has the highest frequency

Schematic Representation of the Working of a Wavelet

79

s

d1 a1 d2 a2

d3 a3 a stands for approximate signal d stands for detailed version Fig. 6.3. Tree Mode Analysis of a Signal

of P / 2 radians instead of P radians. Discarding every other sample in this manner will subsample the signal by two and the signal will now have half the number of points. The scale of the signal is doubled. The low pass filtering removes the high frequency information but leaves the scale unchanged. Only subsampling process changes the scale. Resolution is related to the amount of information in the signal and therefore it is affected by the filtering operations. Half band low pass filtering removes half of the frequencies and can be interpreted as losing half of the information. Hence resolution is halved after filtering operation. However the subsampling operation after filtering does not affect the resolution, since removing half of the spectral components from the signal makes half the number of samples redundant. Thus lowpass filtering halves the resolution, but leaves the scale unchanged. The signal is then subsampled by 2 since half of the numbers of samples are redundant. This is represented by y[n] =

+∞

h[k]∗ x[2n − k]

(6.11)

k=−∞

The DWT analyzes the signal at different frequency bands with different resolutions by decomposing the signal into a course approximation and detail information. DWT employs two sets of functions namely scaling functions and wavelet functions which are associated with low pass and high pass filters respectively (Fig. 6.4) (http:// wavelettutorial/rabipolikar/rowan(tutorials)).

80

Wavelet Transforms and MicroARTMAP

Fig. 6.4. Shifting and Scaling

The original signal x(n) is first passed through a half band high pass filter g(n) and a low pass filter h(n). According to Nyquist’s rule, half the samples are eliminated after filtering, since the signal now has a highest frequency of P / 2 radians instead of P radians. The signal can therefore be subsampled by 2, simply discarding every other sample. This constitutes one level of decomposition and can be expressed as yhigh [k] = x[n]∗ g[2k − n] (6.12) n

ylow [k] =

n

x[n]∗ h[2k − n]

(6.13)

Schematic Representation of the Working of a Wavelet

81

f=0~π

x [n]

g [n]

h [n] f = 0 ~ π/2

f = π/2 ~ π 2

2

Level 1 DWT coefficients

g [n]

h [n]

f = π/4 ~ π/2

f = 0 ~ π/4 2

2

Level 2 DWT coefficients

g [n]

h [n]

f = π/8 ~ π/4

f = 0 ~ π/8 2

2

Level 3 DWT coefficients Fig. 6.5. DWT of a signal

where yhigh [k] and ylow [k] are outputs of the high pass and low pass filters, after subsampling by 2. The above procedure called as subband coding can be repeated for further decomposition. At every level, the filtering and subsampling will result in half the number of samples (and hence half the time resolution) and half the frequency band spanned (and hence double the frequency resolution). Fig. 6.5 illustrates this procedure where x[n] is the original signal to be decomposed, h[n] and g[n] are the low pass and high pass filters. The bandwidth of the signal at every level is marked as ‘f’ in the figure. Suppose original signal x[n] has 512 sample points, spanning a frequency band of 0 to P radians, at the first decomposition level the signal is passed through the high pass and low pass filters followed by subsampling by 2. The output of high pass filter has 256 points and spans the frequencies from P / 2 to P radians. These 256 samples constitute first level of DWT coefficients. The output of the low pass filter also has 256 samples, but it spans the other half of the frequency band say the frequencies from 0 to P / 2 radians.

82

Wavelet Transforms and MicroARTMAP

This signal is then passed through the same low pass and high pass filters for further decomposition. The output of the second low pass filter followed by subsampling has 128 samples spanning a frequency band of 0 to P / 4 radians and the output of the second high pass filter followed by subsampling has 128 samples spanning a frequency band of P / 4 to P / 2 radians. The second high pass filtered signal constitutes the second level of DWT coefficients. This process continues till two samples are left. For this specific example considered which is handwritten character recognition there are eight levels of decomposition, each having half the number of samples of the previous level. The DWT of the original signal is obtained by concatenating all coefficients starting from last level of decomposition. The DWT will have the same number of coefficients as the original signal. The frequencies that are prominent in the original signal will appear as high amplitudes in that region of the DWT signal that includes those particular frequencies. The time localization of these frequencies will not be lost.

6.11 Handwritten Characters Recognition Using Wavelet Transforms and MicroARTMAP Calculating wavelet coefficients at every possible scale involves a lot of work and it generates a lot of data. Hence a subset of scales and positions are chosen. The scale and position are chosen based on the powers of 2 called the dyadic scale and position. Such an analysis is obtained from the discrete wavelet transform (DWT). The Mallat algorithm [71] yields a fast wavelet transform – a box into which a signal passes, and out of which wavelet coefficients emerge quickly. The wavelet used is “dbl” for our analysis. The signal in a form of a matrix with 64 inputs for each handwritten alphabet is given and DWT transforms are performed for dbl wavelet. The program wave1.m is used to optimise the input. The multiple level decomposition yields optimised number of inputs to the original signal. Here decomposition is done to the third level and 8 inputs are obtained using dbl wavelet form. The obtained coefficients are given as input to the MicroARTMAP where training is given for the hand written character samples. The compressed inputs are given to MicroARTMAP. The MicroARTMAP is trained with the 8 inputs and is able to recognize hand written characters with slight (± 5 per cent from the actual value) variation of the input (Refer to Appendix B).

6.12 Wavelet Transforms in Two Dimensions The one-dimensional wavelet transforms are easily extended to two-dimensional functions like images. In two-dimensions, a two dimensional scaling function ϕ(x, y) and three two-dimensional wavelets ψ H (x, y), ψ v (x, y) and ψ D (x, y) are required. Each is the product of a one-dimensional scaling function ϕ and corresponding wavelet ψ . The four products that are considered are the separate scaling function ϕ(x, y) = ϕ(x) ϕ(y)

(6.14)

Wavelet Transforms in Two Dimensions

83

and separable “directionally sensitive” wavelets ψ H (x, y) = ψ (x) ϕ(y)

(6.15)

ψ v (x, y) = ϕ(x) ψ (y)

(6.16)

ψ D (x, y) = ψ (x) ψ (y)

(6.17)

These wavelets measure functional variations - intensity or gray level variations for images along different directions [84]. ψ H measures variations along columns for example, horizontal edges. ψ v responds to variations along rows like vertical edges. ψ D corresponds to variations along diagonals. The scaled and translated basis functions are given by ϕj,m,n(x, y) = 2j/2 ϕ(2j x − m, 2j y − n)

(6.18)

ψ ij,m,n(x, y) = 2j/2ψ (2j x − m, 2j y − n); i = {H, V, D}

(6.19)

The index i refers to the values H, V and D. The discrete wavelet transform function f(x, y) of size M x N is M−1 N−1 1 Wϕ (j0 , m, n) = √ f(x, y)ϕj0 ,m,n(x, y) MN x=0 y=0

(6.20)

M−1 N−1 1 f(x, y)ψ ij,m,n(x, y) Wiψ (j, m, n) = √ MN x=0 y=0

(6.21)

and

where i= {H,V,D} where j0 is an arbitrary starting scale and Wϕ (j0 , m, n) coefficients define an approxii mation of f(x, y) at scale j0 . The Wψ (j, m, n) coefficients add horizontal, vertical and diagonal details for scales j ≥ j0 . Usually j0 = 0 and N = M = 2J so that j = 0, 1, 2, ..., J − 1 and m, n = 0, 1, 2, ..., 2j − 1. i Given Wϕ and Wψ the function f(x,y) is obtained via the inverse discrete wavelet transform 1 f(x, y) = √ Wϕ (j0 , m, n)ϕj0 ,m,n (x, y) MN m n 1 i +√ Wψ (j, m, n)ψ ij,m,n (x, y) MN i=H,V,D j=j0 n

(6.22)

84

Wavelet Transforms and MicroARTMAP

6.13 The Two-Dimensional DWT The two-dimensional DWT can be implemented using digital filters and down samplers. The one-dimensional FWT of the rows of f(x, y) is considered followed by the one-dimensional FWT of the resulting columns. The two-dimensional FWT filters the scale j+1 approximation coefficients to construct the scale j approximation and detail coefficients. In two-dimensional case we get 3 sets of detail coefficients – the horizontal, vertical and diagonal details [84], [90].

Wϕ (j, m, n)

Wϕ H(j, m, n)

Wϕ V(j, m, n)

WϕD(j, m, n)

Wϕ (j+1, m, n)

Fig. 6.6. Wavelet transform applied to an image

LL

HL

LH

HH

IMAGE

LL′′

HL′′

LH′′

HH′′

LH

HL

HH

Fig. 6.7. Multiresolution scheme after one and two levels of decomposition

The image f (x, y) is used as the Wϕ (J, m, n) input convolving its rows with hϕ (−n) and hψ (−n) and down sampling its columns, two subimages whose horizontal resolutions reduced by a factor 2 are got. The high pass or detail component characterizes the images’ high frequency information with vertical orientation; the low pass approximation component contains its low-frequency, vertical information [68], [69]. Both subimages are then filtered column wise and down sampled to yield four quarter size output subimages – Wϕ , Wψ H ,Wψ V and Wψ D (Fig. 6.6 and Fig. 6.7), where X axis represents the frequency and Y axis represents the amplitude. Reconstruction of Original Image At each iteration, four scale j approximation and detail subimages are upsampled and convolved with two one-dimensional filters – one operating on the subimages column and the other on its row. Addition of the results yields the scale j+1 approximation and the process is repeated till original image is reconstructed [114].

Recognition of Handwritten Alphabets Using Wavelet Packets and MicroARTMAP

85

6.14 Recognition of Handwritten Alphabets Using Wavelet Packets and MicroARTMAP The hand written alphabet is generally seen to be corrupted uniformly with a form of additive or multiplicative white noise. The general wavelet procedure for denoising the image (suppressing the noise part) is as follows. 1. A wavelet suitable to the problem and the number of levels for decomposition are chosen by trial and error and the level of decomposition is assumed as two as we can get the accurate prediction. The FWT of the noisy image is computed. 2. The detail coefficients are thresholded. They may be hard thresholded by setting to zero the elements whose absolute values are lower than the threshold, or by soft thresholding which involves first setting to zero the elements whose absolute values are lower than the threshold and then scaling the nonzero coefficients towards zero. Soft thresholding eliminates discontinuity. 3. A wavelet reconstruction is performed based on the original approximation coefficients at level J-P and the modified detail coefficients for levels J-1 to J-P. A fast wavelet transform decomposes a function into a series of logarithmically related frequency bands. The low frequencies are grouped into narrow bands, while the high frequencies are grouped into wider bands. In order to have greater control over the partitioning of the time-frequency plane (smaller bands at high frequencies), the FWT is more generalized to yield a flexible decomposition called wavelet packet. This increases the computational complexity from O(M) to O (M log M). 6.14.1

The Decomposition Space Tree

The decomposition of an image is represented as a binary tree (Fig. 6.8). The detailed structure of the initial tree is considered. The appropriate FWT scaling coefficients are linked to the nodes of the tree. The root node is assigned the highest scale approximation coefficients, which are samples of the function itself ; while the leaves inherit the transforms approximation and detail coefficient outputs. The lone intermediate node is a filter bank approximation that is ultimately filtered to become two leaf nodes. The coefficients of each node are weights of a linear expansion that produces a band-limited piece of the root node. Any such piece is an element of a known scaling or wavelet subspace and hence the generating coefficients be replaced by the corresponding subspace. The subspace analysis tree is thus obtained for the given image. In general, a P-scale FWT analysis tree supports P unique decompositions. 6.14.2

Analysis Tree

Analysis tree provide a compact and informative way of representing multiscale wavelet transforms. They are simple to draw, take less space than their corresponding filter and subsampler-based block diagrams and hence are easy to spot valid decompositions. The analysis tree provides an efficient mechanism for representing wavelet packets which are but conventional wavelet transforms in which the details are iteratively filtered. The

86

Wavelet Transforms and MicroARTMAP

Fig. 6.8. The Decomposition Tree

three-scale FWT analysis tree becomes a three-scale wavelet packet tree. The threescale packet tree almost triples the number of decompositions (and associated time frequency tilings) that are available from the three scale FWT tree. The packet based transforms provide improved control of the partitioning of the decomposed function’s spectrum. A P-scale two dimensional wavelet packet tree supports D(P + 1) = [D(P)]4 + 1 unique expansions where D(1) = 1. Thus a three scale tree offers 83,522 possible decompositions. 6.14.3

Finding Optimal Decomposition

As a single wavelet packet tree presents numerous decomposition options, an efficient algorithm for finding optimal decomposition with respect to application specific criteria is highly desirable and a classical entropy based criteria is applicable in many situations. The potential decompositions serve as the starting point for the compression process. A reasonable criteria for selecting a decomposition for the compression of the image is the additive cost function. E(f) =

m,n

|f(m, n)|

(6.23)

Recognition of Handwritten Alphabets Using Wavelet Packets and MicroARTMAP

87

This function measures the entropy or information content of two-dimensional function f. Low entropy values of f(m, n) = 0 provide little to no information whereas high entropy values indicate functions with many non-zero values [65]. 6.14.4

Efficient Algorithm for Minimal Entropy Solutions

1. Compute the entropy of the parent node and the entropy of its four offsprings EA , EH , EV , ED . For a two-dimensional wavelet packet decomposition, the parent is a two dimensional array of approximation or detail coefficients ; the offsprings are the filtered approximation, horizontal, vertical and diagonal details (EA , EH , EV , ED ). 2. If the combined entropy of the offspring is less than the entropy of the parent i.e., EA + EH + EV + ED < EP , include the offspring in the analysis tree. If combined entropy of offspring is greater than or equal to that of the parent, prune the offspring, keeping only the parent. It is a leaf of the optimized analysis tree. 6.14.5

Denoising Using MATLAB for Handwritten Characters

The wavelet tool box of MATLAB6 is used to decompose an image and denoise it. DWT is applied to the hand written characters to separate the horizontal, vertical and

Fig. 6.9. The Best Tree

88

Wavelet Transforms and MicroARTMAP

diagonal details at different levels (upto 3). DWT is applied based on Haar Wavelet at level 2 on the image of alphabet A. Level 2 decomposition is good enough for simple noises (Fig. 6.9). A way to find good thresholds is not so easy and automation of finding these thresholds is very hard to compute. This leads to statistical method to recover the best threshold and it is not sure that the algorithm finds the best one. Hence manual thresholding stays the best way to find the best denoising result. 6.14.6

MicroARTMAP and Wavelet Packets

A variety of wavelets and thresholding methods are available to choose for denoising a particular signal. It is found that for certain kind of signals certain wavelets and thresholding methods maximally remove noise. This is found out by simulation only [98]. The factors influencing the simulation are (i) The sample size (ii) The test function (iii) The values of (Signal standard deviation) The simulation run is repeated 100 times. Thus one can find the performance of different thresholding methods on different kinds of signals that is corrupted with different

Fig. 6.10. Analysed and Denoised Signal

Recognition of Handwritten Alphabets Using Wavelet Packets and MicroARTMAP

89

Fig. 6.11. 2-D Wavelet Selection Upto Level 5

levels of noise. The Symmlet8 wavelet basis of Daubechies [30] and Coiflet3 basis are commonly used wavelet system for simulation experiments in denoising. The image is first scanned and saved as jpeg. Discrete 2-dimensional wavelet packet is considered. The wavelet used is the Haar wavelet with an entropy threshold value of 233 by trial and error [98]. The decomposition tree, initial tree and best tree and wavelet are got (Fig. 6.8 and Fig. 6.9). The 2D wavelet coefficient selection upto level 5 is shown in Fig. 6.11. This image is analyzed, denoised and from the residuals mean, median, mode, standard deviation, mean absolute deviation and median absolute deviation are obtained (Fig. 6.10). For each image of the handwritten alphabet A, the above features are observed and fed as inputs to the MicroARTMAP. For every image, the above inputs are given and the MicroARTMAP is trained to recognize the alphabet with these as inputs. The inputs are given in Table 6.1. The sample size is 100 x 100 and wavelet used is Haar which is simple, fast and memory efficient. Moreover Haar wavelets are exactly reversible without edge effects and upto level 5. Other wavelet transforms like Daubechies, Mexican hat wavelets and Morlet wavelets produce high resolution which are expensive and are characterized by jogged translations. The scanner output type is grayscale for depth 8. The MicroARTMAP is trained with these 6 characteristics namely mean, median, mode, standard deviation, mean absolute deviation and median absolute deviation for a particular character both with noise and without noise. The MicroARTMAP gets trained

90

Wavelet Transforms and MicroARTMAP Table 6.1. Statistical Measures for the Image given as Input to μARTMAP PA=[

496.3 492.6 488 465.7 472.3

506 506 506 494 506

506.2 506.3 506 498.4 506.1

50.94 57.34 64.33 94.02 94.77

0 0 0 6 0

18.22; 24.25; 31.47; 55.41; 55.96

];

459.8 439.2 449.2 438 447

506 506 500 500 500

506 506.1 506.1 498.1 506

98.37 129.7 118.5 126.7 127

0 0 6 6 6

71.78; 101.6; 86.15; 97.01; 85.5

];

X=[

480.5 486.5 474.4 476.8 478.1

506 506 506 506 506

506 506.2 506.1 506.1 506

80.27 71.33 84.45 84.44 79.63

0 0 0 0 0

44.52; 35.2; 52.54; 50.45; 47.29

];

2=[

472.8 479.8 477.6 477.7 476.5

506 506 506 506 506

506 506.2 506.1 505.9 506.1

86.96 76.31 77.6 76.98 78.87

0 0 0 0 0

55.27; 44.57; 47.69; 47.26; 48.86

];

495.4 497.3 495.5 495.7 496

506 506 506 506 506

506.1 506.3 506.5 506.4 506.3

49.05 44.6 48.89 45.71 48.17

0 0 0 0 0

18.9; 15.8; 18.47; 16.87; 26.28

];

491.4 489.9 492.4 491.6 482.1

506 506 506 506 506

506.5 506.5 506.7 506.4 506.3

56.17 60 52.81 56.46 72.27

0 0 0 0 0

26.28; 28.95; 23.42; 25.59; 41.59

];

A=[

500 494.9 494.1 494.4 491.8

510 510 510 510 510

506.5 506.6 506.6 506.6 506.5

48.42 58.09 58.03 56.24 65.5

0 0 0 0 0

28.32; 26.52; 28.13; 27.5; 32.52

];

N=[

495.5 494.6 490.4 494.3 485.2

506 506 506 506 506

506.2 506.4 506.2 506.1 506.2

52.34 51.43 62.12 56.11 75.18

0 0 0 0 0

19.5; 20.3; 27.43; 21.85; 37.58

];

479.5 488.7 495.3 486.3 491.3

494 506 506 506 506

491.5 506.4 506.5 506.1 506.4

60.26 62.11 49.26 65.77 59.95

6 0 0 0 0

27.14; 30.37; 19.32; 33.94; 26.41

];

MA=[

Y=[

Z=[

S=[

Measures used Mean, Median, Mode, Standard Deviation, Median Absolute Deviation, Mean Absolute Deviation.

Summary

91

and is able to recognize the hand written character for slight (±5 per cent from the actual values) variations of these 6 characteristics.

6.15 Summary Wavelet transforms are applied on the handwritten characters. The features are observed and fed as inputs to the MicroARTMAP. The MicroARTMAP is trained to recognize the characters with these inputs.

7 Gesture and Signature Recognition Using MicroARTMAP

7.1 Introduction The development of Artificial Neural Networks started 50 years ago, in order to understand the working of the brain. The microARTMAP, a new neural network architecture and the well-known Back Propagation based Multi-layer Perceptron (MLP) are compared in the context of hand gesture recognition. MicroARTMAP is a neural network paradigm where fuzzy logic is incorporated. Comparative study between BPN and MicroARTMAP is carried out on the basis of learning convergence and recognition accuracy [4], [66], [92], [112].

7.2 Gestures Webster’s dictionary defines gestures as “the use of motion of limbs or body as a means of expression; a movement usually of the body or limbs that expresses or emphasis an idea, sentiment or attitude.” However, in the domain of HCI (Human – Computer Interaction) the notion of gestures is different. In a computer controlled environment one uses the human hand as a manipulator in the human – machine communication (control of computer/machine functions through gestures). Let h(t) ∈ S be a vector that describes the pose of the hands and/or arms and their spatial position within an environment at time t in the parameter space S. A hand gesture is represented by a trajectory in the parameter space S over a suitably defined interval I.

7.3 Gesture Recognition The primary goal of gesture recognition is to create a system which can identify specific human gestures and use them to either convey information or for device control. The keyboard, mouse, light pen, trackball and keypad are used to interact with the computer but lack in speed. Hence vision based user friendly interfaces are used for man-machine V.K. David and S. Rajasekaran: Pattern Recog. Using Neural & Funct. Net., SCI 160, pp. 93–113. c Springer-Verlag Berlin Heidelberg 2009 springerlink.com

94

Gesture and Signature Recognition Using MicroARTMAP

interaction. This can lead to new speedy interfaces that will allow new commands that are not possible with the current input devices [102]. Other categories similar to gesture recognition are voice and hand writing recognition [33].

7.4 Voice Recognition Speech and handwriting recognition systems are similar to gesture recognition systems, as these perform recognition of moving things thereby, leaving a “trajectory” in space and time. By exploring the literature of speech and handwriting recognition, one can study the classification and identification schemes which might aid in developing a gesture recognition system [89]. Typical speech recognition systems match transformed speech against a stored representation. Most systems use some form of spectral representation, such as spectral templates. Speech recognition systems are classified along the following dimensions: Speaker Dependent Versus Independent Whether the system can recognize the speech of many different individuals without training or does it have to be trained for a specific voice? Currently, speaker dependent systems are more accurate as they need not account for large variations in words. Discrete or Continuous Whether the speaker needs to separate individual words by short silences or can the system recognize continuous sentences? Isolated-word recognition systems have a high accuracy rate, as these systems know when each word ends. Vocabulary Size Vocabulary is usually task dependent. Hence a small vocabulary is easier to recognize than a large one. Recognition Rate Commercial products strive for 95% recognition rate, and such results occur only in laboratory environments. Studies have shown that human beings have an individual word recognition rate of 99.2%. State of the art speech recognition systems anyhow have the capacity to understand a large vocabulary. Limitations of Voice Recognition in Controlling a Robot • Complexity in controlling a robot increases if a person gets affected from common cold as the voice changes. • The channel usually preferred is air and hence the frequencies other than voice prevalent in air may tend to control the robots.

Hand Writing Recognition

95

7.5 Hand Writing Recognition Handwriting can be considered as a type of gesture. On-line (also called “real time” or “dynamic”) recognition machines identify handwriting as a user writes. On-line devices have the advantage of capturing the dynamic information of writing, including the number of strokes, the ordering of strokes, and the direction and velocity profile of each stroke. On-line recognition systems are interactive allowing users to correct recognition errors, adapting to the system, and to see the immediate results of an editing command. Most on-line tablets capture writing as a sequence of coordinate points [43]. Recognition is complicated in part, because there are many different ways of generating the same character. For example, the four lines of letter E can be drawn in any order. Handwriting tablets must take into account character blending and merging, which are similar to the continuous speech problem. Moreover different characters may look similar. To overcome these problems, handwriting tablets pre-process the characters, and then perform some type of shape recognition. Preprocessing typically involves properly spacing the characters and filtering out noise from the tablet. The more complicated processing occurs during character recognition. Features based on both static and dynamic character information can be used for recognition. Some systems use binary decision trees to prune possible characters by examining simple features first, such as searching for the dots above the letters “i” and “j”. Other systems create zones, which define the directions a pen point can travel (usually eight), and a character is defined in terms of a connected set of zones. A lookup table or a dictionary is used to classify the characters [40]. Another scheme draws its classification method from signal processing in which curves from unknown forms are matched against prototype characters. They are matched as functions of time or as Fourier coefficients. To reduce errors, an elastic matching scheme (stretching and bending drawn curves) is used. Alternatively, pen strokes can be divided into basic components, which are then connected by rules and matched to characters. This method is called Analysis-by-Synthesis. These systems use dynamic programming methods to match real and modeled strokes. Examination of handwriting tablets reveal that the dynamic features of characters make on-line recognition possible as compared to speech as it is easier to recognize isolated characters. Most systems lag in recognition by more than a second, and the recognition rates are not very high. They reach reported rates of 95% only due to very careful writing. They are best used for filling out forms which have predefined prototypes and set areas for characters [45], [72], [73], [77]. Limitations of Hand Writing Recognition in Controlling a Robot Hand writing analysis is tedious as the robot should first understand the job and then implement it. Hence controlling the robots by means of gestures is preferred. Actions speak better than words when understood in the correct sense and hence hand gestures are used to communicate with the robots.

96

Gesture and Signature Recognition Using MicroARTMAP

7.6 Hand Gestures in HCI (Human – Computer Interaction) Gestures are created by a static hand or body pose, or by a physical motion in two or three dimensions, and can be translated by a computer into either symbolic commands or trajectory motion commands [33], [36], [66], [91], [105]. Visual interpretation of hand/arm movements have a tremendous advantage over other techniques that require the use of mechanical transducers: as it is non-obstructive. There are few restrictions imposed on the user’s movements that may be caused by the weight or discomfort of mechanical devices. Hand gestures are a new mode for HCI. Our hands are used constantly to interact with things: pick them up, move them, transform their shape, or to activate them in some way. In some unconscious way fundamental ideas like ‘stop’, ‘come here’, ‘over there’, ‘no’ and ‘agreed’ are communicated. Hand gesture recognition has various applications in computer games and machinery control (e.g. crane). Computer recognition of hand gestures may provide a more natural-computer interface by allowing people to point or rotate a CAD model by just rotating their hands.

Hand/Arm Movements

Gestures

Manipulative

Unintentional movements

Communicative

Acts

Mimetic

Symbols

Deictic

Referential

Fig. 7.1. Gestural Taxonomy for HCI

Modalizing

Hand Gestures in HCI (Human – Computer Interaction)

97

Gesture Taxonomy for HCI The lack of clear definition of gestures in general raises another issue on taxonomy of gestures. They vary from “autonomous” (gestures that occur independent of speech) from gesticulation (gestures that occur associated with speech). The taxonomy that seems most appropriate for HCI purpose is shown in Fig. 7.1. Hand gestures can be classified in two categories: static and dynamic. A static gesture is a particular hand configuration pose represented by a single image. A dynamic gesture is a moving gesture, represented by a sequence of images. Focus is on the recognition of static images. The gestures for study were acquired through the web camera (Fig. 7.2) and were stored (Fig. 7.3). The gesture images are processed to reduce noise due to illumination defects. The Neural Networks used to recognize these gestures are the Back Propagation Network (BPN) and MicroARTMAP. The output of the two neural networks are then given to the application phase, which is a robotic arm. The robotic arm has two degrees of freedom. Two stepper motors are used to obtain these two degrees of freedom. The three main phases involved are Gesture processing, Neural networks and the Application phase (the Implementation in robotics). PARALLEL PORT

GESTURE

Pin 2-9 (8 lines)

WEB CAMERA

COMPUTER 4 LINES M1

STEPPER MOTOR DRIVER

M2

STEPPER MOTOR DRIVER 4 LINES

Image Processing & Neural Networks NN – Tool Box (MATLAB) ULN2003- Stepper Motor Driver Motors: M1-Rotary motion M2- up & down. Fig. 7.2. Overall Block Diagram

98

Gesture and Signature Recognition Using MicroARTMAP

Gesture 1

Gesture 2

Gesture 3

Gesture 4

Gesture 5 Fig. 7.3. Acquired Gestures

7.7 Gesture Processing Gesture processing involves Gesture Acquisition, Gesture Preprocessing and Feature Extraction [91], [101]. 7.7.1

Gesture Acquisition – Requirements

The entire process of obtaining the images was performed using the experimental setup as described below. • A constant black background. • The illumination parameters of the web camera maintained uniform. • The laboratory room was maintained at constant illumination throughout so that the background and the camera lens were parallel to each other.

Gesture Processing

99

Acquired Gestures The gesture images were acquired by means of a web camera as shown in Fig. 7.3. 7.7.2

Gesture Preprocessing

Image preprocessing seeks to modify and prepare the pixel values to a form that is more suitable for subsequent operations, in other words to clean up noisy images by means of smoothening. The Figs. 7.4(a) to 7.4(d) show the sequence of steps involved in preprocessing. The Fig. 7.4(a) shows the gesture converted to gray scale and smoothening done by means of applying Low Pass Gaussian Filters. Fig. 7.4(b) shows the Fig. 7.4(a) converted to binary image. Fig. 7.4(c) is eroded to remove noise in Fig. 7.4(b). The removal position constraint as shown in Fig 7.4(d) implies that the image can be taken anywhere in the scope of web camera, provided the full image of the gesture is obtained.

(a)

(b)

(c)

(d)

Fig. 7.4. Gesture Preprocessing

7.7.3

Feature Extraction

Feature extraction is regarded as the major step in gesture processing phase. Three different methods for feature extraction were used namely Statistical Approach, Block Processing and Wavelet Approach. 7.7.4

Statistical Approach

In the statistical approach the components of an RGB image were separated and used. Value of mean and standard deviation for the red pixel intensity, the blue pixel intensity and the green pixel intensity of the entire image was determined. The image was converted into an indexed image and the mean & standard deviation were taken as features. Perimeter of the image was also considered as a feature. The following nine parameters were used for training the neural network. The nine parameters used were 1. Mean of red pixel intensity of the original image 2. Mean of green pixel intensity of the original image 3. Mean of blue pixel intensity of the original image

100

Gesture and Signature Recognition Using MicroARTMAP

(a)

(b)

(c)

(d)

(e) Fig. 7.5. Block Processing - 16 Features

4. 5. 6. 7. 8. 9.

Standard deviation of red pixel intensity of the original image Standard deviation of green pixel intensity of the original image Standard deviation of blue pixel intensity of the original image Mean of pixel intensity of the indexed image Standard deviation of pixel intensity of the indexed image Perimeter of the image

Wavelet Approach

7.7.5

101

Block Processing

In block processing the image was split into different regions and features were extracted from those regions. A data set with 16 features were used for processing. The image separated from the background (for example Fig. 7.4(d)) was considered and used for further processing. Referring to Fig. 7.3 it is well understood that the lower half part of every gesture is the same. Hence it is sufficient if the upper half of image is split into 16 regions as shown in Figs. 7.5(a), 7.5(b), 7.5(c), 7.5(d), 7.5(e). In each region, the ratio of total ON pixels to the total OFF pixels is found. These 16 features are fed to the neural network as inputs.

7.8 Wavelet Approach Input signals are preprocessed by a DWT (Discrete Wavelet Transform) extracting information from image in both time and frequency domains. The output signal of the preprocessing module is then fed into an ANN that classifies the transient. The DWT considerably simplifies the input signal of the ANN; it reduces the volume of input data of the ANN without loss of information. This dramatically reduces the training stage in the ANN and increases the overall performance of the digital relay.

Fig. 7.6. Images After Wavelet Transform

0.3679 0.4448 0.4435 0.4359 0.4348

0.4688 0.5402 0.5402 0.5402 0.5357

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

0 0.5604 0 0.6044 0.0238 0.6703 0.0536 0.6845 0.0130 0.6044

0.6964 0.7232 0.7545 0.7571 0.7548

0.0313 0.0333 0.0333 0.0476 0.0495

0.0045 0 0 0 0.0134

0 0 0 0 0

0 0 0 0 0

0.1374 0.1361 0.1939 0.1939 0.1813

0.8061 0.8974 0.8242 0.7806 0.8516

0.8810 0.2747 0.8810 0.2857 0.8631 0.2527 0.8681 0.2381 0.8631 0.2619

0.6205 0.6384 0.6384 0.6134 0.6571

0.1891 0.1364 0.0943 0.1171 0.1154

0.5966 0.6143 0.6741 0.6161 0.5026

0.6718 0.6845 0.6429 0.6051 0.6667

0.2722 0.3247 0.3500 0.4107 0.3247

0 0 0 0 0

0 0 0 0 0

0.3047 0.3238 0.4089 0.4000 0.3776 0.2417 0.2143 0.1520 0.1739 0.1818

0.9853 0.9905 0.9958 0.9792 0.9238 0 0 0 0 0

0.1538 0.1538 0.2143 0.2143 0.2088

0.3913 0.4348 0.4583 0.4509 0.4493

0.5000 0.5582 0.5582 0.5625 0.5603

0.8090 0.8117 0.7967 0.7610 0.8304

0.2094 0.2593 0.2005 0.1531 0.2423

GESTURE 5 0.9810 1.0000 0.9615 0.9286 0.9835

1.0000 0.9957 0.9957 0.9878 1.0000

0.2455 0.2205 0.1641 0.1770 0.1818

0.9922 0.9952 0.9978 0.9888 0.9605

0.2016 0.2450 0.2423 0.2143 0.2500

0.9761 1.0000 0.9808 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000

GESTURE 4

0 0 0 0 0

GESTURE 3

0 0 0 0 0

0.9538 1.0000 0.6615 0.5345 0.9615 1.0000 0.6369 0.5747 1.0000 1.0000 0.6051 0.5744 1.0000 1.0000 0.5774 0.6458 1.0000 1.0000 0.6250 0.5714 0.0381 0.0536 0.0275 0.0429 0.0714

1.0000 1.0000 1.0000 1.0000 1.0000

GESTURE 2

0.5625 0.5929 0.6460 0.6381 0.6481

GESTURE 1 0.0820 0.0804 0.1000 0.0533 0.0190

0.5000 1.0000 0.1208 0.5625 0.9917 0.1083 0.5625 0.9917 0.1083 0.5625 0.9765 0.0583 0.5625 1.0000 0.1250

0.3913 0.4348 0.4583 0.4444 0.4348

1.0000 1.0000 1.0000 1.0000 1.0000

0.9901 1.0000 0.9698 0.9286 0.9918

0.7294 0.7143 0.6676 0.6607 0.7083

0.1518 0.1466 0.1466 0.1034 0.1540

0 0 0 0 0

0.1313 0.1339 0.1530 0.1048 0.0617

Table 7.1. Data Set - Block Processing With 16 Features

0.7667 0.7958 0.7958 0.7708 0.8000

0.7101 0.6863 0.6583 0.6694 0.7333

1.0000 1.0000 1.0000 1.0000 1.0000

1.0000 1.0000 1.0000 1.0000 1.0000

0.2722 0.2727 0.2160 0.2377 0.2394

1.0000 1.0000 1.0000 1.0000 1.0000

0.6000 0.6417 0.6417 0.5917 0.6625

0 0 0 0 0

0.6406 0.6625 0.7250 0.6578 0.6489

0.8462 0.9571 1.0000 1.0000 0.8631 0.9405 1.0000 1.0000 0.8513 0.9590 0.9952 0.9692 0.8615 0.9762 0.9000 0.9619 0.8690 0.9643 1.0000 1.0000

0.8333 1.0000 1.0000 0.8718 0.9091 1.0000 1.0000 0.8833 0.8333 1.0000 1.0000 0.8872 0.9167 1.0000 1.0000 0.8393 0.8606 1.0000 1.0000 0.9111

0.1511 0.1689 0.1689 0.1500 0.1867

0.0725 0.0612 0.0194 0.0290 0.0727

0.8750 0.9200 0.9333 0.9333 1.0000

102 Gesture and Signature Recognition Using MicroARTMAP

Neural Network for Gesture Recognition

7.8.1

103

DWT Selection

Wavelet transforms are fast and efficient means of analyzing images (transient voltage and current signals). The wavelet transform not only decomposes a signal into frequency bands, but provides a non uniform division of the frequency domain(i.e. the wavelet transform uses short windows at high frequencies and long windows for low frequency components). Wavelet analysis deals with the expansion of functions in terms of a set of basis functions (wavelets), which are generated from a mother wavelet by operations of dilations and translations. Converting the position constraint removed image into wavelet domain by applying 2 Dimensional Daubechies (db) wavelet of order 1 the approximate information (cA), vertical information (cV), Horizontal information (cH) and Diagonal information (cD) are obtained as shown in Fig. 7.6. Prior to applying wavelet transform every image is resized to a matrix of 64 x 64. Hence after taking a wavelet transform db1 an image of size 32 x 32 for cA, cV, cH, cD is obtained. cA, cV, cH, cD are split into a matrix of 2 x 2. These 16 features are fed to the neural network for the recognition phase (Table 7.1).

7.9 Neural Network for Gesture Recognition Neural Networks are inspired by the biological nervous systems. As in nature, the network function is determined largely by the connections between elements. We can train a neural network to perform a particular function by adjusting the values of the connections (weights) between elements [34], [44], [81], [94], [97], [100].

Target

Input

Neural Network including connections (called weights) between neurons

Compare Output

Adjust weights Fig. 7.7. Neural Net block diagram

104

Gesture and Signature Recognition Using MicroARTMAP

Neural Networks are adjusted, or trained in such a way that given a particular input a target output is reached. The network is adjusted based on a comparison of the output and the target till the network output matches the target shown in Fig. 7.7. Many input/target pairs are used to train this network. Neural Networks have been trained to perform complex functions in various fields of application like Pattern Recognition, Identification and Classification of Speech and Vision and Control Systems. 7.9.1

Application- Robotics – Robotic Arm Model

The benefit of gesture recognition is the fact that unique gestures can be interpreted to make specific movements. The motor at the bottom is used to rotate the whole setup. It can rotate upto 180 degrees or even 360 degrees, but in order to create an analogy with the arm, its movement is restricted to 90 degrees. The second motor is used to rotate a screw, which can move the arm up and down. The motor at the bottom has to rotate the whole setup and it is given greater delay. The motor on the top has smaller torque but can rotate faster since it is given lesser delay. This is due to the fact that the arm needs to move from one plane to another quite rapidly (Fig. 7.8).

STEPPER MOTOR M1 FOR LINEAR MOTION

STEPPER MOTOR M2 ROTATIONAL MOTION

Fig. 7.8. Robotic Arm Model

Neural Network for Gesture Recognition

B

C

A

105

PLANE B

O

B’

PLANE A C’

A’

O’ Fig. 7.9. Scope of the Movement

A simple code language is used for the robotic arm to move to various positions. The scope of the movement of the arm is shown in Fig. 7.9. The constraints involved in the movement of the robotic arm are: • There are only two planes in which the robotic arm can move via plane A and plane B. • In each plane the robotic arm can move only to three different points 45 degrees apart. These 6 points are represented by 6 different gestures of the palm of which some are as shown in Fig. 7.3. These images were taken in complete experimental setup. Instructions assigned to each gestures (Fig. 7.9) Gesture 1 : Arm move from line O’C’ to line O’B’ Gesture 2 : Arm move from line O’B’ to line O’A’ Gesture 3 : Arm move from line O’A’ to line OA (to upper plane) Gesture 4 : Arm move from line OA to line OB Gesture 5 : Arm move from line OB to line OC In the Fig. 7.3, the number of times the stepper motor 1 has to rotate for the required angle is given by the formula E = R/M Where E = number of times the stepper motor has to be energized R = required angle and M = minimum step angle

106

Gesture and Signature Recognition Using MicroARTMAP

The number of times the stepper motor 2 has to rotate for required angle is given by the formula Basic Length Unit (BLU) = Lead Screw Pitch / Number of revolutions Number of pulses required = (Distance between planes A & B) / BLU • For movement of robotic arm from position 1 to position 2, the coils of stepper motor M1 has to be energized 25 times, M2 must be kept in ideal condition. • For movement of robotic arm from position 2 to position 3, the coils of stepper motor M1 has to be energized again 25 times, M2 must be kept in ideal condition. • For movement of robotic arm from position 3 to position 4, the coils of stepper motor M1 has to be energized 50 times, the coils of M2 must be energized to move from lower plane to upper plane by interpolation method. • For movement of robotic arm from position 4 to position 5, the coils of stepper motor M1 has to be energized 25 times, M2 must be kept in ideal condition. • For movement of robotic arm from position 4 to position 5, the coils of stepper motor M1 has to be energized again 25 times, M2 must be kept in ideal condition.

7.10 Interface Circuit The interface circuit stands as an interface between the PC and the stepper motors. Output of the Neural Network is obtained via the parallel port and given to a 25 pin connector and then given to the stepper motors via the IC ULN 2003 (Fig. 7.10 and Fig. 7.11).

25 DB C O N N E C T O R

S M D

To Stepper Motor M1 for Rotational Motion

S M D

To Stepper Motor M2 for Linear Motion. (Here circular motion is converted to linear

SMD - Stepper Motor Driver Fig. 7.10. Interface Circuit

Block Processing: For 16 Features

PINS 1 2,3,4,5,6,7

FUNCTIONS - No Connection - Input

8

- Ground

9

- Vcc

10,11,12,13,14,15

107

1 2 3 4 5 6 7 8

U L N 2 0 0 3

16 15 14 13 12 11 10 9

- Outputs

Fig. 7.11. Pin Configuration of ULN 2003

7.11 Back Propagation Network The different feature extraction techniques applied are given below 1. Statistical Approach 2. Block Processing with a data set of 16 features 3. Block Processing with a data set of 64 features The discussion of results is based on the success rate which is the number of gestures correctly identified to the total number of gestures. Results were also checked for same person’s gesture and different person’s gestures. Same person’s gestures imply that the test samples were the gestures of the person whom we used for training. Different person’s gestures imply that the test samples were the gestures of persons who were not used for training.

7.12 Statistical Approach As discussed, the total number of features extracted were 9. The statistical approach is done through BPN algorithm. The results obtained are shown in Table 7.2 and Fig. 7.12.

7.13 Block Processing: For 16 Features The position constraint is removed and the image is split into 16 regions. Features obtained are as discussed before and given in Table 7.3 and Fig. 7.13.

108

Gesture and Signature Recognition Using MicroARTMAP

Table 7.2. BPN-Statistical Approach

Input neurons: 9 Hidden neurons: 6 (optimum)

Output neurons: 5

Activation function : Tansigmoidal

Learning rate: 0.1

Gestures

Success rate

Gesture1

1/10

Gesture2

1/10

Gesture3

0/10

Gesture4

0/10

Gesture5

2/10

Fig. 7.12. BPN - Statistical Approach

Block Processing: For 16 Features

109

Table 7.3. BPN-Block Processing with 16 features

Input neurons: 16

Output neurons: 5

Hidden neurons: 6

Activation function : Tansigmoidal

Learning rate: 0.1

Gestures

Same person’s gesture

Different person’s gesture

Gesture1

10/10

5/10

Gesture2

10/10

4/10

Gesture3

8/10

3/10

Gesture4

9/10

5/10

Gesture5

9/10

4/10

Fig. 7.13. BPN - Block Processing

110

Gesture and Signature Recognition Using MicroARTMAP

7.14 Wavelet Approach As discussed the image is transformed to the wavelet domain and features are extracted. The success rate for the wavelet approach based on BPN is shown Table 7.4 and Fig. 7.14. Table 7.4. BPN-Wavelet Approach

Input neurons: 16

Output neurons: 5

Hidden neurons : 6

Activation function : Tansigmoidal

Learning rate :0.1

Gestures

Same person’s gestures

Different person’s gesture

Gesture1

9/10

8/10

Gesture2

10/10

9/10

Gesture3

10/10

5/10

Gesture4

8/10

6/10

Gesture5

10/10

4/10

Fig. 7.14. BPN - Wavelet Approach

MicroARTMAP

111

In BPN algorithm the total number of hidden neurons selected is 6, since it is found that the value is optimum by carrying out various experiments.

7.15 MicroARTMAP Block Processing Based on block processing technique, different person’s gestures success rate are given in Table 7.5. Table 7.5. MicroARTMAP - Block Processing with 16 features

Gestures

Same person’s gesture

Different person’s gesture

Gesture1

10/10

6/10

Gesture2

9/10

8/10

Gesture3

10/10

8/10

Gesture4

9/10

7/10

Gesture5

9/10

6/10

Wavelet Approach Different person’s gestures are best identified when wavelets and MicroARTMAP are used and this is given by the Table 7.6. MicroARTMAP when trained with the high dimensional dataset learns much faster than the BPN. Table 7.6. MicroARTMAP - Wavelet Approach

Gestures

Same person’s gesture

Different person’s gesture

Gesture1

10/10

8/10

Gesture2

9/10

9/10

Gesture3

10/10

9/10

Gesture4

9/10

8/10

Gesture5

9/10

9/10

112

Gesture and Signature Recognition Using MicroARTMAP

Details The resolution of the picture obtained from the web camera was 120 x 160. After resizing the picture to 60 x 30, the total pixels were 180. With 150 pixels about 150 features can be extracted. Having 30 pictures for 5 gestures a 150 x 150 matrix was obtained. When these 22500 features were used to train the microARTMAP it took 0.035 seconds. While using BPN it took 35 seconds. Hence the learning rate of the microARTMAP is efficient when compared to BPN. Applications Nuclear Power Stations and Hazardous Industries Gesture Recognition would be useful to the industry in situations that are dangerous and inaccessible. In Nuclear Power Stations a Robot could perform actions according to the directions that is given from a safe place outside. This application finds its use in mines, caves, underwater explorations and volcanoes. Virtual Reality Games In Virtual reality games the user does gestures which would be recognized by the system and hence the user can play without using the keyboard or a mouse or a joystick. Clickless Mouse The Clickless mouse is one of the applications of Gesture Recognition. The computer can recognize human gestures and act accordingly. Sign language codes can be used to instruct the system to perform certain operations. Master – Slave Tele Robot Operator The Master – Slave Tele Robot operator is an application where a set of Robots can be used in places where the actions of the Slave Robot is determined by the action of the master. For example, when a Master Robot finishes the drilling process and keeps the drilling machine aside, the Slave Robot or a set of Robots can perform the welding. The Master Robot needs to be monitored for its action. Based on the Master Robot’s gestures, the Slave Robots are given instructions. The results show that the MicroARTMAP out-performs its counterpart BPN, in both learning convergence & recognition accuracy.

7.16 Signature Recognition Using MicroARTMAP and Block Processing The signature sample image was split into different regions and features were extracted from those regions. A data set of eight features were obtained for each sample signature making use of the program signnew.m refer Appendix C. The image (Fig. 7.15) is split. In each region, the ratio of total ON pixels to the total OFF pixels is found out. Eight features are obtained and fed as inputs to the MicroARTMAP [5]. The signatures are best identified when wavelets and MicroARTMAP are used. The time taken to train the MicroARTMAP is 0.0130 seconds.

Summary

113

Fig. 7.15. BPN - Wavelet Approach

7.17 Summary The microARTMAP neural network and the Back Propagation Network are compared in the context of hand gesture recognition. The gesture processing involves Gesture Acquisition, Gesture Preprocessing and Feature Extraction. Three different methods for feature extraction are used, namely Statistical Approach, Block Processing and Wavelet Approach. The wavelet transforms used are dbl. Hand signatures are best identified when wavelets and MicroARTMAP were used. A trained MicroARTMAP system translates into a set of IF – THEN rules at any stage of learning, thus proving itself useful in the areas of large databases. They organize, clarify and predict neural and psychological data concerning memory recognition and attention.

8 Solving Scheduling Problems with Competitive Hopfield Neural Networks

8.1 Introduction In various scheduling applications, the Hopfield Neural Network is commonly applied to obtain an optimal solution [48], [50]. A competitive learning rule reduces the network complexity and provides a highly effective means of obtaining this optimal solution. This new technique is called the Competitive Hopfield Neural Network (CHNN) technique [93]. The job schedule problem of a multiprocess on a multiprocessor system that includes both timing as well as resource constraints via a CHNN helps in obtaining a system of schedules [24], [31]. The energy function designed to illustrate the timing and resource constraints is proposed as in [93]. According to CHNN, the scheduling problem is considered as a minimization of an energy function. It is found that the energy change is invariably negative [110], [115].

8.2 The Energy Function The scheduling problem considers N jobs (or processes) and M machines (or processors). The following assumptions were made (i) a job can be segmented and the execution of each segment is preemptive. (ii) different segments of a job cannot be assigned to different machines, implying that no job migration is allowed between machines (iii) the execution time of each job is predetermined. The execution time of each job can be estimated by calculating the machine cycles or employing some heuristic rule method. The constraints imposed are a deadline and an execution time for each job with limited available system resources. The scheduling involves three variables namely jobs, machine and time. They are depicted in the Fig. 8.1. The x axis denotes the job variable, with i representing a specific job ranging from 1 to N, the y axis represents the machine variable and each point j on the axis represents a dedicated machine from 1 to M. The z axis denotes the time variable, with k representing a specific time, which should be less than or equal to T, the deadline of the job. V.K. David and S. Rajasekaran: Pattern Recog. Using Neural & Funct. Net., SCI 160, pp. 115–122. c Springer-Verlag Berlin Heidelberg 2009 springerlink.com

116

Solving Scheduling Problems with Competitive Hopfield Neural Networks

Fig. 8.1. 3-D Hopfield Neural Network

A neuron indicated by the state variable Vijk is defined as representing whether or not a job i is executed on a machine j at a certain time k. The neuron Vijk = 1 denotes that the job i is arranged to execute on machine j at time k = 0 otherwise.

(8.1)

The competitive HNN can be considered a 1 - out - of - N confine rule. This CHNN converges during network evolutions. The energy function of the CHNN is given as follows E=

N M T T M C2 Vijk Vij1 k1 2 i=1 j=1 j =1 k=1

1 j=j1

k1 =1

⎞2 ⎛ N M T C3 ⎝ + Vijk − P⎠ 2 i=1

+

j=1 k=1

N M T C5 Vijk G2ijk H(Gijk ) 2 i=1 j=1 k=1

N M T M F N C6 Vijk Ris Vi1 j1 k Ri1 s + 2 i =1 j =1 s=1 i=1 j=1 k=1

1 i1 =i

1 j1 =j

(8.2)

Simulation Example Case (i)

117

The synaptic interconnection strength, Wxyzijk and bias input θijk are given as Wxyzijk = − C2 δ(x, i)(1 − δ(y, j)) − C3 δ(x, j) − C6 (1 − δ(x, i))(1 − δ(y, j))δ(z, k) Σ Rxs Ris s

where C2 , C3 , C5 and C6 are weighing factors and C5 2 θxyz = −C3 Pi + G H(G) 2 1 if a = b where δ(a, b) = 0 if a = b The total input to the neuron (i, j, k) is obtained on δE = Wxyzijk Vxyz − θijk Netijk = − δVijk x y z

(8.3)

(8.4)

(8.5)

In the CHNN, a competitive winner-take-all rule is imposed to update the neuron states. The neurons on the same column of a dedicated processor at a certain time compete with one another to decide which specific job should be the winning neuron. The neuron that receives the maximum total input is the winning neuron. The output of the winner neuron is set to 1 and the output states of all the other neurons on the same column are set to 0. The update rule of the neuron for the ith column is 1 if Netxjk = Max Netijk i=1 Vxjk = (8.6) 0 otherwise. where N etxjk is the maximum of total neuron input and is equivalent to the dynamic threshold on a McCulloch-Pitts neuron [67].

8.3 Algorithm 1. randomly set the initial neuron states 2. define the synaptic weights Wij and threshold values θj - (storage learning phase) 3. Calculate the total neuron input. Impose the winner-take-all rule to decide the output neuron state (recalling searching phase) 4. Replace the random initial states with the output neuron states obtained in step (3). Repeat the iterations in steps (3) and (4) iterate until no state change occurs in any iteration.

8.4 Simulation Example Case (i) Problem Consider the scheduling problem with the following. Number of jobs = 5, Machines = 2, resources = 4 and time limit = 8.

118

Solving Scheduling Problems with Competitive Hopfield Neural Networks Table 8.1a. Resource requested Matrix

R1

R2

R3

R4

Process 1

1

0

0

0

Process 2

0

1

0

0

Process 3

0

0

1

0

Process 4

0

0

0

1

Process 5

1

0

0

1

Table 8.1b. Timing Constraints Matrix

Time Required

Time limit

Process 1

2

3

Process 2

5

8

Process 3

3

4

Process 4

4

8

Process 5

2

5

The resource requested matrix, timing constraints matrix, initial states and weighting of factor matrix for the problem are given in Table 8.1a, Table 8.1b, Table 8.1c and Table 8.1d. Small scale shared memory multiprocessors are used commonly in an environment where multiprocessors are executed concurrently while sharing processors and other system resources. The approach is to map the problem constraint into the energy function (equation 8.2) of the CHNN. This is done to resolve the multi constraint schedule problem. The energy function is found to work efficiently in investigating certain scheduling problems. The program was terminated for E = 0. Table 8.1d defines the constants for the energy function Eq. 8.2. The simulation involves scheduling 5 processes in two processors. The scheduling results are given in Fig. 8.2. The result was not unique. The time required to perform process 1 was 1 unit, for process 2 it was 6 units and process 5 it was 1 unit in Machine 1.

Example Case (ii)

119

Table 8.1c. Initial States for CHNN Case (i)

1

2

3

4

5

6

7

8

Process 1

0

0

0

0

0

0

0

0

Process 2

0

0

0

0

0

0

0

0

Process 3

0

0

0

0

0

0

0

0

Process 4

0

0

0

0

0

0

0

0

Process 5

0

0

0

0

0

0

0

0

Process 1

1

1

1

1

1

1

1

1

Process 2

1

1

1

1

1

1

1

1

Process 3

1

1

1

1

1

1

1

1

Process 4

1

1

1

1

1

1

1

1

Process 5

1

1

1

1

1

1

1

1

Machine 1

Machine 2

Table 8.1d. Weighting Factor of CHNN

Constants for CHNN C2

C3

C5

C6

4.0

1.0

3.0

1.0

The time required to perform process 2 was 1 unit, process 3 it was 1 unit and process 4 it was 6 units as shown in Fig. 8.2.

8.5 Example Case (ii) Problem Consider the scheduling problem with the following. Jobs = 5, machines = 2, resources = 4, time = 8

120

Solving Scheduling Problems with Competitive Hopfield Neural Networks

Fig. 8.2. Simulation results Case (i)

Fig. 8.3. Simulation results obtained Case (ii)

The previous resource requested matrix, timing constraints matrix (Table 8.1a and Table 8.1b) initial states Table 8.2 and weighting factor matrix Table 8.1d are taken. The simulation involves scheduling 5 processes in two processors. The scheduling results are given in Fig. 8.3. The results were not unique for different simulations. The time taken to perform process 1 was 1 unit and process 2 was 6 units and process 5 was 1 unit in the first machine. The time taken to perform process 2 was 1 unit, process 3 was 4 units and process 4 was 3 units and is shown in Fig. 8.3 by the second machine. An important feature of a scheduling algorithm is its efficiency. The parameter most relevant to the time a neural network takes to find a solution is the number of iteration needed to converge to a solution. The CHNN requires 5-15 iterations to converge to a solution. The number of neurons for the network is NMT (N – Number of jobs ; M – Number of machines and T – The deadline of the job). The computational time for each iteration is equal to the total neurons (NMT) multiplied by the computation time for each neuron proportional to NM. This algorithm has an execution time proportional to 0 (N2 M2 T). Finding the solution using exhaustive search was slow on a desktop

Example Case (ii)

121

Table 8.2. Initial states for CHNN Case (ii)

1

2

3

4

5

6

7

8

Process 1

1

1

1

1

1

1

1

1

Process 2

1

1

1

1

1

1

1

1

Process 3

1

1

1

1

1

1

1

1

Process 4

1

1

1

1

1

1

1

1

Process 5

1

1

1

1

1

1

1

1

Process 1

1

1

1

1

1

1

1

1

Process 2

1

1

1

1

1

1

1

1

Process 3

1

1

1

1

1

1

1

1

Process 4

1

1

1

1

1

1

1

1

Process 5

1

1

1

1

1

1

1

1

Machine 1

Machine 2

PC. Using CHNN, with 5 jobs each job requiring one unit time, the computational time of each neuron is 5 times one state computation (interconnection, net value and other computations). The CHNN consists of 5 x 1 x 5 neurons and 15 iterations, the most required to find the solution. Hence 5 x 5 x 5 x 15 = 1875 state computations exist on the whole for the CHNN algorithm and more time (in minutes) taken for it to converge for the above problems. The solution focuses on the problem of resource utilization for case (i) and case (ii). 1. Feasible schedules can be obtained by randomly assigning the initial states for the scheduling problem. 2. The initial energy value decreases and hence the errors decrease 3. The rate of convergence is initial-state dependent. 4. The synaptic weight matrix though symmetric (Wxyzijk = Wijkxyz ) has a selffeed back interconnection implying Wxyzijk = 0. The network may hence oscillate during network evolution and a solution is not guaranteed. 5. The form θxyz in the energy equation 8.2 has been modified by removing the term Ckt for obtaining the results when the program was terminated for E = 0.

122

Solving Scheduling Problems with Competitive Hopfield Neural Networks

8.6 Summary A Hopfield neural network was used to obtain an optimal solution for scheduling applications. This problem was considered as a minimization of an energy function and the energy change was negative. The energy function from [93] was considered and the term θxyz has been modified by removing the term Ckt and results obtained but were not unique for different run of the program. When a neural network is used to solve a scheduling problem it becomes a pattern recognition problem.

9 Functional Networks

9.1 Introduction Functional networks were introduced by Castillo [22] as a powerful alternative to ANN. Unlike Neural Networks (Adeli and Huang, 1995) [1], Functional Networks use domain knowledge in addition to data knowledge. The network’s initial topology is derived based on the modeling of the properties of the real world. Once this topology is available, functional equations allow one to obtain a much simpler equivalent topology. Although functional networks also can deal with data only, the class of problems where functional networks are most convenient are the classes where the two sources of knowledge both about domain and data are available. In this chapter, functional networks is applied a) for identification of rocks and b) hot extrusion of steel.

9.2 Functional Networks The main property of the Neural Network is its ability to learn from data by using structural and parametric learning methods. In Neural network, learning process is achieved by estimating the connection weights and by minimizing the error function. Functional Networks [22] is a generalization of the Neural Network bringing together domain knowledge and data. There is no restriction of neural function in functional neurons and arbitrary functions are allowed. Another important property of functional network is the possibility of dealing with functional constraints of the model. The functional network uses two types of learning a) structural learning b) parametric learning. In structural learning, the initial topology of the network, based on some properties available to the designer are arrived at and finally a simplification is made using functional equation to get a simpler architecture. In parametric learning, the neuron functions are estimated by considering the combination of shape functions. Functional networks consists of the following elements (Fig. 9.1). 1. Storing units a) One layer of input storing units: This layer contains input data X1 , X2 , X3 etc. b) Intermediate layer units storing intermediate information f4 , f5 . These units evaluates a set of input values, coming from the previous layer and delivers a set of output values to the next layer. c) A layer of output units f6 . V.K. David and S. Rajasekaran: Pattern Recog. Using Neural & Funct. Net., SCI 160, pp. 123–134. c Springer-Verlag Berlin Heidelberg 2009 springerlink.com

124

x1

Functional Networks

x4

f1

x7 f4

x2

f6

f2

x9

x5 f5 x8 x3

f3 x6 Fig. 9.1. Functional Network

2. Layer of Computing Units f1 , f2 , f3 . A neuron in the computing unit evaluates a set of input values coming from a previous layer. 3. A set of Directed Links. The functions are not arbitrary but they are determined by the structure of the network, like x7 = f4 (x4 , x5 , x6 ), as shown in Fig. 9.1. In addition to data, information about other properties of the function, such as associativity, commutativity and invariance are considered in selecting the final network. In a given functional network, neural functions are arbitrary but in neural networks they are either Sigmoidal, linear or radial basis functions. In functional networks, functions (in which weights are incorporated) are learned, and in neural networks, weights are learned. In some functional networks, the learning method leads to global minimum in a single step. Neural networks work well if the input and output data are normalized in the range of 0 to 1 but in Functional networks there is no such restriction. It can be pointed out that neural networks are special cases of functional networks.

9.3 Procedure to Work with Functional Networks The following eight-step procedure is used. Step 1. Step 2. Step 3. Step 4.

Statement of the problem Initial topology Simplification of initial topology using functional equations Arrive at conditions to hold for uniqueness

The Associativity Functional Network

Step 5. Step 6. Step 7. Step 8.

125

Data collection Parametric learning by considering the linear combination of shape functions Model Validation If step 7 is satisfactory the model is ready to be used.

The learning method of functional network consists of obtaining the neural functions based on a set of data D = (Ii , Oi )(i = 1, 2 . . . . . . n). The learning process is based on minimizing the Euclidean Norm of error function given by 1 (Oi − F(i))2 2 n

E=

(9.1)

i=1

The approximate neural function ‘fi (x)’ may be arranged as fi (x) =

m

(9.2)

aij φij (X)

j=1

Where φ are ‘shape functions’ with algebraic expressions (1, x, x2 , . . . . . . . . . xn ) or Trigonometric functions such as (1, sin (x), cos(x), sin(2x), cos(2x), sin(3x), cos(3x)) or exponential functions. The associative optimization function may lead to a system of linear or nonlinear algebraic equations.

9.4 The Associativity Functional Network Assume that for two inputs x1 , x2 the output x3 is given. We can construct functional network as shown in Fig. 9.2 using the functions f1 , f2 , and f3 as fs (xs ) =

ms

(9.3)

asi φsi

i=1

for s = 1, 2 and ms can be any order

X1

f1(X1) +

f3(X3)

X2

f2(X2) Fig. 9.2. Associativity Functional network

f3-1

X3

126

Functional Networks

φsi can be polynomial, trigonometric or exponential or any admissible functions and herein we call them the shape functions. In this example only polynomial expressions are used such as < 1, x, x2 , x3 . . . , >. The function f3 can be expressed as f3 (x3 ) =

2

a3i φ3i

(9.4)

i=1

From the input functions the following is constructed ˆf3 (x3 ) = f1 (x1 ) + f2 (x2 )

(9.5)

Then the error in the jth data is given by ej = f1 (x1j ) + f2 (x2j ). − f3 (x3j )

(9.6)

The error can be written in matrix form as

⎧ ⎫ ⎪ ⎪ a 11 ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ a 12 ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ a13 ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ . . . ⎪ ⎪ ⎪ ⎪ ⎪ ⎨a ⎪ ⎬ 21 ej =< 1, x1j , x21j , . . . 1, x2j , x22j . . . .., −1, −x3j > ⎪ a22 ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ a 23 ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ .. ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ a31 ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩a ⎪ ⎭ 32

(9.7)

or ej =< bj > {a}

(9.8)

The sum of the squares of the error for all the data is given by E=

ndata

ndata

eT j ej =< a > (

j=1

{bj } < bj >){a} =< a > [A]{a}

(9.9)

j=1

To have uniqueness of solution assuming initial values {xk0 } and the function αk , the following must be satisfied. fk (xk0 ) =

mk

aki φki (xk0 ) = αk

(9.10)

i=1

and writing it in matrix form ⎡ ⎤ {φ10 } {0} {0} ⎢ ⎥ < a > ⎣ {0} {φ20 } {0} ⎦ − < α1 α2 α3 >= 0 {0} {0} {φ30 }

(9.11)

Multiple Regression Methods and Functional Networks

127

or (9.12)

< a > [Φ0 ]− < α >= 0 Using the Lagrangian multiplier technique, an augmented function is defined as R = E+ < a > [Φ0 ]{λ}− < λ > {α}

(9.13)

R =< a > [A]{a}+ < a > [Φ0 ]{λ}− < λ > {α}

(9.14)

or R is minimized by considering ∂R = 2[A]{a} + [Φ0 ]{λ} = 0 ∂{a} ∂R = [Φ0 ]T {a} = {α} ∂λ or

2[A] [Φ0 ] [Φ0 ]T [0]

{a} {λ}

=

{0} {α}

(9.15)

(9.16)

or [G]{u} = {v}

(9.17)

It is noted that [G] matrix is symmetric. Once unknowns {u} are solved for any given x1i , x2i one can write ˆf3 (x3i ) = f1 (x1i ) + f2 (x2i ) = a31 + a32 x3i

(9.18)

x3i = (ˆf3 (x3i ) − a31 )/a32

(9.19)

or If higher order functions are assumed for f3 (x3i ), then non-linear equations have to be solved for x3i using the bisection or Newton-Raphson method. This is time consuming and hence for all the problems considered only a first order function has been assumed for f3 (x3i ).

9.5 Multiple Regression Methods and Functional Networks Multiple regression is a method for using data to sort out the predictive value of the competing predictors. Multiple regression uses many independent variables to predict or explain variation in a quantitative dependent variable. It is an extremely useful way to assess the explanatory value of any different, plausible predictors. In many management situations, there are several possible predictors of a result [54]. Multiple regression is used to sort out these plausible predictors to explain variation in the dependent variable [79], [96]. Two examples are considered and the results obtained by both multiple regression method as well as functional network method coincide. Functional networks are used for prediction.

128

Functional Networks

9.6 Rock Identification by Functional Networks Example 1 Rock Parameters The properties of rock mass is influenced either directly or indirectly by a large number of parameters like a) their chemical composition b) the inter - granular structure i.e relative properties of the grain and matrix elements c) the size and shape of the grains d) the nature of the existing deformation in the rock mass which depends on existing geological conditions under the rock mass during formation. The exact dependence of the rock mass property on these various parameters is very difficult to be exactly and explicitly formulated and are more of the site-specific nature and they vary from place to place. Chakravarty and Misra [23] predicted rock strength based on micro properties of rock mass and fractal dimension. In this work nine rock properties for each rock type have been considered. Physical Property a) Specific gravity:- This is the relationship, which the weight of all other substances bears to the weight of same volume of pure cold water. b) Porosity:- The shape, size and the nature of packing of grains of a rock give rise to the property of porosity or development of the pore spacing within a rock. Numerically it is expressed as the ratio between the total volume of pore spaces and the total volume of rock sample. Porosity is an important engineering property in the sense that it accounts for the adsorption value of the stones. In most cases, and also that a high porosity signifies a lesser density, which generally means lesser compressive strength. Strength Property a) Compressive strength:- The compressive strength of a rock depends on a number of factors such as its mode of formation, its composition, texture and structure, its moisture content and extent of weathering it has already suffered. Igneous rocks being crystalline in character, compact and interlocks in texture and uniform in structure possesses very high compressive strength compared to sedimentary and metamorphic rocks. b) Tensile strength:- Rocks are rarely subjected to direct tensile load. In the laboratory transverse tensile strength is determined by performing the bending test. When a stone is intended for use as a beam or a lintel, its transverse tensile strength is determined as modulus of rupture. Generally transverse tensile strength is 1/20 to 1/10 of the compressive strength. c) Cohesion:- Cohesion is the resistance to shear when the rock or stone is not subjected any normal stress and the shear resistance of the rock is given by τ = C + σ tan(φ)

(9.20)

where τ, C, σ and φ denote shear strength, cohesion intercept, normal stress and the angle of internal friction. d) Angle of internal friction:- φ in the above equation.

Rock Identification by Functional Networks

129

Elastic Properties Young’s modulus:- As with other solid materials, modulus of elasticity of rocks indicates their deformation under loads. The deformation is recovered when loads are removed. It is determined in accordance with Hook’s law which states that in elastic substances stress is directly proportional to strain and it is expressed as σ E= (9.21) ε where σ and ε denote stress and strain. The slope of the line drawn from the point of origin or zero load as a tangent to the curve is taken initial tangent modulus and it is commonly referred to as Modulus of Elasticity of the material. Index Properties a) Protodyakonov b) Fractal dimension:- The rock properties, which are the characteristics of the nature of formation, are mostly found to be dependent on the nature of the surface that Table 9.1. Normalized Input and Output for Rock Classification Problem Rock Type

specific porosity compress gravity strength

tensile cohesion internal strength friction

youngs modulus

index Protodyakonov

Fractal dimension

quartz-0.1

2.658

0.2

188.89

8.69

34.5

63

102

20

2.054

granite-0.2

2.76

0.77

169.81

9

32

56

92

20

2.238

sandstone-0.3

2.06

16.87

44.96

4.99

18

42

41.6

3.4

2.48

lime stone(HG)-0.4

2.65

11.23

59.92

6.35

14

46

47.5

8.24

2.564

lime stone(LG)-0.5

2.04

15.52

47.2

5.2

6

40

35

6.3

2.434

shale-0.6

2.15

18.5

48.53

4.64

4

42

12.5

5.54

2.217

Max

2.8

20

200

10

40

70

120

25

2.6

Min

2

0

40

4

0

30

0

0

2

Output

NORMALISED VALUES

(input)

0.1

0.8225

0.01

0.930563

0.781667

0.8625

0.825

0.85

0.80

0.09

0.2

0.95

0.0385

0.811313

0.833333

0.8

0.65

0.766667

0.80

0.396667

0.3

0.075

0.8435

0.031

0.165

0.45

0.3

0.346667

0.136

0.8

0.4

0.8125

0.5615

0.1245

0.391667

0.35

0.4

0.395833

0.3296

0.94

0.5

0.05

0.776

0.045

0.2

0.15

0.25

0.291667

0.252

0.723333

0.6

0.1875

0.925

0.053313

0.106667

0.1

0.3

0.104167

0.2216

0.361667

130

Functional Networks

are produced after rock failure. They were studied by Chakravarty and Misra [23] by Scanning Electron Microscope. The scale invariance affine transformed based statistic namely fractal dimension was calculated for the surfaces of rocks by various methods and in this work fractal dimension calculated using Box counting method is considered. Rock Determination Using Functional Networks Six different rock types considered are a) quartz b) sand stone c) lime stone (High grade) d) lime stone (Low grade) e) granite and f) shale. Experimental results were given by Chakravarty and Misra [23] and are given in Table 9.1. Inputs and outputs are normalized with respect to maximum and minimum values. Functional network (FN) of first order for both input and output data is applied assuming initial parameters as 0.5 and α as 0.2. The rocks are classified as 0.1 - Quartz, 0.2Granite, 0.3- Sand Stone, 0.4- Lime Stone with High Grade, 0.5- Lime Stone with Low Grade, 0.6- Shale. The undetermined parameters are given in Table 9.2. The equation for rock classification obtained from FN is given by rc = 0.05926 I1 + 0.0516 I2 − 0.8008 I3 + 0.0433 I4 − 0.2197 I5 −0.23093 I6 − 0.18087 I7 + 0.9329 I8 − 0.24778 I9 + 0.57197 (9.22) where rc denotes the classification of rock and Ii are the normalized values of inputs. Table 9.2. Undetermined Parameters

a n0

a n1

I1: specific gravity

0.4638

-0.5276

I2: porosity

0.4298

-0.4596

I3: compressive strength

-3.3656

7.130

I4: tensile strength

0.3928

-0.3857

I5:cohesion

-0.778

1.9568

I6:internal friction

-0.8281

2.056

I7:youngs modulus

-0.6051

1.6103

I8:Protodyakonov

4.353

-8.306

I9:fractal dimension

-0.9032

2.206

O1: output - rock type 0.1 - 0.6

4.2517

-8.903

Input/Output

Hot Extrusion of Steel

131

Rock classification is also carried out using multiple linear regression with excel package and the equation obtained is rc = 0.178517 I1 + 0.4988 I2 − 0.4966 I3 + 0.28655 I4 − 0.31427 I5 −0.36344 I6 + 0.202859 I7 + 1.22335 I8 − 0.15413 I9 + 0.066171 (9.23) The correlation co-efficient between values from FN and actual value is 0.972457 and FN and multiple regression is 0.978.

9.7 Hot Extrusion of Steel Example 2 Finite Element Simulation A number of finite element simulations are performed for forward hot extrusion of a preform for transmission shaft with various die angles and punch velocities using the automatic remeshing scheme and using six noded triangular elements. A solid steel cylinder at 1050 deg centigrade is with Dies that are kept at 100 deg centigrade. The material parameters chosen are m = 0.02; n = 0.01; α = 0.2Sp−m . mm−p where m,n and α denote strain rate sensitivity index, strain hardening exponent and material thermal constant [46]. The geometry is axi-symmetric in nature and hence only one half of the part is idealized. The variation of forging force with different die angles and punch velocities along with equivalent strain, equivalent stress and equivalent strain rate are obtained. Simulation The simulated data obtained with Finite Element Analysis are used in Functional Networks. It consists of two inputs namely die angle (x1 ) and punch velocity (x2 ) and the output being forging load (x3 ). All the inputs and outputs are normalized with respect to their maximum values viz: 100, 1400 and 400 respectively and the normalized inputs and outputs are shown in Table 9.3. The initial values (x1 )0 , (x2 )0 , (x3 )0 can be chosen as 0.5 and α1 , α2 , α3 as 0.2. Second order function is applied for inputs and first order function is applied for the output. The undetermined parameters are given in Table 9.4. Using the undetermined parameters, the equation Eq. can be written as (1.53078 − 3.46517 x3 ) = (0.11179 + 1.2938 x1 − 2.2349 x21 −3.4129 + 10.303 x2 − 6.5149 x22 where xi are the normalized values of input and output.

(9.24)

132

Functional Networks Table 9.3. Normalized Inputs and Output

S. No. Die angle/100

Velocity/1400

Forging load/400

Functional

1

9.00E-01

2.86E-01

8.75E-01

8.77E-01

2

9.00E-01

4.29E-01

6.18E-01

6.33E-01

3

9.00E-01

5.71E-01

4.50E-01

4.62E-01

4

9.00E-01

7.14E-01

3.25E-01

3.64E-01

5

9.00E-01

8.57E-01

3.00E-01

3.37E-01

6

7.50E-01

2.86E-01

8.38E-01

7.73E-01

7

7.50E-01

4.29E-01

5.50E-01

5.30E-01

8

7.50E-01

5.71E-01

4.30E-01

3.58E-01

9

7.50E-01

7.14E-01

2.80E-01

2.60E-01

10

7.50E-01

8.57E-01

2.50E-01

2.34E-01

11

6.00E-01

5.71E-01

3.75E-01

2.84E-01

12

4.50E-01

2.86E-01

6.83E-01

6.53E-01

13

4.50E-01

4.29E-01

4.55E-01

4.09E-01

14

4.50E-01

5.71E-01

3.25E-01

2.38E-01

15

4.50E-01

7.14E-01

1.95E-01

1.40E-01

16

4.50E-01

8.57E-01

1.25E-01

1.13E-01

17

3.00E-01

2.86E-01

6.50E-01

6.37E-01

18

3.00E-01

4.29E-01

2.75E-01

3.93E-01

19

3.00E-01

5.71E-01

3.18E-01

2.22E-01

20

3.00E-01

7.14E-01

8.40E-02

1.23E-01

21

3.00E-01

8.57E-01

6.60E-02

9.69E-02

22

1.50E-01

2.86E-01

6.20E-01

6.49E-01

23

1.50E-01

4.29E-01

4.30E-01

4.05E-01

24

1.50E-01

5.71E-01

2.93E-01

2.34E-01

(Correlation co-efficient between functional and actual values = 0.972457).

Hot Extrusion of Steel Table 9.4. Undetermined Parameters

Input/output

a s0

a s1

as2

I1

0.11179

1.2938

-2.2349

I2

-3.4129

10.303

-6.5149

O1

1.53078

-3.46157

1.00 0.90 0.80 0.70 forging load/400 0.60 0.50 0.40 0.30 0.20 0.10 0.00 0

5

10

15

20

data no actual

functional

Fig. 9.3. Comparison of actual vs functional networks

25

133

134

Functional Networks

Eq.(9.23) may be rewritten in a such a way to get the forging load in normalized form (x3 ) in terms of normalized inputs as x3 = 1.3935 − (10.3737 x1 + 2.976 x2 ) + (0.6456 x21 + 1.882 x22 )

(9.25)

Converting the above equation from normalized values to actual values the forging load FL (x3 ) can be given as FL = 558.32 − (1.495 x1 + 0.8502 x2 ) + (0.025824 x22 + 0.000362 x22 )

(9.26)

The prediction of forging load by neural network is compared with actual values in Fig. 9.3 and the accuracy is quite good with a correlation coefficient of 0.972676. To determine the optimum value (minimum) for forging load, the Expression given in Eq. (9.25) has to be differentiated with respect to input parameters as ∂(FL) = −1.4952 + 0.05164 x1 = 0; x1 = 28.945 deg ∂x1 ∂(F L) = −0.8502 + 0.000724 x2 = 0; x2 = 1174.3 mm/sec ∂x2

(9.27)

With the above values the minimum forging load is given by 37.4 tonnes as against 26.4 obtained for 30 deg die angle and 1200mm/sec punch velocity given in Table 9.3. This may be due to error in the data in Table 9.3 i.e. forging load for 30 deg die angle and 800 velocity is given as 127 but it should be less than 110. Similarly functional network may be applied to determine the equivalent strain, stress and strain rate with respect to die angle and velocity.

9.8 Summary Functional Networks introduced by Castillo et al. [21], [22] proved to be a powerful alternative to standard Neural Networks. Neural Functions are learned in Functional networks rather than weights. Regression approximately maps a set of given exemplar input points into a given set of output points. This association process is equivalent to recognition. When a neural network is trained and tested on a good set of exemplar data that is sufficiently large to determine a solution set of weights then it becomes a valid nonlinear model of a system that maps the given inputs to the desired outputs.

10 Conclusions and Suggestions for Future Work

10.1 Conclusions In this work, four specific Neural Network architectures viz., (i) Kohonen’s Self Organizing Map (SOM), (ii) MicroARTMAP and (iii) Competitive Hopfield Neural Network (CHNN), (iv) The Back Propagation Network (BPN) and Functional Networks (FN) have been applied to different problems. Compared to BPN , functional networks are easy to apply to practical problems such as rock identification and hot extrusion of steel. Classification of soil is done with the help of MicroARTMAP and the results are compared with those obtained by SLNN. For response spectrum MicroARTMAP is applied to obtain M,S,H,R and MicroARTMAP takes less time for computation when compared with BPN. Wavelet transforms have also been used to compress data. 1. SOM is found to be useful in recognizing English as well as Tamil alphabets. 2. MicroARTMAP performs fast, stable learning in supervised settings when it is used to solve category proliferation problems. 3. It is also seen that the MicroARTMAP recognizes hand written alphabets (English and Tamil) and also words. 4. When MicroARTMAP is augmented with Hu’s moment based feature extractor, hand written alphabets and numbers are recognized even if the characters are translated, scaled or rotated. 5. It is also seen how wavelet transforms have been used for compression of data. Due to less number of inputs MicroARTMAP, takes less time for recognition. 6. MicroARTMAP can effectively be applied to area of civil engineering and is seen to perform effectively in soil classification, earthquake parameter identification and to obtain the ultimate load capacity of plates. 7. A simpler approach like “Block Processing” could be applied along with MicroARTMAP for signature recognition. 8. After training the MicroARTMAP network using 36 alpha numeric exemplar patterns, the network was able to achieve 100 per cent recognition rate. The same recognition rate was achieved with one pixel shifted in right directions (up, down, left, right, up left, up-right, down-right and down-left). An average recognition rate V.K. David and S. Rajasekaran: Pattern Recog. Using Neural & Funct. Net., SCI 160, pp. 135–136. c Springer-Verlag Berlin Heidelberg 2009 springerlink.com

136

Conclusions and Suggestions for Future Work

of about 92 per cent was achieved with two pixels shifted in eight directions. The recognition rate of the two pixels shift cases became about 98.6 per cent after training with 36 exemplar patterns and 360 distorted patterns. The one pixel shift case stayed at 100 per cent recognition rate. 9. Complement coding is an alternative normalization rule. This has the advantage of preserving amplitude information. This uses on cell and off-cell pairs to normalize input vectors. Inputs preprocessed into complement coding form are automatically normalized. Hence proliferation of categories are avoided in MicroARTMAP. 10. The CHNN was used to solve the scheduling problem. The energy function proposed converged and was negative. 11. The functional networks can replace regression analysis and are applied to specific problems 1) hot extrusion of steel 2) rock identification and the results are compared. The correlation between the actual values through multiple regression and the calculated values through functional network are very well correlated with correlation coefficient of 0.978 whereas the correlation co-efficient between values for FN and actual vales is 0.972457.

10.2 Suggestions for Future Work • Kohonen Networks can be modified using fuzzy concepts by considering fuzzy inputs. The Kohonen network may be given fuzzy inputs instead of crisp ones. Fuzzy competitive learning can be introduced. Instead of winner-take-all rule, a learn according to how-well-it-wins rule can be used. Instead of one dominating winning neuron, we can assign a degree of winning to each neuron depending on its distance to the current training pattern. • The μARTMAP can be made to learn each input as it is received on-line rather than performing off-line. The training size was from 100 to 8000 and this may be increased to 1,00,000 randomly chosen for better results. • The input patterns to MicroARTMAP may have imprecise or incomplete features (due to noise corruption). Extraction of features from a given pattern may become costly. So fuzzy inputs can be used instead of exact numerical values. • Neural networks maps inputs to outputs. It approximates a function or relation of cause and effect and acts as an universal approximator. Future systems may extend well beyond problems in control and machine intelligence and one has to model systems that one can control, shape or predict. One may have to deal with large-scale nonlinear systems with many variables. Users may not have experts or common sense to give rules nor neural networks possessing data or only sparse data to learn these rules. Hence neural like learning algorithms may frame these rules (automate framing rules). • The concepts of genetic algorithms [42], cellular automata and cellular genetic algorithms may be applied to the inputs given to MicroARTMAP to improve their performance. • Functional networks may be applied to predict values in other applications. The concept of equivalent functional networks may be studied and a universal functional network obtained.

References

1. Adeli, H., Hung, S.L.: Machine Learning Neural Networks, Genetic Algorithms and Fuzzy Systems. John Wiley and Sons, New York (1995) 2. Aleksander, I., Morton, H.: An Introduction to Neural Computing. Chapman and Hall, London (1990) 3. Amari, S.: Learning Patterns and Pattern Sequences by Self Organizing Nets of Threshold Elements. IEEE Transactions on Computers C-21, 1197–1206 (1972) 4. Anzai, Y.: Pattern Recognition and Machine Learning. Academic Press, Englewood Cliffs (1992) 5. Bajaj, R., Chaudhury, S.: Signature Verification Using Multiple Neural Classifiers. Pattern Recognition 30(1), 1–7 (1997) 6. Benedetto, J.J., Frazier, M.W. (eds.): Wavelets, Mathematics and Applications. CRC Press, Boca Raton (1994) 7. Brigham, E.O.: The Fast Fourier Transform. Prentice-Hall, Englewood Cliffs (1978) 8. Burrus, C.S., Gopinath, R.H., Guo, H.: Introduction to Wavelets and Wavelet Transforms. In: A Primer, Prentice-Hall, Englewood Cliffs (1998) 9. Looney, C.G.: Pattern Recognition Using Neural Networks: Theory and Algorithms for Engineers and Scientists (1997) 10. Carpenter, G.A., Grossberg, S., Markuzon, N., Reynolds, J., Rosen, D.B.: Fuzzy ARTMAP: A Neural-Network Architecture for Incremental Supervised Learning of Analog Multidimensional Maps. IEEE Trans. Neural Networks 3, 698–713 (1992) 11. Carpenter, A., Tan, H.A.: Rule Extraction: From Neural Architecture to Symbolic Representation. Connection Sci. 7(1), 3–27 (1995) 12. Carpenter, G.A., Milenova, B.L., Noeske, B.W.: Distributed ARTMAP: A Neural Network for Fast Distributed Supervised Learning. Neural Network 11(4), 793–813 (1998) 13. Carpenter, G.A., Gjaja, M.: Fuzzy ART Choice Functions, Technical Report CAS CNSTR-93-060, Boston University. In: Proceedings of the World Congress on Neural Networks (WCNN 1994) (1994) 14. Carpenter, G.A., Grossberg, S. (eds.): Pattern Recognition by Self Organizing Neural Networks. MIT Press, Cambridge (1991) 15. Carpenter, G.A., Grossberg, S., Reynolds, J.H.: ARTMAP: Supervised Real-Time Learning and Classification of Non-stationary Data by a Self organizing Neural Network. Neural Networks 4, 565–588 (1991) 16. Carpenter, G.A., Grossberg, S., Rosen, D.B.: Fuzzy ART: Fast Stable Learning and Categorization of Analog Patterns by an Adaptive Resonance System. Neural Networks 4, 759–771 (1991)

138

References

17. Carpenter, G.A., Grossberg, S., Rosen, D.B.: A Neural Network Realization of Fuzzy ART, Technical Report CAS CNS-TR-91-021, Boston University (1991) 18. Carpenter, G.A., Grossberg, S.: The Art of Adaptive Pattern Recognition by a Self Organizing Neural Network. IEEE Computer Magazine 21(3), 77–88 (1988) 19. Carpenter, G.A., Grossberg, S., Rosen, D.B.: ART 2A: An Adaptive Resonance Algorithm for Rapid Category Learning and Recognition. Neural Networks 4, 493–504 (1991) 20. Casagrande, A.: Classification and identification of soils. Trans. ASCE 113, 60–64 (1498) 21. Castillo, E., Ruiz–Cobo, R.: Functional Equations and Modeling in Science and Engineering. Marcel Dekker, New York (1992) 22. Castillo, E.: Functional Networks. Neural Processing Letters 7, 151–159 (1998) 23. Chakravarty, D., Misra, B.: Application of Neural Networks for the Rock Property Prediction. In: Proceedings of National Conference on Neural Networks and Fuzzy Systems, Anna University, Chennai, July 23-25, pp. 166–172 (1997) 24. Chen, R.M., Huang, Y.M.: Multiconstraint Task Scheduling in Multiprocessor System by Neural Network. In: Proceedings of the IEEE Tenth International Conference on Tools with Artificial Intelligence, pp. 288–294 (1998) 25. Christian, B.: Wavelets – a Primer. Universities Press Pvt. Ltd., Hyderabad (2003) 26. Coifman, R.R., Wickerhauser, M.V.: Wavelets and Adapted Waveform Analysis, A Toolkit for Signal Processing and Numerical Analysis. In: Proceedings of Symposia in Applied Mathematics, vol. 47, pp. 119–153 (1993) 27. Coifman, R.R., Wickerhauser, M.V.: Wavelets and Adapted Wave-form Analysis, pp. 399– 424 (1991) 28. Daubechies, I.: Orthonormal Bases of Compactly Supported Wavelets Communication on Pure and Applied Mathematics, vol. 41, pp. 909–996 (1988) 29. Daubechies, I. (ed.): Different Perspectives on Wavelets. AMS, Providence (1993) 30. Daubechies, I.: Ten Lectures on Wavelets. SIAM, Philadelphia (1992) 31. Dixon, M.X., Cole, G.R., Bellgard, M.I.: Using the Hopfield Network with Mean Field Annealing to Solve the Shortest Path Problem in a Communication Network. In: International Conference on Neural Networks, vol. 5, pp. 2652–2657 (1995) 32. Dubois, D., Prade, H.: Fuzzy Sets and Systems: Theory and Applications. Academic Press, Boston (1980) 33. Duric, Z., Li, F., Wechsler, H.: Recognition of Arm Movements. In: Fifth IEEE International conference on Automatic Face and Gesture Recognition, P 0348 (2002) 34. Gose, E., Johnsonbaugh, R., Jost, S.: Pattern Recognition and Image Analysis. Prentice Hall of India, New Delhi (2000) 35. Gomez-Sanchez, E., Dimitriadis, Y.A., Cano Izquierdo, J.M., Coronado, J.L.: μ ARTMAP Use of Mutual Information for Category Reduction in Fuzzy ARTMAP. IEEE Transactions on Neural Networks 13(1), 58–69 (2002) 36. Revuelta, F.F., Garc´ıa Chamizo, J.M., Pic´o, F.I.: Hand Gesture Recognition based on Morphologic Features. In: IX simposium Nacional de Reconoimiento de Formas Y Analisis de Imagenes, Benicassim Mayo (2001) 37. Freeman, J.A., Skapura, D.M.: Neural Networks Algorithms, Applications and Programming Techniques. Addison-Wesley, Reading (1991) 38. Limin, F.: Neural Networks in Computer Intelligence. McGraw Hill Inc., New York (1994) 39. Fukumi, M., Omatu, S., Takeda, F., Kosaka, T.: Rotation Invariant Neural Pattern Recognition System with Application to Coin Recognition. IEEE Transactions on Neural Networks 3(2), 241–251 (1992) 40. Gader, P., et al.: Fuzzy and Crisp Handwritten Character Recognition Using Neural Networks. In: Conference Proceedings of the 1992 Artificial Neural Networks in Engineering Conference, vol. 3, pp. 421–424 (1992)

References

139

41. Carpenter, G.A., Grossberg, S.: Fuzzy Adaptive Resonance Theory – Advanced Research Projects Agency ARPA, the office of Naval Research (ONR N00014-91-J-4100) 42. Goldberg, D.E.: Genetic Algorithms in Search: Optimization and Machine Learning. Addison-Wesley, Reading (1989) 43. Gomez-Sanchez, E., Gago-Gonzalez, Y.A., Dimitriadis, Y.A., Canolzquierdo, J.M., Coronado, J.I.: Experimental Study of a Novel Neuro Fuzzy System for On Line Handwritten Recognition. Pattern Recognition Lett. 19(3), 357–364 (1998) 44. Gupta, M.M., Rao, D.H. (eds.): Neuro-Control Systems: Theory and Applications. IEEE Press, NY (1994) 45. Guyon, I., Albrecht, P., Cun, Y.L., Denker, J., Hubbard, W.: Design of a Neural Network Character Recognizer for a Touch Terminal. Pattern Recognition 24(2), 105–117 (1991) 46. Hans Raj, K., Mangla, P., Srivatsav, S., Patvardhan, C.: A Neural Network Based Process Advisor for Hot Extrusion of Steel. In: Proceedings of International Conference on Cognitive Systems, ICCS 1997, New Delhi, pp. 767–777 ( December 1997) 47. Hecht-Neilsen, R.: Neuro Computing. Addison Wesley, Reading (1990) 48. Hopfield, J.J., Tank, D.W.: Computing with Neural Circuits: A Model. Science 233, 625– 633 (1986) 49. Hopfield, J.J.: Neural Networks and Physical Systems with Emergent Collective Computational Abilities. Proc. of the National Academy of Sciences 79, 2554–2558 (1982) 50. Hopfield, J.J., Tank, D.W.: Neural Computation of Decision in Optimization Problems. Biol. Cybernet 52, 141–152 (1985) 51. Hu, M.K.: Visual Pattern Recognition by Moment Invariants. IRE Trans. Inf. Theory 8, 179–187 (1962) 52. Ibarra Pico, F.: Memorias Associativas Ortogonales Y sus applicaciones en analisis de lectura. Institut de Cultura Juan Gil – Albert ISBN: 84-7784-258-2 (1997) (with English translation) 53. Walker, J.S.: A Primer on Wavelets and their Scientific Applications. Chapman and Hall, Boca Raton (1999) 54. Chandran, J.S.: Statistics for Business and Economics. Vikas Publishing House, New Delhi (2000) 55. Kartalopoulos, S.: Understanding Neural Networks and Fuzzy Logic. IEEE Press, NY (1996) 56. Tom, K.: Simplified Fuzzy ARTMAP, AI Expert, pp. 18–25 (November 1993) 57. Klir, J., Folger, T.A.: Fuzzy Sets, Uncertainty and Information. Prentice Hall, New York (1988) 58. Klir, G.J., Bo, Y.: Fuzzy Sets and Fuzzy Logic. Prentice Hall of India, New Delhi (1997) 59. Kohonen, T.: Self-organized Formation of Topologically Correct Feature Maps. Biol. Cybernet 43, 59–69 (1982) 60. Kohonen, T.: Self-organization and Associative Memory, 3rd edn. Springer, Berlin (1989) 61. Kohonen, T.: Correlation Matrix Memories. IEEE Transactions on Computers 21, 353–359 (1972) 62. Kohonen, T.: The Self Organizing Map. Proc. IEEE 78(9), 1464–1480 (1990) 63. Bart, K.: Fuzzy Engineering. Prentice Hall International, Englewood Cliffs (1997) 64. Bart, K.: Neural Networks and Fuzzy Systems: A Dynamical Systems Approach to Machine Intelligence. Prentice-Hall, New York (1992) 65. Kosko, B.: Fuzzy Entropy and Conditioning. Information Sciences 40(2), 165–174 (1986) 66. Murakami, K., Taguchi, H.: Gesture Recognition Using Recurrent Neural Networks. Journal of the ACM 1(1), 237–242 (1991) 67. Lee, K., Takefuji, Y., Funabiki, N.: A Parallel Improvement Algorithm for the Bipartite Subgraphy Problem. Case Western Reserve University, CAISR Technical Report TR91-105 (1991)

140

References

68. Lee, J.S., Sun, Y.N., Chen, C.H.: Multi-scale Corner Detection by Using Wavelet Transforms. IEEE Trans. Image Processing 4(1), 100–104 (1995) 69. Liang, K.C., Kuo, C.C.J.: Waveguide: A Joint Wavelet-Based Image Representation and Description System. IEEE Trans. Image Processing 8(11), 1619–1629 (1991) 70. Mallat, S.: A Wavelet Tour of Signal Processing. Academic Press, New York (1998) 71. Mallat, S.: A theory for multiresolution signal decomposition, the wavelet representation. IEEE Pattern Analysis and Machine Intelligence 11, 674–693 (1989) 72. Martin, G.L., Pittman, J.A.: Recognizing Hand Printed Letters and Digits Using Back Propagation Learning. Neural Computation 3(2), 258–267 (1991) 73. Merino, E.P., Dimitriadis, Y.A., Garefa, R.G., Lopez, J.C.: A Dictionary Based Neural network Scheme for On-Line Handwriting Recognition. In: Simner, M., Leedham, C., Thomassen, A. (eds.) Handwriting and Drawing Research: Basic and Applied, pp. 343– 358. IOS Press, Amsterdam (1996) 74. Meyer, Y.: Wavelets and Operators. Cambridge University Press, Cambridge (1992) 75. Meyer, Y.: Wavelets: Algorithms and Applications. SIAM, Philadelphia (1993) 76. Moore, B.: ART 1 and Pattern Clustering. In: Touretzky, D., Hinton, G., Sejnowski, T. (eds.) Proceedings of the 1988 Connectionist Models Summer School, pp. 174–185. Morgan Kaufmann Publishers, San Francisco (1989) 77. Nellis, J., Stonham, T.J.: A Neural Network Character Recognition System that Dynamically Adapts to an Author’s Writing Style. In: Conference Proceedings of the 1992 Artificial Neural Networks in Engineering Conference, vol. 3, pp. 975–979 (1992) 78. Pal, S.K., Majumdar, D.K.D.: Fuzzy Mathematical Approach to Pattern Recognition. Wiley Eastern Ltd., New Delhi (1986) 79. Papoulis, A.: Probability, Random Variables and Stochastic Processes, 3rd edn. McGraw Hill, New York (1991) 80. Parado-Hernandez, E., Gomez-Sanehez, E., Dimitriadis, Y.A.: Study of Distributed Learning as a Solution to Category Proliferation in Fuzzy ARTMAP Based Neural Systems. Neural Networks 16(7), 1039–1057 (2003) 81. Patterson, D.W.: Artificial Neural Networks: Theory and Applications. Prentice Hall, Englewood Cliffs (1996) 82. Maes, P., Darell, T., Blumberg, B., Penland, A.: The Alive System: Full-Body Interaction with Autonomous Agents. In: Computer Animation 1995 Conference. IEEE Press, Geneva (1995) 83. Punmia, B.C., Jain, A.K.: Soil Mechanics and Foundations. Laxmi Publications (2005) 84. Gonzalez, R.C., Woods, R.E.: Digital Image Processing, 2nd edn. Pearson Education, London (2004) 85. Rajasekaran, S., Suresh, D., Vijayalakshmi Pai, G.A.: Application of Sequential Learning Neural Networks to Civil Engineering Problems. Engineering with Computers 18, 138–147 (2002) 86. Rajasekaran, S., Vijayalakshmi Pai, G.A.: Simplified fuzzy ARTMAP as pattern recognizer. Comput. Civil Engg., ASCE 14(2), 92–99 (2000) 87. Rajasekaran, S., Vijayalakshmi Pai, G.A.: Image Recognition Using Simplified Fuzzy Artmap Augmented With a Moment Based Feature Extractor. Pattern Recognition, Artif. Intell. 14(8), 1081–1094 (2000) 88. Rajasekaran, S., Vijayalakshmi Pai, G.A.: Neural Networks, Fuzzy Logic and Genetic Algorithms. Prentice Hall India Pvt. Ltd., New Delhi (2004) 89. Sharma, R., Vladimir, L.P., Huang, T.S., et al.: Speech / Gesture Interface to a Visual Computing Environment. IEEE Computer Graphics and Applications 20(2), 29–37 (2000) 90. Raghuveer, R.M., Ajit, B.S.: Wavelet Transforms – Introduction to Theory and Applications. Pearson Education Limited, Singapore (1998)

References

141

91. Watson, R.: A Survey of Gesture Recognition Techniques., Technical Report TCD – CS – 93 – 11, Department of Computer Science, Trinity College, Dublin, Ireland (1993) 92. Rosenblatt, F.: Principles of Neurodynamics. Spartan Books (1962) 93. Chen, R.M., Huang, Y.-M.: Competitive Neural Network to Solve Scheduling Problems. Neuro Computing 37, 177–196 (2001) 94. Salzberg, S.L.: Learning with Nested Generalized Exemplars. Kluwer Academic Publishers, Dordrecht (1990) 95. Robert, S.: Digital Image Processing and Computer Vision. John Wiley and Sons Inc., Chichester (1989) 96. Robert, S.: Pattern Recognition – Statistical Structural and Neural Approaches. John Wiley and Sons Inc., Chichester (1992) 97. Simpson, P.: Fuzzy Min-Max Classification with Neural Networks. Heuristics: The Journal of Knowledge Engineering 4, 1–9 (1991) 98. Soman, K.P., Ramachandran, K.I.: Insight Into Wavelets From Theory To Practice. Prentice Hall of India, New Delhi (2004) 99. Takagi, H., Konda, T., Kojima, Y.: Neural Network Design on Approximate Reasoning and its Applications to Pattern Recognition. In: Proc. of the Intl. Conf. of Fuzzy Logic and Neural Networks, Iizuka, Japan, pp. 671–674 (July 1990) 100. Takeda, M., Goodman, J.W.: Neural Networks for Computation: Number Representation and Programming Complexity. Appl. Opt. 25, 3033–3046 (1986) 101. Baudel, T., Beaudounin, M.: Lafon CHARADE: Remote control of Objects using Free Hand Gestures. Communications of the ACM 36(7), 28–35 (1993) 102. Darell, T.J., Penland, A.P.: Space – time gestures. In: IEEE Conference on Vision and Pattern Recognition, New York (June 1993) 103. Rao, V., Rao, H.: C++ Neural Networks and Fuzzy Logic. MIS Press, New York (2000) 104. Verzi, S.J., Heileman, G.L., Georgiopaulos, M., Healy, M.J.: Boosted ARTMAP. In: Proc. IEEE World Congr. Comput. Intell., WCCI 1998, Anchorage, AK, pp. 395–400 (May 1998) 105. Pavlovis, V.L., Sharma, R., Huang, T.S.: Visual Interpretation of Hand Gestures For Human Computer Interaction: A Review. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(7), 677–695 (1997) 106. Wasserman, Philip, D.: Advanced Methods in Neural Computing. Van Nostrand Reinhold, New York (1993) 107. Wasserman, Philip, D.: Neural Computing. Van Nostrand Reinhold, New York (1989) 108. Whang, B.: Elasto Plastic Orthotropic Plates and Shells. In: Proceedings of the Symposium on Application for Finite Element Methods in Civil Engineering, ASCE, pp. 481–515 (November 1969) 109. Williamson, J.: Gaussian ARTMAP: A Neural Network for Fast Incremental Learning of Noisy Multidimensional Maps. Neural Network 9(5), 881–897 (1996) 110. Willems, T.M., Rooda, T.E.: Neural networks for job-shop scheduling. Control Eng. Practice 2(1), 31–39 (1994) 111. Yager, R. (ed.): Fuzzy Sets and Applications: Selected Papers by L.Z. Zadeh. WileyInterscience, New York (1987) 112. Yager, R.R.: Modeling and Formulating Fuzzy Knowledge Bases using Neural Networks. Neural Networks 7(8), 1273–1283 (1994) 113. Yamakawa, T., Tomoda, S.: A Fuzzy Neuron and its Application to Pattern Recognition. In: Proc. Third Intl. Fuzzy System Association Congress, Japan, pp. 30–38 (1989) 114. You, J., Bhattacharya, P.: A Wavelet-Based Course-to-Fine Image Matching Scheme in a Parallel Virtual Machine Environment. IEEE Trans. Image Processing 9(9), 1547–1559 (2000)

142

References

115. Huang, Y.-M., Chen, R.-M.: Scheduling Multiprocessor Job with Resource and Timing Constraints using Neural Network. IEEE Trans. System Man Cybernet, Part B 29(4), 490– 502 (1999) 116. Zadeh, L.: Fuzzy Sets. Information and Control 8, 338–353 (1965) 117. Zadeh, L.A.: The Role of Fuzzy Logic in the Management of Uncertainty in Expert Systems. In: Gupta, Kandel, Bandler, Kiszka (eds.) Approximate Reasoning in Expert System. Elsevier, NY (1985) 118. Zhang, J., Morris, A.J.: A Sequential Learning approach for single hidden layer neural networks. Neural Networks 11(1), 65–80 (1998) 119. Zimmermann, H.J.: Fuzzy Sets: Decision Making and Expert Systems. Kluwer Academic Publishers, Boston (1987)

Websites ftp : //wuarchive.wustl.edu:/doc/techreports/wustl.edu/math/wawa.ps.Z http : //en.wikipedia.org/wiki/Wavelet http : //wavelet tutorial/rabipolikar/rowan(tutorials) http : //www.amara.com/IEEE Wave/IEEEWavelet.html. http : //www.amara.com/current/wavelet.html. http : //www.amara.com/concurrent/wavelet.html http : //perso.wanadoo.fr/polyvalens/clemens/wavelets/wavelets(tutorials) http : //www.wavelet.org.

Appendix A: MicroARTMAP MATLAB Implementation

MicroARTMAP has two modes of operation: TRAINING: input/output pairs are provided to learn the relation between them, and this knowledge is stored in weight matrices. TEST: input is provided, together with weight matrices, and the output is estimated. Parameters of MicroARTMAP beta rho beta rho 1 2 rho ----A--- ----B--- entropy step parameter mic=[1.0 0.0 1.0 1.0 0 0.08 0.02]’; MicroARTMAP admits the following commands: MAINTENANCE MicroARTMAP(‘create’, TAG, PAR, WA, WB, WAB) MicroARTMAP(‘destroy’, TAG) LIST=MicroARTMAP(‘list’) MicroARTMAP(‘set parameters’, TAG, PAR) PAR=MicroARTMAP(‘get parameters’, PAR) MicroARTMAP(‘set weights’, TAG, WA, WB, WAB) [WA WB WAB]= MicroARTMAP(‘get weights’, TAG) USED=MicroARTMAP(’get used’, TAG) OPERATION MicroARTMAP(’train’, TAG, C, INP, OUT) OUT=MicroARTMAP(’test’, TAG, INP) DOCUMENTATION MicroARTMAP(’ver’) Parameters are as follows: TAG is a string of at most 10 characters. PAR is the parameters vector. V.K. David and S. Rajasekaran: Pattern Recog. Using Neural & Funct. Net., SCI 160, pp. 143–154. c Springer-Verlag Berlin Heidelberg 2009 springerlink.com

144

MicroARTMAP MATLAB Implementation

WA, WB and WAB are weights matrices. INP is a matrix with input data, each row a pattern. OUT is a matrix with the output data, each row a pattern. LIST is a character array USED is [Na Nb], the number of units used in ARTa and ARTb The input matrix INP and the output matrix OUT contain a pattern in each row, and therefore have the same number of rows. Also all input and output elements are normalised to [0,1]. If NaN are present in matrix INP in either test or incremental learning modes, corresponding entry in matrix OUT will be NaN. Also unpredicted samples will be a row of NaN in matrix OUT. WA contains weights of input module, and it is an M by N matrix where N must be two times the number of columns of INP, and M is the maximum number of memory units. WB contains weights of output module, and it is an M by N matrix where N must be two times the number of columns of OUT, and M is the maximum number of memory units. WAB contains weights of inter-ART map, and it is matrix of Ma, the number of rows in WA, by Mb, the number of rows in WB. It contains frequencies of associations between ARTa and ARTb categories. All weights are initialised to 1 before first learning. PAR is a column vector containing the following eight parameters: input module: output module: control or error:

BETA : learning rate (w,v weights) RHO : vigilance factor BETA : learning rate (w,v weights) RHO : vigilance factor h max : maximum contribution to entropy H max : maximum entropy RHOstep: step for incrementing RHO

TAG can have any values, only ‘all’ is reserved. MICRO(‘destroy’, ‘all’) clears all networks from memory. MICRO(‘ver’) prints MEX file version MICRO is case sensitive to commands and tags. EXAMPLE INP=[...]; OUT=[...]; PAR=[1.0 0.0 1.0 0.9 0.0 0.1 0.02]’; [Mi Ni]=size(INP); [Mo No]=size(OUT); UnitsA=501; UnitsB=501;

[0,1] [0,1] [0,1] [0,1] [0,infty] [0,infty] [0,1]

MicroARTMAP MATLAB Implementation

145

WA=ones(UnitsA, 2*Ni); WB=ones(UnitsB, 2*No); WAB=ones(UnitsA,1); Create NET MICRO(‘create’, ‘net1’, WA, WB, WAB); List existing networks list=MICRO(‘list’) Train network MICRO(‘train’, ‘net1’, INP, SUP) Get weights [WA WB WAB]=MICRO(‘get weights’, ‘net1’); Test on some input INP=[...]; OUT=MICRO(‘test’, ‘net1’, INP); Destroy network MICRO(‘destroy’, ‘net1’) Clear MEX file clear micro handpgm1.m A = [0 0 1 0 0 0 1 0 1 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 1 1 0 0 0 1; 1 0 1 0 0 0 1 0 1 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 1 1 0 0 0 1; 0 1 1 0 0 0 1 0 1 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 1 1 0 0 0 1; 0 0 0 0 0 0 1 0 1 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 1 1 0 0 0 1; 0 0 1 0 1 0 1 0 1 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 1 1 0 0 0 1; 0 0 1 0 0 1 1 0 1 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 1 1 0 0 0 1; 0 0 1 0 0 0 0 0 1 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 1 1 0 0 0 1; 0 0 1 0 0 0 1 1 1 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 1 1 0 0 0 1; 0 0 1 0 0 0 1 0 1 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 1 1 0 0 0 1; 0 0 1 0 0 0 1 0 1 0 0 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 1 1 0 0 0 1]; B = [1 1 1 1 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0; 0 1 1 1 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0; 1 0 1 1 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0; 1 1 0 1 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0; 1 1 1 0 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0; 1 1 1 1 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0; 1 1 1 1 0 0 0 0 0 1 1 0 0 0 1 1 1 1 1 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0; 1 1 1 1 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0; 1 1 1 1 0 1 0 1 0 1 1 0 0 0 1 1 1 1 1 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0; 1 1 1 1 0 1 0 0 0 0 1 0 0 0 1 1 1 1 1 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0]; C = [0 1 1 1 0 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 1 0 1 1 1 0; 1 1 1 1 0 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 1 0 1 1 1 0; 0 0 1 1 0 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 1 0 1 1 1 0; 0 1 0 1 0 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 1 0 1 1 1 0;

146

0 0 0 0 0 0

1 1 1 1 1 1

MicroARTMAP MATLAB Implementation

1 1 1 1 1 1

0 1 1 1 1 1

0 1 0 0 0 0

1 1 0 1 1 1

0 0 0 0 0 0

0 0 0 0 1 0

0 0 0 0 0 1

1 1 1 1 1 1

1 1 1 1 1 1

0 0 0 0 0 0

0 0 0 0 0 0

0 0 0 0 0 0

0 0 0 0 0 0

1 1 1 1 1 1

0 0 0 0 0 0

0 0 0 0 0 0

0 0 0 0 0 0

0 0 0 0 0 0

1 1 1 1 1 1

0 0 0 0 0 0

0 0 0 0 0 0

0 0 0 0 0 0

0 0 0 0 0 0

1 1 1 1 1 1

0 0 0 0 0 0

0 0 0 0 0 0

0 0 0 0 0 0

1 1 1 1 1 1

0 0 0 0 0 0

1 1 1 1 1 1

1 1 1 1 1 1

1 1 1 1 1 1

0; 0; 0; 0; 0; 0];

D = [1 1 1 1 0 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0; 0 1 1 1 0 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0; 1 0 1 1 0 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0; 1 1 0 1 0 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0; 1 1 1 0 0 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0; 1 1 1 1 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0; 1 1 1 1 0 0 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0; 1 1 1 1 0 1 0 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0; 1 1 1 1 0 1 0 0 1 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0; 1 1 1 1 0 1 0 0 0 0 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0]; E = [1 1 1 1 1 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1; 0 1 1 1 1 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1; 1 0 1 1 1 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1; 1 1 0 1 1 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1; 1 1 1 0 1 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1; 1 1 1 1 0 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1; 1 1 1 1 1 0 0 0 0 0 1 0 0 0 0 1 1 1 1 1 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1; 1 1 1 1 1 1 1 0 0 0 1 0 0 0 0 1 1 1 1 1 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1; 1 1 1 1 1 1 0 1 0 0 1 0 0 0 0 1 1 1 1 1 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1; 1 1 1 1 1 1 0 0 1 0 1 0 0 0 0 1 1 1 1 1 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1]; F = [1 1 1 1 0 1 1 1 1 1 1 0 1 1 1 1 1 1 0 1 1 1 1 1 1 0 1 1 1 1 1 1 0 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

1 0 0 0 0 0 0 1 0 0

1 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0 1 0 0 0 0 1 1 0 0 1 0 0 0 0 1 0 1 0 1 0 0 0 0 1

0 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 0 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0

0 1 1 1 1 1 1 1 1 1

0 1 0 0 0 0 1 0 0 0 0; 0 0 0 0 1 0 0 0 0; 0 0 0 0 1 0 0 0 0; 0 0 0 0 1 0 0 0 0; 0 0 0 0 1 0 0 0 0; 0 0 0 0 1 0 0 0 0; 0 0 0 0 1 0 0 0 0; 0 0 0 0 1 0 0 0 0; 0 0 0 0 1 0 0 0 0; 0 0 0 0 1 0 0 0 0];

G = [0 1 1 1 0 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 1 1 1 0 0 1 0 0 1 1 1 0; 1 1 1 1 0 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 1 1 1 0 0 1 0 0 1 1 1 0; 0 0 1 1 0 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 1 1 1 0 0 1 0 0 1 1 1 0; 0 1 0 1 0 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 1 1 1 0 0 1 0 0 1 1 1 0; 0 1 1 0 0 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 1 1 1 0 0 1 0 0 1 1 1 0;

MicroARTMAP MATLAB Implementation

0 0 0 0 0

1 1 1 1 1

1 1 1 1 1

1 1 1 1 1

1 0 0 0 0

1 0 1 1 1

0 0 1 0 0

0 0 0 1 0

0 0 0 0 1

1 1 1 1 1

1 1 1 1 1

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

1 1 1 1 1

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

1 1 1 1 1

0 0 0 0 0

0 0 0 0 0

1 1 1 1 1

1 1 1 1 1

1 1 1 1 1

0 0 0 0 0

0 0 0 0 0

1 1 1 1 1

0 0 0 0 0

0 0 0 0 0

1 1 1 1 1

1 1 1 1 1

1 1 1 1 1

147

0; 0; 0; 0; 0];

H = [1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1; 0 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1; 1 1 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1; 1 0 1 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1; 1 0 0 1 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1; 1 0 0 0 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1; 1 0 0 0 1 0 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1; 1 0 0 0 1 1 1 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1; 1 0 0 0 1 1 0 1 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1; 1 0 0 0 1 1 0 0 1 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1]; I = [1 0 1 1 1 0 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

1 1 1 1 0 1 1 1 1 1

1 1 1 1 1 0 1 1 1 1

1 0 0 0 0 0 1 0 0 0

1 0 0 0 0 0 0 1 0 0

0 1 1 1 1 1 1 1 0 1

0 0 0 0 0 0 0 0 0 1

1 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 1 1 1 1 1 1 1 1 1

0 0 0 0 0 0 0 0 0 0

1 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 1 1 1 1 1 1 1 1 1

0 0 0 0 0 0 0 0 0 0

1 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 1 1 1 1 1 1 1 1 1

0 0 0 0 0 0 0 0 0 0

1 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 1 1 1 1 1 1 1 1 1

0 0 0 0 0 0 0 0 0 0

1 0 0 0 0 0 0 0 0 0

0 1 1 1 1 1 1 1 1 1

0 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1

1 1 1; 1; 1; 1; 1; 1; 1; 1; 1; 1];

J = [1 0 1 1 1 0 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

1 1 1 1 0 1 1 1 1 1

1 1 1 1 1 0 1 1 1 1

1 0 0 0 0 0 1 0 0 0

1 0 0 0 0 0 0 1 0 0

0 1 1 1 1 1 1 1 0 1

0 0 0 0 0 0 0 0 0 1

1 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 1 1 1 1 1 1 1 1 1

0 0 0 0 0 0 0 0 0 0

1 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 1 1 1 1 1 1 1 1 1

0 0 0 0 0 0 0 0 0 0

1 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 1 1 1 1 1 1 1 1 1

0 0 0 0 0 0 0 0 0 0

1 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 1 1 1 1 1 1 1 1 1

0 0 0 0 0 0 0 0 0 0

1 0 0 0 0 0 0 0 0 0

0 1 1 1 1 1 1 1 1 1

0 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1

1 0 0 0 0 0 0 0 0 0

1 0 0; 0; 0; 0; 0; 0; 0; 0; 0; 0];

K = [1 0 0 1 0 1 0 1 0 0 1 1 0 0 0 1 0 0 0 0 1 1 0 0 0 1 0 1 0 0 1 0 0 1 0; 0 0 0 1 0 1 0 1 0 0 1 1 0 0 0 1 0 0 0 0 1 1 0 0 0 1 0 1 0 0 1 0 0 1 0; 1 1 0 1 0 1 0 1 0 0 1 1 0 0 0 1 0 0 0 0 1 1 0 0 0 1 0 1 0 0 1 0 0 1 0; 1 0 1 1 0 1 0 1 0 0 1 1 0 0 0 1 0 0 0 0 1 1 0 0 0 1 0 1 0 0 1 0 0 1 0; 1 0 0 0 0 1 0 1 0 0 1 1 0 0 0 1 0 0 0 0 1 1 0 0 0 1 0 1 0 0 1 0 0 1 0; 1 0 0 1 1 1 0 1 0 0 1 1 0 0 0 1 0 0 0 0 1 1 0 0 0 1 0 1 0 0 1 0 0 1 0; 1 0 0 1 0 0 0 1 0 0 1 1 0 0 0 1 0 0 0 0 1 1 0 0 0 1 0 1 0 0 1 0 0 1 0;

148

MicroARTMAP MATLAB Implementation

1 0 0 1 0 1 1 1 0 0 1 1 0 0 0 1 0 0 0 0 1 1 0 0 0 1 0 1 0 0 1 0 0 1 0; 1 0 0 1 0 1 0 0 0 0 1 1 0 0 0 1 0 0 0 0 1 1 0 0 0 1 0 1 0 0 1 0 0 1 0; 1 0 0 1 0 1 0 1 1 0 1 1 0 0 0 1 0 0 0 0 1 1 0 0 0 1 0 1 0 0 1 0 0 1 0]; L = [1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1; 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1; 1 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1; 1 0 1 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1; 1 0 0 1 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1; 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1; 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1; 1 0 0 0 0 1 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1; 1 0 0 0 0 1 0 1 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1; 1 0 0 0 0 1 0 0 1 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 1 1]; M = [1 0 0 0 1 1 1 0 1 1 1 0 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1; 0 0 0 0 1 1 1 0 1 1 1 0 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1; 1 1 0 0 1 1 1 0 1 1 1 0 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1; 1 0 1 0 1 1 1 0 1 1 1 0 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1; 1 0 0 1 1 1 1 0 1 1 1 0 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1; 1 0 0 0 0 1 1 0 1 1 1 0 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1; 1 0 0 0 1 0 1 0 1 1 1 0 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1; 1 0 0 0 1 1 0 0 1 1 1 0 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1; 1 0 0 0 1 1 1 1 1 1 1 0 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1; 1 0 0 0 1 1 1 0 0 1 1 0 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1]; N = [0 0 0 0 0 1 0 0 0 1 1 1 0 0 1 1 0 1 0 1 1 0 0 1 1 1 0 0 0 1 0 0 0 0 0; 1 0 0 0 0 1 0 0 0 1 1 1 0 0 1 1 0 1 0 1 1 0 0 1 1 1 0 0 0 1 0 0 0 0 0; 0 1 0 0 0 1 0 0 0 1 1 1 0 0 1 1 0 1 0 1 1 0 0 1 1 1 0 0 0 1 0 0 0 0 0; 0 0 1 0 0 1 0 0 0 1 1 1 0 0 1 1 0 1 0 1 1 0 0 1 1 1 0 0 0 1 0 0 0 0 0; 0 0 0 1 0 1 0 0 0 1 1 1 0 0 1 1 0 1 0 1 1 0 0 1 1 1 0 0 0 1 0 0 0 0 0; 0 0 0 0 1 1 0 0 0 1 1 1 0 0 1 1 0 1 0 1 1 0 0 1 1 1 0 0 0 1 0 0 0 0 0; 0 0 0 0 0 0 0 0 0 1 1 1 0 0 1 1 0 1 0 1 1 0 0 1 1 1 0 0 0 1 0 0 0 0 0; 0 0 0 0 0 1 1 0 0 1 1 1 0 0 1 1 0 1 0 1 1 0 0 1 1 1 0 0 0 1 0 0 0 0 0; 0 0 0 0 0 1 0 1 0 1 1 1 0 0 1 1 0 1 0 1 1 0 0 1 1 1 0 0 0 1 0 0 0 0 0; 0 0 0 0 0 1 0 0 1 1 1 1 0 0 1 1 0 1 0 1 1 0 0 1 1 1 0 0 0 1 0 0 0 0 0]; O = [0 1 1 1 0 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 0 1 1 1 0; 1 1 1 1 0 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 0 1 1 1 0; 0 0 1 1 0 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 0 1 1 1 0; 0 1 0 1 0 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 0 1 1 1 0; 0 1 1 0 0 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 0 1 1 1 0; 0 1 1 1 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 0 1 1 1 0; 0 1 1 1 0 1 1 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 0 1 1 1 0; 0 1 1 1 0 1 0 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 0 1 1 1 0;

MicroARTMAP MATLAB Implementation

149

0 1 1 1 0 1 0 0 1 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 0 1 1 1 0; 0 1 1 1 0 1 0 0 0 0 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 0 1 1 1 0]; P = [1 1 1 1 0 1 1 1 1 1 1 0 1 1 1 1 1 1 0 1 1 1 1 1 1 0 1 1 1 1 1 1 0 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

1 0 0 0 0 0 0 1 0 0

1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0; 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0; 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0; 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0; 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0; 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0; 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0; 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0; 1 0 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0; 0 1 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0];

Q = [0 1 1 1 0 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 0 0 1 1 0 1 1 1 1; 1 1 1 1 0 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 0 0 1 1 0 1 1 1 1; 0 0 1 1 0 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 0 0 1 1 0 1 1 1 1; 0 1 0 1 0 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 0 0 1 1 0 1 1 1 1; 0 1 1 0 0 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 0 0 1 1 0 1 1 1 1; 0 1 1 1 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 0 0 1 1 0 1 1 1 1; 0 1 1 1 0 0 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 0 0 1 1 0 1 1 1 1; 0 1 1 1 0 1 1 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 0 0 1 1 0 1 1 1 1; 0 1 1 1 0 1 0 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 0 0 1 1 0 1 1 1 1; 0 1 1 1 0 1 0 0 1 1 1 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 0 0 1 1 0 1 1 1 1]; R = [1 1 1 1 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0 1 1 0 0 0 1 0 1 0 0 1 0 0 1 0; 0 1 1 1 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0 1 1 0 0 0 1 0 1 0 0 1 0 0 1 0; 1 0 1 1 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0 1 1 0 0 0 1 0 1 0 0 1 0 0 1 0; 1 1 0 1 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0 1 1 0 0 0 1 0 1 0 0 1 0 0 1 0; 1 1 1 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0 1 1 0 0 0 1 0 1 0 0 1 0 0 1 0; 1 1 1 1 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0 1 1 0 0 0 1 0 1 0 0 1 0 0 1 0; 1 1 1 1 1 0 0 0 0 1 1 0 0 0 1 1 1 1 1 0 1 1 0 0 0 1 0 1 0 0 1 0 0 1 0; 1 1 1 1 1 1 1 0 0 1 1 0 0 0 1 1 1 1 1 0 1 1 0 0 0 1 0 1 0 0 1 0 0 1 0; 1 1 1 1 1 1 0 1 0 1 1 0 0 0 1 1 1 1 1 0 1 1 0 0 0 1 0 1 0 0 1 0 0 1 0; 1 1 1 1 1 1 0 0 1 1 1 0 0 0 1 1 1 1 1 0 1 1 0 0 0 1 0 1 0 0 1 0 0 1 0]; S = [0 1 1 1 1 1 1 1 1 1 0 0 1 1 1 1 0 1 0 1 1 1 0 1 1 0 1 1 0 1 1 1 0 1 0 1 1 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 0 1 1 1 1 1

1 0 0 0 0 0 0 1 0 0

1 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0 1 0 0 0 0 1 1 0 0 1 0 0 0 0 1 0 1 0 1 0 0 0 0 1

0 1 1 1 1 1 1 1 1 1

1 1 1 1 1 0 0 0 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1

0 0 0 0 0 0 0 0 0 0

1 0 0 0 0 1 1 1 1 1 0; 0 0 0 1 1 1 1 1 0; 0 0 0 1 1 1 1 1 0; 0 0 0 1 1 1 1 1 0; 0 0 0 1 1 1 1 1 0; 0 0 0 1 1 1 1 1 0; 0 0 0 1 1 1 1 1 0; 0 0 0 1 1 1 1 1 0; 0 0 0 1 1 1 1 1 0; 0 0 0 1 1 1 1 1 0];

150

MicroARTMAP MATLAB Implementation

T = [1 1 1 1 1 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0; 0 1 1 1 1 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0; 1 0 1 1 1 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0; 1 1 0 1 1 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0; 1 1 1 0 1 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0; 1 1 1 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0; 1 1 1 1 1 1 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0; 1 1 1 1 1 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0; 1 1 1 1 1 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0; 1 1 1 1 1 0 0 1 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0]; U = [1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 0 1 1 1 0; 0 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 0 1 1 1 0; 1 1 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 0 1 1 1 0; 1 0 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 0 1 1 1 0; 1 0 0 1 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 0 1 1 1 0; 1 0 0 0 0 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 0 1 1 1 0; 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 0 1 1 1 0; 1 0 0 0 1 1 1 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 0 1 1 1 0; 1 0 0 0 1 1 0 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 0 1 1 1 0; 1 0 0 0 1 1 0 0 1 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 0 1 1 1 0]; V = [0 0 0 0 0 1 0 0 0 1 1 0 0 0 1 0 1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0; 1 0 0 0 0 1 0 0 0 1 1 0 0 0 1 0 1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0; 0 1 0 0 0 1 0 0 0 1 1 0 0 0 1 0 1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0; 0 0 1 0 0 1 0 0 0 1 1 0 0 0 1 0 1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0; 0 0 0 1 0 1 0 0 0 1 1 0 0 0 1 0 1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0; 0 0 0 0 1 1 0 0 0 1 1 0 0 0 1 0 1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0; 0 0 0 0 0 0 0 0 0 1 1 0 0 0 1 0 1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0; 0 0 0 0 0 1 1 0 0 1 1 0 0 0 1 0 1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0; 0 0 0 0 0 1 0 1 0 1 1 0 0 0 1 0 1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0; 0 0 0 0 0 1 0 0 1 1 1 0 0 0 1 0 1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0]; W = [1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 1 0 1 1 1 0 0 0 1; 0 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 1 0 1 1 1 0 0 0 1; 1 1 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 1 0 1 1 1 0 0 0 1; 1 0 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 1 0 1 1 1 0 0 0 1; 1 0 0 1 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 1 0 1 1 1 0 0 0 1; 1 0 0 0 0 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 1 0 1 1 1 0 0 0 1; 1 0 0 0 1 0 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 1 0 1 1 1 0 0 0 1; 1 0 0 0 1 1 1 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 1 0 1 1 1 0 0 0 1; 1 0 0 0 1 1 0 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 1 0 1 1 1 0 0 0 1; 1 0 0 0 1 1 0 0 1 1 1 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 1 0 1 1 1 0 0 0 1];

MicroARTMAP MATLAB Implementation

151

X = [1 0 0 0 1 0 1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 1 0 1 0 1 0 0 0 1; 0 0 0 0 1 0 1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 1 0 1 0 1 0 0 0 1; 1 1 0 0 1 0 1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 1 0 1 0 1 0 0 0 1; 1 0 1 0 1 0 1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 1 0 1 0 1 0 0 0 1; 1 0 0 1 1 0 1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 1 0 1 0 1 0 0 0 1; 1 0 0 0 0 0 1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 1 0 1 0 1 0 0 0 1; 1 0 0 0 1 1 1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 1 0 1 0 1 0 0 0 1; 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 1 0 1 0 1 0 0 0 1; 1 0 0 0 1 0 1 1 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 1 0 1 0 1 0 0 0 1; 1 0 0 0 1 0 1 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 1 0 1 0 1 0 0 0 1]; Y = [1 0 0 0 1 0 1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0; 0 0 0 0 1 0 1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0; 1 1 0 0 1 0 1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0; 1 0 1 0 1 0 1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0; 1 0 0 1 1 0 1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0; 1 0 0 0 0 0 1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0; 1 0 0 0 1 1 1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0; 1 0 0 0 1 0 1 1 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0; 1 0 0 0 1 0 1 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0; 1 0 0 0 1 0 1 0 1 1 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0]; Z = [1 1 1 1 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0; 0 1 1 1 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0; 1 0 1 1 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0; 1 1 0 1 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0; 1 1 1 0 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0; 1 1 1 1 0 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0; 1 1 1 1 1 1 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0; 1 1 1 1 1 0 1 0 1 0 0 0 1 0 0 0 1 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0; 1 1 1 1 1 0 0 1 1 0 0 0 1 0 0 0 1 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0; 1 1 1 1 1 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0]; PA = [1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1; 0 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1; 1 1 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1; 1 0 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1; 1 0 0 1 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1; 1 0 0 0 0 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1; 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1; 1 0 0 0 1 1 1 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1; 1 0 0 0 1 1 0 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1; 1 0 0 0 1 1 0 0 1 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1];

152

MicroARTMAP MATLAB Implementation

MA = [1 0 1 1 1 1 0 1 0 1 1 0 1 0 1 1 0 1 0 1 1 0 0 1 1 1 1 0 1 0 1 1 0 1 0 1 1 0 1 0 1 1 0 1 0 1 1 1 1 1 1 0 1 0 1 1 0 1 0 1 1 0 1 0 1 1 0 1 0 1 0 0 1 1 1 0 1 0 1 1 0 1 0 1 1 0 1 0 1 1 0 1 0 1 0 1 0 1 1 0 1 0 1 1 0 1 0 1 1 0 1 0 1 1 0 1 0 1 0 1 1 0 1 0 1 0 1 1 0 1 0 1 1 0 1 0 1 1 0 1 0 1 0 1 1 1 0 0 1 0 1 1 0 1 0 1 1 0 1 0 1 1 0 1 0 1 0 1 1 1 1 1 1 0 1 1 0 1 0 1 1 0 1 0 1 1 0 1 0 1 0 1 1 1 1 0 0 0 1 1 0 1 0 1 1 0 1 0 1 1 0 1 0 1 0 1 1 1 1 0 1 1 1 1 0 1 0 1 1 0 1 0 1 1 0 1 0

0 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1

0 0 0 0 0 0 0 0 0 0

1 1 1 1 1 1 1 1 1 1

1 0 0 0 0 0 0 0 0 0

0 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1

0 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1

1 1 1 1; 1; 1; 1; 1; 1; 1; 1; 1; 1];

YA = [1 0 0 1 0 1 1 1 0 1 0 0 0 1 0 1 1 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0

0 1 1 1 1 0 1 1 1 1

1 1 1 1 1 1 0 1 1 1

0 0 0 0 0 0 0 1 0 0

1 1 1 1 1 1 1 1 0 1

1 0 0 0 0 0 0 0 0 1

0 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1

0 0 0 0 0 0 0 0 0 0

1 1 1 1 1 1 1 1 1 1

1 0 0 0 0 0 0 0 0 0

0 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1

0 0 0 0 0 0 0 0 0 0

1 1 1 1 1 1 1 1 1 1

1 0 0 0 0 0 0 0 0 0

0 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1

0 0 0 0 0 0 0 0 0 0

1 1 1 1 1 1 1 1 1 1

1 0 0 0 0 0 0 0 0 0

0 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1

0 0 0 0 0 0 0 0 0 0

1 1 1 1 1 1 1 1 1 1

1 0 0 0 0 0 0 0 0 0

0 1 1 1 1 1 1 1 1 1

1 0 0 0 0 0 0 0 0 0

0 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1

0 1 1 1 1 1 1 1 1 1

1 1 1 1; 1; 1; 1; 1; 1; 1; 1; 1; 1];

LA = [1 0 0 1 0 1 1 1 0 1 0 0 0 1 0 1 1 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0

1 1 1 1 1 0 1 1 1 1

1 1 1 1 1 1 0 1 1 1

0 0 0 0 0 0 0 1 0 0

1 1 1 1 1 1 1 1 0 1

1 0 0 0 0 0 0 0 0 1

0 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1

0 0 0 0 0 0 0 0 0 0

1 1 1 1 1 1 1 1 1 1

1 0 0 0 0 0 0 0 0 0

0 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1

0 0 0 0 0 0 0 0 0 0

1 1 1 1 1 1 1 1 1 1

1 0 0 0 0 0 0 0 0 0

0 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1

0 0 0 0 0 0 0 0 0 0

1 1 1 1 1 1 1 1 1 1

1 0 0 0 0 0 0 0 0 0

0 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1

0 0 0 0 0 0 0 0 0 0

1 1 1 1 1 1 1 1 1 1

1 0 0 0 0 0 0 0 0 0

1 1 1 1 1 1 1 1 1 1

1 0 0 0 0 0 0 0 0 0

0 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1

1 1 1 1; 1; 1; 1; 1; 1; 1; 1; 1; 1];

INP = [A’ B’ C’ D’ E’ F’ G’ H’ I’ J’ W’ X’ Y’ Z’ PA’ MA’ YA’ LA’]’; a1 = [0 0 0 0 1; 0 0 0 0 1; 0 0 0 0 1; 0 1; 0 0 0 0 1; 0 0 0 0 1; 0 0 0 0 1]; a2 = [0 0 0 1 0; 0 0 0 1 0; 0 0 0 1 0; 1 0; 0 0 0 1 0; 0 0 0 1 0; 0 0 0 1 0]; a3 = [0 0 0 1 1; 0 0 0 1 1; 0 0 0 1 1; 1 1; 0 0 0 1 1; 0 0 0 1 1; 0 0 0 1 1]; a4 = [0 0 1 0 0; 0 0 1 0 0; 0 0 1 0 0; 0 0; 0 0 1 0 0; 0 0 1 0 0; 0 0 1 0 0]; a5 = [0 0 1 0 1; 0 0 1 0 1; 0 0 1 0 1; 0 1; 0 0 1 0 1; 0 0 1 0 1; 0 0 1 0 1];

K’ L’ M’ N’ O’ P’ Q’ R’ S’ T’ U’ V’ 0 0 0 0 1; 0 0 0 0 1; 0 0 0 0 1; 0 0 0 0 0 0 1 0; 0 0 0 1 0; 0 0 0 1 0; 0 0 0 0 0 0 1 1; 0 0 0 1 1; 0 0 0 1 1; 0 0 0 0 0 1 0 0; 0 0 1 0 0; 0 0 1 0 0; 0 0 1 0 0 1 0 1; 0 0 1 0 1; 0 0 1 0 1; 0 0 1

MicroARTMAP MATLAB Implementation

a6 = [0 0 1 1 0; 0 0 1 1 0; 0 0 1 1 0; 0 0 1 1 0; 0 0 1 1 1 0; 0 0 1 1 0; 0 0 1 1 0; 0 0 1 1 0]; a7 = [0 0 1 1 1; 0 0 1 1 1; 0 0 1 1 1; 0 0 1 1 1; 0 0 1 1 1 1; 0 0 1 1 1; 0 0 1 1 1; 0 0 1 1 1]; a8 = [0 1 0 0 0; 0 1 0 0 0; 0 1 0 0 0; 0 1 0 0 0; 0 1 0 0 0 0; 0 1 0 0 0; 0 1 0 0 0; 0 1 0 0 0]; a9 = [0 1 0 0 1; 0 1 0 0 1; 0 1 0 0 1; 0 1 0 0 1; 0 1 0 0 0 1; 0 1 0 0 1; 0 1 0 0 1; 0 1 0 0 1]; a10 = [0 1 0 1 0; 0 1 0 1 0; 0 1 0 1 0; 0 1 0 1 0; 0 1 0 1 1 0; 0 1 0 1 0; 0 1 0 1 0; 0 1 0 1 0]; a11 = [0 1 0 1 1; 0 1 0 1 1; 0 1 0 1 1; 0 1 0 1 1; 0 1 0 1 1 1; 0 1 0 1 1; 0 1 0 1 1; 0 1 0 1 1]; a12 = [0 1 1 0 0; 0 1 1 0 0; 0 1 1 0 0; 0 1 1 0 0; 0 1 1 0 0 0; 0 1 1 0 0; 0 1 1 0 0; 0 1 1 0 0]; a13 = [0 1 1 0 1; 0 1 1 0 1; 0 1 1 0 1; 0 1 1 0 1; 0 1 1 0 0 1; 0 1 1 0 1; 0 1 1 0 1; 0 1 1 0 1]; a14 = [0 1 1 1 0; 0 1 1 1 0; 0 1 1 1 0; 0 1 1 1 0; 0 1 1 1 1 0; 0 1 1 1 0; 0 1 1 1 0; 0 1 1 1 0]; a15 = [0 1 1 1 1; 0 1 1 1 1; 0 1 1 1 1; 0 1 1 1 1; 0 1 1 1 1 1; 0 1 1 1 1; 0 1 1 1 1; 0 1 1 1 1]; a16 = [1 0 0 0 0; 1 0 0 0 0; 1 0 0 0 0; 1 0 0 0 0; 1 0 0 0 0 0; 1 0 0 0 0; 1 0 0 0 0; 1 0 0 0 0]; a17 = [1 0 0 0 1; 1 0 0 0 1; 1 0 0 0 1; 1 0 0 0 1; 1 0 0 0 0 1; 1 0 0 0 1; 1 0 0 0 1; 1 0 0 0 1]; a18 = [1 0 0 1 0; 1 0 0 1 0; 1 0 0 1 0; 1 0 0 1 0; 1 0 0 1 1 0; 1 0 0 1 0; 1 0 0 1 0; 1 0 0 1 0]; a19 = [1 0 0 1 1; 1 0 0 1 1; 1 0 0 1 1; 1 0 0 1 1; 1 0 0 1 1 1; 1 0 0 1 1; 1 0 0 1 1; 1 0 0 1 1]; a20 = [1 0 1 0 0; 1 0 1 0 0; 1 0 1 0 0; 1 0 1 0 0; 1 0 1 0 0 0; 1 0 1 0 0; 1 0 1 0 0; 1 0 1 0 0]; a21 = [1 0 1 0 1; 1 0 1 0 1; 1 0 1 0 1; 1 0 1 0 1; 1 0 1 0 0 1; 1 0 1 0 1; 1 0 1 0 1; 1 0 1 0 1]; a22 = [1 0 1 1 0; 1 0 1 1 0; 1 0 1 1 0; 1 0 1 1 0; 1 0 1 1 1 0; 1 0 1 1 0; 1 0 1 1 0; 1 0 1 1 0]; a23 = [1 0 1 1 1; 1 0 1 1 1; 1 0 1 1 1; 1 0 1 1 1; 1 0 1 1 1 1; 1 0 1 1 1; 1 0 1 1 1; 1 0 1 1 1]; a24 = [1 1 0 0 0; 1 1 0 0 0; 1 1 0 0 0; 1 1 0 0 0; 1 1 0 0 0 0; 1 1 0 0 0; 1 1 0 0 0; 1 1 0 0 0]; a25 = [1 1 0 0 1; 1 1 0 0 1; 1 1 0 0 1; 1 1 0 0 1; 1 1 0 0 0 1; 1 1 0 0 1; 1 1 0 0 1; 1 1 0 0 1]; a26 = [1 1 0 1 0; 1 1 0 1 0; 1 1 0 1 0; 1 1 0 1 0; 1 1 0 1 1 0; 1 1 0 1 0; 1 1 0 1 0; 1 1 0 1 0]; a27 = [1 1 0 1 1; 1 1 0 1 1; 1 1 0 1 1; 1 1 0 1 1; 1 1 0 1 1 1; 1 1 0 1 1; 1 1 0 1 1; 1 1 0 1 1];

153

0; 0 0 1 1 0; 0 0 1 1; 0 0 1 1 1; 0 0 1 0; 0 1 0 0 0; 0 1 0 1; 0 1 0 0 1; 0 1 0 0; 0 1 0 1 0; 0 1 0 1; 0 1 0 1 1; 0 1 0 0; 0 1 1 0 0; 0 1 1 1; 0 1 1 0 1; 0 1 1 0; 0 1 1 1 0; 0 1 1 1; 0 1 1 1 1; 0 1 1 0; 1 0 0 0 0; 1 0 0 1; 1 0 0 0 1; 1 0 0 0; 1 0 0 1 0; 1 0 0 1; 1 0 0 1 1; 1 0 0 0; 1 0 1 0 0; 1 0 1 1; 1 0 1 0 1; 1 0 1 0; 1 0 1 1 0; 1 0 1 1; 1 0 1 1 1; 1 0 1 0; 1 1 0 0 0; 1 1 0 1; 1 1 0 0 1; 1 1 0 0; 1 1 0 1 0; 1 1 0 1; 1 1 0 1 1; 1 1 0

154

MicroARTMAP MATLAB Implementation

a28 = [1 1 1 0 0; 1 1 1 0 0; 1 1 1 0 0; 0 0; 1 1 1 0 0; 1 1 1 0 0; 1 1 1 0 0]; a29 = [1 1 1 0 1; 1 1 1 0 1; 1 1 1 0 1; 0 1; 1 1 1 0 1; 1 1 1 0 1; 1 1 1 0 1]; a30 = [1 1 1 1 0; 1 1 1 1 0; 1 1 1 1 0; 1 0; 1 1 1 1 0; 1 1 1 1 0; 1 1 1 1 0]; OUT = [a1’ a2’ a3’ a4’ a5’ a6’ a7’ a8’ a17’ a18’ a19’ a20’ a21’ a22’ a23’ a24’

1 1 1 0 0; 1 1 1 0 0; 1 1 1 0 0; 1 1 1 1 1 1 0 1; 1 1 1 0 1; 1 1 1 0 1; 1 1 1 1 1 1 1 0; 1 1 1 1 0; 1 1 1 1 0; 1 1 1 a9’ a10’ a11’ a12’ a13’ a14’ a15’ a16’ a25’ a26’ a27’ a28’ a29’ a30’]’

tic TAG=(‘SALO1’); PAR=[1.0 0.0 1.0 1.0 0 0.08 0.002]’; [Mi Ni]=size(INP); [Mo No]=size(OUT); UnitsA=3; UnitsB=3; WA=ones(UnitsA, 2*Ni); WB=ones(UnitsB, 2*No); WAB=(ones(UnitsA,UnitsB)); MICRO(‘create’, TAG, PAR, WA, WB, WAB) LIST=MICRO(‘list’) MICRO(‘train’, TAG, INP, OUT) OUT=MICRO(‘test’, TAG , INP); for i=1:4 INP1=input(‘enter the leter code’); OUT1=MICRO(‘test’, TAG , INP1) end toc

Appendix B: DWT on db1 Wavelets - number.m

Inputs to MicroArtMap using DWT on db1 wavelet on handwritten numbers zer=[1 0 1 1 1 1 0 0 1 1 0 0 1 1 1 0 1 1 0 0 0 0 1 0 1 0 0 0 1 0 1 1 0 0 0 0 1 0 0 1 1 0 0 1 1 0 0 0 1 1 1 1 0 0; 1 1 1 1 1 1 0 0 1 1 0 0 1 1 1 0 1 1 0 0 0 0 1 0 1 0 0 0 1 0 1 1 0 0 0 0 1 0 0 1 1 0 0 1 1 0 0 0 1 1 1 1 0 0; 1 0 0 1 1 1 0 0 1 1 0 0 1 1 1 0 1 1 0 0 0 0 1 0 1 0 0 0 1 0 1 1 0 0 0 0 1 0 0 1 1 0 0 1 1 0 0 0 1 1 1 1 0 0; 1 0 1 0 1 1 0 0 1 1 0 0 1 1 1 0 1 1 0 0 0 0 1 0 1 0 0 0 1 0 1 1 0 0 0 0 1 0 0 1 1 0 0 1 1 0 0 0 1 1 1 1 0 0; 1 0 1 1 0 1 0 0 1 1 0 0 1 1 1 0 1 1 0 0 0 0 1 0 1 0 0 0 1 0 1 1 0 0 0 0 1 0 0 1 1 0 0 1 1 0 0 0 1 1 1 1 0 0];

0 0 0 0 1 0 1 0 0 0

one=[0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0; 0 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0; 0 0 1 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0; 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0; 0 0 0 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0;]

0 1 0 0 0 0 0 0 0 1

two=[ 0 1 1 1 1 1 1 1 0 1 1 0 0 0 1 0 0 0 1 0 0 1 1 1 1 1 1 0 1 1 0 0 0 1 0 0 0 1 1 0 1 0 1 1 1 1 1 0 1 1 0 0 0 1 0 0 0 1 1 0 1 1 0 1 1 1 1 0

0 1 1 0 1 0 1

1 0 1 0 1 0 1

1 0 1 1 1 1 1

1 1 1 1 1 1 1

1 1 1 1 1 1 1

1 1 1 0 1 0 1

1 0 1 1 1 1 1

1 1 0 1 0 1 0

0 1 1 0 1 0 1

1 0 1 0 1 0 1

1 0 0 0 0 0 0

0 0 0 1 0 1 0

0 1 0 1 0 1 0

0 1 0 1 0 1 0

0 0 0 1 0 1 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 1 0 1 0 0 0 0

1 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 1 0

0 0 0 0 1 1 0 0; 0 0 0 1 1 0 0; 0 0 0 1 1 0 0; 0 0 0 1

1 1 1 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0

V.K. David and S. Rajasekaran: Pattern Recog. Using Neural & Funct. Net., SCI 160, pp. 155–158. c Springer-Verlag Berlin Heidelberg 2009 springerlink.com

156

DWT on db1 Wavelets - number.m

1 1 0 0 0 1 0 0 0 1 1 0 0 1 1 1 0 1 1 0 0 0 1 1 1 1 0 0; 0 1 1 1 0 1 1 1 0 1 1 1 1 1 1 1 0 1 1 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 1 1 0 0 0 1 0 0 0 1 1 0 0 1 1 1 0 1 1 0 0 0 1 1 1 1 0 0;] thr=[0 1 1 1 1 1 1 0 0 1 1 0 1 1 1 0 0 0 0 0 1 1 1 0 0 0 0 1 1 1 0 0 0 0 0 0 1 1 0 1 1 1 1 1 1 1 0 0 1 1 1 1 1 0; 0 0 1 1 1 1 1 0 0 1 1 0 1 1 1 0 0 0 0 0 1 1 1 0 0 0 0 1 1 1 0 0 0 0 0 0 1 1 0 1 1 1 1 1 1 1 0 0 1 1 1 1 1 0; 0 1 0 1 1 1 1 0 0 1 1 0 1 1 1 0 0 0 0 0 1 1 1 0 0 0 0 1 1 1 0 0 0 0 0 0 1 1 0 1 1 1 1 1 1 1 0 0 1 1 1 1 1 0; 0 1 1 0 1 1 1 0 0 1 1 0 1 1 1 0 0 0 0 0 1 1 1 0 0 0 0 1 1 1 0 0 0 0 0 0 1 1 0 1 1 1 1 1 1 1 0 0 1 1 1 1 1 0; 0 1 1 1 0 1 1 0 0 1 1 0 1 1 1 0 0 0 0 0 1 1 1 0 0 0 0 1 1 1 0 0 0 0 0 0 1 1 0 1 1 1 1 1 1 1 0 0 1 1 1 1 1 0];

0 1 1 1 1 0 0 0 0 0

fou=[ 0 1 1 1 1 1 1 0 0 1 1 0 1 1 1 0 0 0 0 0 1 1 1 0 0 0 0 1 1 1 0 0 0 0 0 0 1 1 0 1 1 1 1 1 1 1 0 0 1 1 1 1 1 0; 0 0 1 1 1 1 1 0 0 1 1 0 1 1 1 0 0 0 0 0 1 1 1 0 0 0 0 1 1 1 0 0 0 0 0 0 1 1 0 1 1 1 1 1 1 1 0 0 1 1 1 1 1 0; 0 1 0 1 1 1 1 0 0 1 1 0 1 1 1 0 0 0 0 0 1 1 1 0 0 0 0 1 1 1 0 0 0 0 0 0 1 1 0 1 1 1 1 1 1 1 0 0 1 1 1 1 1 0; 0 1 1 0 1 1 1 0 0 1 1 0 1 1 1 0 0 0 0 0 1 1 1 0 0 0 0 1 1 1 0 0 0 0 0 0 1 1 0 1 1 1 1 1 1 1 0 0 1 1 1 1 1 0; 0 1 1 1 0 1 1 0 0 1 1 0 1 1 1 0 0 0 0 0 1 1 1 0 0 0 0 1 1 1 0 0 0 0 0 0 1 1 0 1 1 1 1 1 1 1 0 0 1 1 1 1 1 0];

0 1 1 1 1 0 0 0 0 0

1 1 1 1 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0

1 1 1 1 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0

fiv=[0 1 1 1 1 1 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 1 1 0 0 0 1 1 1 0 0 1 1 0 0 0 0 0 0 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1; 0 0 1 1 1 1 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 1 1 0 0 0 0 1 1 0 0 1 1 0 0 0 0 0 0 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1; 0 1 0 1 1 1 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 1 1 0 0 0 0 1 1 0 0 1 1 0 0 0 0 0 0 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1; 0 1 1 0 1 1 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 1 1 0 0 0 0 1 1 0 0 1 1 0 0 0 0 0 0 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1; 0 1 1 1 0 1 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 1 1 0 0 0 0 1 1 0 0 1 1 0 0 0 0 0 0 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1];

0 0 0 1 1 0 0 1 1 1

six=[0 0 1 1 1 1 1 0 0 1 1 1 0 1 1 0 0 1 1 1 0 0 1 0 0 1 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 1 1 0 0; 0 1 1 1 1 1 1 0 0 1 1 1 0 1 1 0 0 1 1 1 0 0 1 0 0 1 1 0 0 0 0 0 1 1 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 1 1 0 0; 0 0 0 1 1 1 1 0 0 1 1 1 0 1 1 0 0 1 1 1 0 0 1 0 0 1 1 0 0 0 0 0 1 1 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 1 1 0 0; 0 0 1 0 1 1 1 0 0 1 1 1 0 1 1 0 0 1 1 1 0 0 1 0 0 1 1 0 0 0 0 0 1 1 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 1 1 0 0;

1 1 1 1 1 0 0 1 1 0

0 0 1 1 0 0 1 1 1 1 0 0 1 1 0 0 1 1 1 1 0 0 1 1 0 0 1 1 1 1 0 0 1 1 0 0 1 1 1 1

1 1 1 1 0 0 1 1 0 0 1 1 1 1 0 0 1 1 0 0 1 1 1 1 0 0 1 1 0 0

DWT on db1 Wavelets - number.m

157

0 0 1 1 0 1 1 0 0 1 1 1 0 1 1 0 0 1 1 1 0 0 1 0 0 1 1 1 1 1 1 0 0 1 1 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 1 1 0 0]; sev=[0 0 1 1 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 1 1 0 0 0 0 0 1 1 0 0 0 0 0 0 0 1 1 0 0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0; 0 1 1 1 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 1 0 0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0; 0 0 0 1 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 1 0 0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0; 0 0 1 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 1 0 0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0; 0 0 1 1 1 0 0 0 0 0 1 1 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 1 0 0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0];

0 0 1 1 0 0 0 0 0 0

eig=[0 1 1 1 1 1 0 0 0 1 1 1 0 1 0 0 0 1 1 0 0 1 0 0 0 0 1 1 0 0 0 1 1 0 1 1 0 0 0 0 1 1 0 1 0 0 0 0 1 1 1 1 0 0; 0 0 1 1 1 1 0 0 0 1 1 1 0 1 0 0 0 1 1 0 0 1 0 0 0 0 1 1 0 0 0 1 1 0 1 1 0 0 0 0 1 1 0 1 0 0 0 0 1 1 1 1 0 0; 0 1 0 1 1 1 0 0 0 1 1 1 0 1 0 0 0 1 1 0 0 1 0 0 0 0 1 1 0 0 0 1 1 0 1 1 0 0 0 0 1 1 0 1 0 0 0 0 1 1 1 1 0 0; 0 1 1 0 1 1 0 0 0 1 1 1 0 1 0 0 0 1 1 0 0 1 0 0 0 0 1 1 0 0 0 1 1 0 1 1 0 0 0 0 1 1 0 1 0 0 0 0 1 1 1 1 0 0; 0 1 1 1 0 1 0 0 0 1 1 1 0 1 0 0 0 1 1 0 0 1 0 0 0 0 1 1 0 0 0 1 1 0 1 1 0 0 0 0 1 1 0 1 0 0 0 0 1 1 1 1 0 0];

1 1 1 1 0 0 0 0 1 1

nin=[0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 1 1 0 0 0 1 1 0 0 1 0 0 0 0 1 1 1 1 0 0 0 0 0 1 1 0 0 0; 0 1 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 1 1 0 0 0 1 1 0 0 1 0 0 0 0 1 1 1 1 0 0 0 0 0 1 1 0 0 0; 0 0 1 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 1 1 0 0 0 1 1 0 0 1 0 0 0 0 1 1 1 1 0 0 0 0 0 1 1 0 0 0; 0 0 0 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 1 1 0 0 0 1 1 0 0 1 0 0 0 0 1 1 1 1 0 0 0 0 0 1 1 0 0 0; 0 0 0 0 1 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 1 1 0 0 0 1 1 0 0 1 0 0 0 0 1 1 1 1 0 0 0 0 0 1 1 0 0 0];

1 1 1 1 0 0 0 1 1 0

INP=[zer’ one’ two’ thr’ fou’ fiv’ six’ sev’ eig’ nin’]’ for i=1:50 Y=INP(i,1:64); [ca1 cd]=dwt(Y,‘db1’); [ca2 cd]=dwt(ca1,‘db1’); [ca3 cd]=dwt(ca2,‘db1’); INP1(i,1:8)=ca3; end

0 1 1 0 0 0 0 0 0 1 0 1 1 0 0 0 0 0 0 1 0 1 1 0 0 0 0 0 0 1 0 1 1 0 0 0 0 0 0 1

1 1 1 0 0 0 0 1 1 1 1 1 1 0 0 0 0 1 1 1 1 1 1 0 0 0 0 1 1 1 1 1 1 0 0 0 0 1 1 1

1 1 1 0 0 0 1 1 0 1 1 1 1 0 0 0 1 1 0 1 1 1 1 0 0 0 1 1 0 1 1 1 1 0 0 0 1 1 0 1

158

DWT on db1 Wavelets - number.m

zer1=[0 0 0 0 1; 0 0 0 0 1; 0 0 0 0 1; 0 0 0 0 1; 0 0 0 0 1]; one1=[0 0 0 1 0; 0 0 0 1 0; 0 0 0 1 0; 0 0 0 1 0; 0 0 0 1 0]; two1=[0 0 0 1 1; 0 0 0 1 1; 0 0 0 1 1; 0 0 0 1 1; 0 0 0 1 1]; thr1=[0 0 1 0 0; 0 0 1 0 0; 0 0 1 0 0; 0 0 1 0 0; 0 0 1 0 0]; fou1=[0 0 1 0 1; 0 0 1 0 1; 0 0 1 0 1; 0 0 1 0 1; 0 0 1 0 1]; fiv1=[0 0 1 1 0; 0 0 1 1 0; 0 0 1 1 0; 0 0 1 1 0; 0 0 1 1 0]; six1=[0 0 1 1 1; 0 0 1 1 1; 0 0 1 1 1; 0 0 1 1 1; 0 0 1 1 1]; sev1=[0 1 0 0 0; 0 1 0 0 0; 0 1 0 0 0; 0 1 0 0 0; 0 1 0 0 0]; eig1=[0 1 0 0 1; 0 1 0 0 1; 0 1 0 0 1; 0 1 0 0 1; 0 1 0 0 1]; nin1=[0 1 0 1 0; 0 1 0 1 0; 0 1 0 1 0; 0 1 0 1 0; 0 1 0 1 0]; OUT=[zer1’ one1’ two1’ thr1’ fou1’ fiv1’ six1’ sev1’ eig1’ nin1’]’ tic TAG=(‘SALO1’); PAR=[1.0 0.0 1.0 1.0 0 0.08 0.002]’; [Mi Ni]=size(INP1); [Mo No]=size(OUT); UnitsA=3; UnitsB=3; WA=ones(UnitsA, 2*Ni); WB=ones(UnitsB, 2*No); WAB=(ones(UnitsA,UnitsB)); MICRO(‘create’, TAG, PAR, WA, WB, WAB) LIST=MICRO(‘list’) MICRO(‘train’, TAG, INP1, OUT) OUT=MICRO(‘test’, TAG , INP1) INP2=[1.7678 1.7678 1.0607 0.7071 0.7071 1.0607 1.4142 1.3]; OUT1=MICRO(‘test’, TAG , INP2) toc

Appendix C: Inputs to ARTMAP for Signatures

Eight features obtained for each sample signature by Block Processing I=imread(‘E:\sample of vasantha(wavwmenu)\sample n\n1’); a=imresize(I,[200,200],’nearest’); imshow(I); figure; imshow(a); BW = im2bw(I,0.8); imshow(BW); i1=BW(1:100,1:50); i2=BW(1:100,51:100); i3=BW(1:100,101:150); i4=BW(1:100,151:200); i5=BW(101:200,1:50); i6=BW(101:200,51:100); i7=BW(101:200,101:150); i8=BW(101:200,151:200); figure; imshow(i1); figure; imshow(i2); figure; imshow(i3); figure; imshow(i4); figure; imshow(i5); figure; imshow(i6); figure; imshow(i7); figure; imshow(i8); c1=0;c2=0;c3=0;c4=0;c5=0;c6=0;c7=0;c8=0; V.K. David and S. Rajasekaran: Pattern Recog. Using Neural & Funct. Net., SCI 160, pp. 159–162. c Springer-Verlag Berlin Heidelberg 2009 springerlink.com

160

Inputs to ARTMAP for Signatures

for i=1:100 for j=1:50 if i1(i,j)==0 c1=c1+1; end end end for i=1:100 for j=1:50 if i2(i,j)==0 c2=c2+1; end end end for i=1:100 for j=1:50 if i3(i,j)==0 c3=c3+1; end end end for i=1:100 for j=1:50 if i4(i,j)==0 c4=c4+1; end end end for i=1:100 for j=1:50 if i5(i,j)==0 c5=c1+1; end end end for i=1:100 for j=1:50 if i6(i,j)==0 c6=c2+1; end end end for i=1:100 for j=1:50 if i7(i,j)==0

Inputs to ARTMAP for Signatures

c7=c3+1; end end end for i=1:100 for j=1:50 if i8(i,j)==0 c8=c4+1; end end end c1 c2 c3 c4 c5 c6 c7 c8 Inputs to MicroARTMAP by Block Processing for Signatures a = [151 200 601 628 152 201 602 629]; b = [ 0 0 469 518 1 1 470 0 ]; c = [3 3 511 764 4 4 512 768]; d = [ 0 0 402 450 1 1 403 451]; e = [150 187 717 578 151 188 718 579]; f = [0 18 590 517 1 19 591 0]; g = [ 82 52 621 893 83 53 622 894]; h = [ 1 108 668 611 2 109 669 612]; INP = [a’ b’ c’ d’ e’ f’ g’ h’]’; a1 = [0 0 0 0 1]; b1 = [0 0 0 1 0]; c1 = [0 0 0 1 1]; d1 = [0 0 1 0 0]; e1 = [0 0 1 0 1]; f1 = [0 0 1 1 0]; g1 = [0 0 1 1 1]; h1 = [0 1 0 0 0]; OUT = [a1’ b1’ c1’ d1’ e1’ f1’ g1’ h1’]’; tic TAG=(‘SALO1’); PAR=[1.0 0.0 1.0 1.0 0 0.08 0.002]’;

161

162

Inputs to ARTMAP for Signatures

[Mi Ni]=size(INP); [Mo No]=size(OUT); UnitsA=3; UnitsB=3; WA=ones(UnitsA, 2*Ni); WB=ones(UnitsB, 2*No); WAB=(ones(UnitsA,UnitsB)); MICRO(‘create’, TAG, PAR, WA, WB, WAB) LIST=MICRO(‘list’) MICRO(‘train’, TAG, INP, OUT) OUT=MICRO(‘test’, TAG , INP); INP1=[1 108 668 611 2 109 669 612]; OUT1=MICRO(‘test’, TAG , INP1) toc

Appendix D: The Competitive Hopfield Neural Network

# include <stdio.h> # include <math.h> # include # include # include <stdlib.h> # include <dos.h> # include <string.h>

/* STEP 1 - INITIAL VALUE SETTING*/ int c2 = 4, c3 = 1, c5 = 3, c6 = 1; /*resource request matrix (Table 6)*/ int r[5][4] = {{1, 0, 0, 0},{0, 1, 0, 0},{0, 0, 1, 0},{0, 0, 0, 1},{1, 0, 0, 1}}; /* timing constraint matrix (Table 7)*/ int T1[5][2]= {{2, 3},{5, 8},{3, 4},{4, 8},{2, 5}}; /*initial state matrix (Table 8)*/ /*int V[2][5][8] ={{ {1, 1, 1, 0, 1, 0, 0, 0, 1, 0}, {1, 2, 0, 1, 0, 1, 0, 1, 0, 0}, {1, 3, 1, 0, 0, 0, 1, 1, 1, 1}, {1, 4, 0, 1, 1, 0, 1, 0, 0, 0}, {1, 5, 0, 1, 0, 1, 0, 1, 0, 1}, {2, 1, 1, 0, 1, 0, 1, 0, 1, 1}, {2, 2, 1, 0, 1, 0, 0, 1, 0, 0}, {2, 3, 0, 1, 0, 1, 0, 1, 0, 1}, {2, 4, 1, 0, 0, 0, 0, 0, 1, 0}, {2, 5, 0, 0, 0, 0, 1, 1, 1, 1} }}; */ V.K. David and S. Rajasekaran: Pattern Recog. Using Neural & Funct. Net., SCI 160, pp. 163–168. c Springer-Verlag Berlin Heidelberg 2009 springerlink.com

164

The Competitive Hopfield Neural Network

int V[2][10][8] ={ { {1, 0, 1, 0, 0, 0, 1, 0} {0, 1, 0, 1, 0, 1, 0, 0} {1, 0, 0, 0, 1, 1, 1, 1} {0, 1, 1, 0, 1, 0, 0, 0} {0, 1, 0, 1, 0, 1, 0, 1} {1, 0, 1, 0, 1, 0, 1, 1} {1, 0, 1, 0, 0, 1, 0, 0} {0, 1, 0, 1, 0, 1, 0, 1} {1, 0, 0, 0, 0, 0, 1, 0} {0, 0, 0, 0, 1, 1, 1, 1} }}; int RxRi[10][10], Wxyzijl[5][5][5][5][5][5]; /*N - Jobs, M - Machines, T - Time, F - resource*/ int N, M, T, F; int x, i, s, y, z, j, k, l, yy, zz, i1, j1, k1,S; int resource, sum; float ec2, ec5, ec6, ec3, EA,Theta[5][5][5]; int result[5][5][5], G[5][5][5]; int deltax , deltax y , deltax z ; int Net[5][5][5], Netxyz[5][5][5], Max; /* ENERGY*/ int energy() { ec2=0; ec5=0; ec6=0; ec3=0; /* ec2 calculation*/ for(i=1;i<=N;i++) for(j=1;j<=M;j++) for (k=1;k<=T;k++) for(j1=1;j1<=M;j1++) for(k1=1;k1<=T;k++) if (j1 <= j) ec2 = ec2 + V[i][j][k] * V[i][j1][k1]; /* ec5 calculation */ for (i=1;i<=N;i++)

The Competitive Hopfield Neural Network

for(j=1;j<=M;j++) for (k=1;k<=T;k++) ec5 = ec5 + V[i][j][k] * (G[i][j][k] * G[i][j][k]) * result[i][j][k]; /* ec6 calculation */ for (i=1;i<=N;i++) for(j=1;j<=M;j++) for (k=1;k<=T;k++) for (i1=1;i1<=N;i1++) for (j1=1;j1<=M;j1++) for (S=1;S<=F;S++) if ((i1 <= i) && (j1 <= j)) ec6 = ec6 + V[i][j][k] * r[i][S] * V[i1][j1][k] * r[i1][S]; /* ec3 calculation */ for(i=1;i<=N;i++) { sum =0; for(j=1;j<=M;j++) for (k=1;k<=T;k++) sum = sum + V[i][j][k]; ec3 = ec3 + (sum-T1[i][1])*(sum-T1[i][1]); } ec2 = (c2/2) * ec2; ec5 = (c5/2) * ec5; ec6 = (c6/2) * ec6; ec3 = (c3/2) * ec3; EA = ec2 + ec3 + ec5 + ec6; return EA; } /*DELTA FUNCTION */ int delta(int b, int c) { int a; if (b == c) a = 1; else a = 0; return a; }

165

166

The Competitive Hopfield Neural Network

/*HEVISIDE FUNCTION */ int Hevi(b) { int a; if (b > 1) a = 1; else a = 0; return a; } void main() { printf("Enter No. of jobs"); scanf("%d",&N); printf("Enter No. of Machines"); scanf("%d",&M); printf("Enter No. of resources"); scanf("%d",&F); printf("Enter Time"); scanf("%d",&T); /* calculation of weight and theta matrix starts here(step 2)*/ /*the following loops computes the resource RxRi*/ for(x=1;x<=N;x++) { for (i=1;i<=N;i++) { resource=0; for(S=1;S<=F;S++) { RxRi[x][i] = resource + r[x][S] * r[i][S]; resource = RxRi[x][i]; } } } /* CALCULATION OF WEIGHT MATRIX */ for(x=1;x<=N;x++) { for(y=1;y<<=M;y++) { for(z=1;z<<=T;z++) { for(i=1;i<<=N;i++)

The Competitive Hopfield Neural Network

{ for(j=1;j<<<=M;j++) { for(l=1;l<<=T;l++) { delta_x = delta(x,i); delta_y = delta(y,j); delta_z = delta(z,k); } Wxyzijl[x][y][z][i][j][l] = -c2*delta_x * (1- delta_y) - c3 * delta_x - c6*(1-delta_x)*(1-delta_y) * delta_z * RxRi[x][i]; printf("Wxyzijk value is %d", Wxyzijl[x][y][z][i][j][l]); printf("press any key ..."); getche(); } } } } } /* CALCULATION OF THETA(x, y, z) - equation 7 */ for (y=1;y<=M;y++) { for (x=1;x<=N;x++) { for(z=1;z<=T;z++) { G[x][y][z]=z-T1[x][2]+1; /* T1(x,2) is deadline of jobs*/ result[x][y][z] = Hevi(G[x][y][z]); /* heavside function */ Theta[x][y][z]= c3*T1[x][1]+(c5/2)*(G[x][y][z]*G[x][y][z])* result[x][y][z]; printf("Theta[x][y][z] %d= ",Theta[x][y][z]); printf("press any key ..."); getche(); } } } /* STEP 3 recalling */ /* initial energy value based on initial neuron states*/ EA = energy(); /* calculate net output eqn 4 and adjust neuron state*/ while ( abs(EA)<.0000001) { /* while loop starts here */ for (yy=1;yy<=M;y++)

167

168

The Competitive Hopfield Neural Network

for (zz=1;zz<=T;z++) { /* loop of zz starts here */ for(x=1;x<=N;x++) for(y=1;y<=M;y++) for(z=1;z<=T;z++) { Net[x][y][z] = 0; Netxyz[x][y][z] = 0; for(j=1;j<=M;j++) for(i=1;i<=N;i++) for(l=1;l<=T;l++) Netxyz[x][y][z] = Wxyzijl[x][y][z][i][j][l] * V[i][j][l] + Netxyz[x][y][z]; Net[x][y][z] = Netxyz[x][y][z] - Theta[x][y][z]; } /* PICK LARGEST NET VALUE */ Max=1; /* always 1*/ for(x=2;x<=N;x++) if(Net[Max][yy][zz]
Appendix E: Moment Invariants for Handwritten Characters

The characteristics obtained for samples using the program HPGMomout.m φ1

φ2

φ3

φ4

φ5

φ6

φ7

.49175E+00 .50320E+00

.13667E+00 .14848E+00

.29460E-01 .37433E-01

.15069E+00 .16274E+00

.13739E-01 .17352E-01

.55703E-01 .62808E-01

-.17347E-17 .22835E-02

.49017E+00

.14221E+00

.36228E-01

.15100E+00

.14939E-01

.56930E-01

.13444E-02

.48652E+00 .50320E+00

.13733E+00 .14848E+00

.29446E-01 .37433E-01

.14694E+00 .16274E+00

.13640E-01 .17352E-01

.54421E-01 .62808E-01

.28189E-17 -.22835E-02

.47823E+00 .51654E+00

.13061E+00 .15068E+00

.27640E-01 .34829E-01

.13883E+00 .17461E+00

.11980E-01 .18480E-01

.50226E-01 .67780E-01

.71265E-03 .28790E-03

.46086E+00

.12242E+00

.26619E-01

.12471E+00

.96889E-02

.43607E-01

-.12143E-16

.47823E+00 .52128E+00

.13061E+00 .15567E+00

.27640E-01 .40689E-01

.13883E+00 .18011E+00

.11980E-01 .20236E-01

.50226E-01 .71057E-01

-.71265E-03 -.13573E-02

.47844E+00

.16473E+00

.55689E-01

.14866E+00

.18808E-01

.60172E-01

.00000E+00

.48329E+00 .49262E+00

.16441E+00 .16460E+00

.54477E-01 .51448E-01

.15207E+00 .15941E+00

.19304E-01 .20118E-01

.61551E-01 .64518E-01

-.34543E-02 -.13417E-02

.49418E+00 .48796E+00

.16467E+00 .16446E+00

.51349E-01 .53943E-01

.16068E+00 .15569E+00

.20242E-01 .19676E-01

.65033E-01 .63005E-01

.95577E-03 .31332E-02

.47561E+00

.16269E+00

.55137E-01

.14588E+00

.18239E-01

.58735E-01

-.36103E-02

.50273E+00 .47844E+00

.18440E+00 .16473E+00

.67334E-01 .55689E-01

.17317E+00 .14866E+00

.26059E-01 .18808E-01

.74183E-01 .60172E-01

-.28948E-02 .00000E+00

.44624E+00

.14686E+00

.48145E-01

.12159E+00

.12960E-01

.46458E-01

-.42772E-03

.49340E+00 .81576E+00

.18460E+00 .56646E+00

.70760E-01 .41498E+00

.16567E+00 .78047E+00

.24933E-01 .62240E+00

.71007E-01 .58328E+00

.46357E-02 .24980E-15

.77632E+00 .86265E+00

.52787E+00 .57994E+00

.37902E+00 .41078E+00

.67917E+00 .89727E+00

.48605E+00 .75618E+00

.49044E+00 .67782E+00

.61755E-01 -.63833E-01

.86343E+00

.57973E+00

.40304E+00

.89931E+00

.75174E+00

.67882E+00

.58997E-01

.84414E+00 .77632E+00

.58579E+00 .52787E+00

.42931E+00 .37902E+00

.85444E+00 .67917E+00

.72283E+00 .48605E+00

.64915E+00 .49044E+00

.18574E+00 .61755E-01

.86265E+00

.57994E+00

.41078E+00

.89727E+00

.75618E+00

.67782E+00

-.63833E-01

.86343E+00 .84414E+00

.57973E+00 .58579E+00

.40304E+00 .42931E+00

.89931E+00 .85444E+00

.75174E+00 .72283E+00

.67882E+00 .64915E+00

.58997E-01 .18574E+00

.79964E+00 .89198E+00

.52625E+00 .69288E+00

.37300E+00 .57509E+00

.72515E+00 .10280E+01

.52706E+00 .11149E+01

.52359E+00 .85092E+00

-.14237E+00 -.13677E+00

V.K. David and S. Rajasekaran: Pattern Recog. Using Neural & Funct. Net., SCI 160, pp. 169–179. c Springer-Verlag Berlin Heidelberg 2009 springerlink.com

170

Moment Invariants for Handwritten Characters

.73162E+00

.46798E+00

.31636E+00

.56895E+00

.33975E+00

.38619E+00

.68783E-02

.73113E+00

.46806E+00

.31670E+00

.56767E+00

.33907E+00

.38586E+00

-.14218E-01

.74328E+00 .60128E+00

.46617E+00 .25114E+00

.30882E+00 .10084E+00

.58817E+00 .29189E+00

.35212E+00 .69518E-01

.39946E+00 .14592E+00

-.40686E-01 .17347E-16

.61393E+00

.25680E+00

.10224E+00

.30864E+00

.76458E-01

.15620E+00

-.14023E-01

.62641E+00 .62912E+00

.25779E+00 .25810E+00

.96425E-01 .96162E-01

.32470E+00 .32832E+00

.80127E-01 .80940E-01

.16452E+00 .16642E+00

-.55759E-02 .33134E-02

.62207E+00

.25737E+00

.10081E+00

.31902E+00

.78779E-01

.16155E+00

.11735E-01

.58904E+00 .63835E+00

.24247E+00 .28736E+00

.96952E-01 .12662E+00

.27471E+00 .35085E+00

.62494E-01 .10284E+00

.13505E+00 .18764E+00

-.11489E-01 -.11993E-01

.55444E+00 .55949E+00

.21877E+00 .21933E+00

.84470E-01 .83267E-01

.23064E+00 .23584E+00

.44752E-01 .45995E-01

.10758E+00 .11016E+00

-.12884E-02 -.36919E-02

.62967E+00

.28709E+00

.13147E+00

.33937E+00

.99291E-01

.18144E+00

.17772E-01

.53850E+00 .54938E+00

.23888E+00 .24361E+00

.10981E+00 .11338E+00

.22171E+00 .23378E+00

.48501E-01 .53274E-01

.10783E+00 .11483E+00

.97145E-16 -.79068E-02

.55876E+00

.24220E+00

.10748E+00

.24292E+00

.54806E-01

.11893E+00

-.31033E-02

.55944E+00 .55144E+00

.24211E+00 .24327E+00

.10659E+00 .11083E+00

.24363E+00 .23573E+00

.54767E-01 .53252E-01

.11923E+00 .11570E+00

.29739E-02 .91064E-02

.53475E+00 .57110E+00

.24660E+00 .27280E+00

.12095E+00 .13606E+00

.22009E+00 .26577E+00

.50463E-01 .71100E-01

.10899E+00 .13823E+00

.14056E-01 -.64409E-02

.50028E+00

.21024E+00

.91887E-01

.17906E+00

.32271E-01

.81669E-01

.55492E-03

.49978E+00 .50561E+00

.21028E+00 .20980E+00

.92056E-01 .90333E-01

.17860E+00 .18322E+00

.32206E-01 .33103E-01

.81516E-01 .83584E-01

-.10775E-02 -.29418E-02

.53472E+00

.20262E+00

.92534E-01

.20414E+00

.38256E-01

.92844E-01

-.23469E-01

.56446E+00 .57661E+00

.23453E+00 .22978E+00

.12591E+00 .11233E+00

.24034E+00 .25285E+00

.58655E-01 .58213E-01

.11894E+00 .12312E+00

-.36004E-01 -.31285E-01

.57418E+00 .55718E+00

.22243E+00 .21296E+00

.10160E+00 .93616E-01

.24919E+00 .22976E+00

.52953E-01 .44568E-01

.11869E+00 .10655E+00

-.27686E-01 -.24037E-01

.52559E+00

.20471E+00

.90596E-01

.19813E+00

.35786E-01

.89794E-01

-.18932E-01

.58925E+00 .47841E+00

.25688E+00 .16286E+00

.14029E+00 .66910E-01

.27524E+00 .14682E+00

.75056E-01 .19810E-01

.14176E+00 .59709E-01

-.47279E-01 -.13032E-01

.48003E+00

.16505E+00

.69729E-01

.14831E+00

.20752E-01

.60890E-01

-.13439E-01

.49143E+00 .64691E+00

.16920E+00 .33424E+00

.72468E-01 .17825E+00

.15773E+00 .38051E+00

.23202E-01 .13910E+00

.65738E-01 .21913E+00

-.13819E-01 .87723E-02

.63205E+00 .66602E+00

.32932E+00 .32396E+00

.17992E+00 .15835E+00

.35777E+00 .40455E+00

.12761E+00 .14290E+00

.20489E+00 .22909E+00

.30829E-01 -.10370E-01

.66942E+00

.32465E+00

.15579E+00

.40970E+00

.14467E+00

.23227E+00

.17046E-01

.65824E+00 .64247E+00

.32718E+00 .32726E+00

.16708E+00 .17455E+00

.39366E+00 .37123E+00

.14179E+00 .13192E+00

.22448E+00 .21178E+00

.43035E-01 -.26864E-01

.68643E+00

.38530E+00

.22716E+00

.45912E+00

.21001E+00

.28381E+00

-.25579E-01

.59429E+00 .59201E+00

.29012E+00 .28990E+00

.14734E+00 .14758E+00

.29745E+00 .29474E+00

.87413E-01 .86338E-01

.15964E+00 .15816E+00

.10266E-01 .33665E-02

.59950E+00 .53902E+00

.28909E+00 .16358E+00

.14408E+00 .33180E-01

.30316E+00 .19826E+00

.88770E-01 .23619E-01

.16250E+00 .80183E-01

-.39003E-02 .62884E-17

Moment Invariants for Handwritten Characters

171

.53570E+00

.16635E+00

.40896E-01

.19567E+00

.24633E-01

.79983E-01

.26074E-02

.52035E+00

.15349E+00

.31315E-01

.17865E+00

.19448E-01

.69997E-01

.49542E-03

.51646E+00 .52035E+00

.15124E+00 .15349E+00

.30375E-01 .31315E-01

.17475E+00 .17865E+00

.18562E-01 .19448E-01

.67937E-01 .69997E-01

-.34694E-17 -.49542E-03

.53570E+00

.16635E+00

.40896E-01

.19567E+00

.24633E-01

.79983E-01

-.26074E-02

.56337E+00 .50091E+00

.17981E+00 .14130E+00

.40355E-01 .26786E-01

.22657E+00 .15916E+00

.31431E-01 .15237E-01

.96147E-01 .59813E-01

.62226E-03 .47250E-04

.49703E+00

.13905E+00

.25998E-01

.15548E+00

.14511E-01

.57957E-01

-.12143E-16

.50091E+00 .72593E+00

.14130E+00 .67568E+00

.26786E-01 .59691E+00

.15916E+00 .64475E+00

.15237E-01 .58152E+00

.59813E-01 .52940E+00

-.47250E-04 .29143E-15

.73745E+00 .75931E+00

.74530E+00 .72382E+00

.70100E+00 .66336E+00

.69585E+00 .73139E+00

.71085E+00 .74225E+00

.60058E+00 .62155E+00

-.96497E-01 -.64862E-01

.76660E+00

.71714E+00

.65099E+00

.74383E+00

.75256E+00

.62896E+00

.24980E-15

.75931E+00 .73745E+00

.72382E+00 .74530E+00

.66336E+00 .70100E+00

.73139E+00 .69585E+00

.74225E+00 .71085E+00

.62155E+00 .60058E+00

.64862E-01 .96497E-01

.67795E+00

.55236E+00

.44112E+00

.51062E+00

.35181E+00

.37900E+00

.30219E-01

.66330E+00 .80305E+00

.56163E+00 .81042E+00

.45287E+00 .78387E+00

.49119E+00 .86594E+00

.33752E+00 .10390E+01

.36766E+00 .77851E+00

.12733E-01 -.36082E-15

.66330E+00 .75507E+00

.56163E+00 .77854E+00

.45287E+00 .77234E+00

.49119E+00 .74469E+00

.33752E+00 .83655E+00

.36766E+00 .65772E+00

-.12733E-01 -.19750E+00

.78858E+00

.93890E+00

.10323E+01

.88075E+00

.12479E+01

.85766E+00

-.40880E+00

.81404E+00 .81944E+00

.89942E+00 .88274E+00

.96553E+00 .92976E+00

.92517E+00 .93316E+00

.12848E+01 .12678E+01

.88020E+00 .87803E+00

-.40248E+00 -.33124E+00

.80478E+00

.88740E+00

.92440E+00

.90319E+00

.12057E+01

.85093E+00

-.21590E+00

.77006E+00 .68691E+00

.91736E+00 .59386E+00

.95038E+00 .51620E+00

.84066E+00 .54284E+00

.11059E+01 .42657E+00

.80403E+00 .41814E+00

-.81273E-01 -.85303E-01

.67088E+00 .86034E+00

.60823E+00 .99928E+00

.53541E+00 .11152E+01

.52200E+00 .10935E+01

.41353E+00 .17669E+01

.40718E+00 .10946E+01

-.90634E-01 -.43421E+00

.67671E+00

.61017E+00

.54534E+00

.53127E+00

.42988E+00

.41535E+00

-.11052E+00

.59242E+00 .59491E+00

.35781E+00 .33320E+00

.22487E+00 .20477E+00

.31969E+00 .31476E+00

.12196E+00 .11256E+00

.18963E+00 .17944E+00

-.69389E-17 -.16690E-01

.56884E+00

.35768E+00

.22826E+00

.29221E+00

.10839E+00

.17359E+00

-.21426E-02

.57613E+00 .56019E+00

.35546E+00 .35255E+00

.22683E+00 .23339E+00

.29949E+00 .27886E+00

.11187E+00 .10180E+00

.17740E+00 .16545E+00

-.14747E-01 .41659E-01

.62860E+00 .64506E+00

.34653E+00 .41887E+00

.21453E+00 .28643E+00

.36035E+00 .41098E+00

.13942E+00 .20145E+00

.21043E+00 .26381E+00

-.61747E-01 -.14097E-01

.53725E+00

.30532E+00

.17880E+00

.24207E+00

.72119E-01

.13271E+00

-.22027E-02

.64198E+00 .56446E+00

.42037E+00 .29825E+00

.28795E+00 .16770E+00

.40758E+00 .26731E+00

.19871E+00 .80146E-01

.26147E+00 .14506E+00

.33890E-01 -.17795E-01

.81159E+00

.65449E+00

.74029E+00

.79060E+00

.88875E+00

.65180E+00

.79900E+00

.72222E+00 .82562E+00

.43028E+00 .74857E+00

.42081E+00 .82120E+00

.51711E+00 .87398E+00

.33902E+00 .10803E+01

.34942E+00 .76185E+00

.26351E+00 .81931E+00

.83565E+00 .86265E+00

.71949E+00 .69018E+00

.72921E+00 .62899E+00

.89155E+00 .95117E+00

.10409E+01 .10525E+01

.75690E+00 .78523E+00

.71706E+00 .62908E+00

172

Moment Invariants for Handwritten Characters

.90664E+00

.66684E+00

.52785E+00

.10640E+01

.11112E+01

.85560E+00

.52656E+00

.82756E+00

.61248E+00

.71031E+00

.81365E+00

.90884E+00

.64777E+00

.85827E+00

.76466E+00 .77469E+00

.61456E+00 .59223E+00

.62343E+00 .55562E+00

.67852E+00 .69383E+00

.63877E+00 .61946E+00

.53843E+00 .53769E+00

.51441E+00 .46783E+00

.80170E+00

.56968E+00

.47798E+00

.74512E+00

.63194E+00

.56302E+00

.44419E+00

.57376E+00 .58420E+00

.18627E+00 .19418E+00

.41024E-01 .47181E-01

.24007E+00 .25341E+00

.34786E-01 .39592E-01

.10335E+00 .11156E+00

.16480E-16 -.14767E-02

.54481E+00

.17099E+00

.38516E-01

.20680E+00

.26612E-01

.85193E-01

.10875E-02

.54092E+00 .54481E+00

.16901E+00 .17099E+00

.37925E-01 .38516E-01

.20266E+00 .20680E+00

.25605E-01 .26612E-01

.82960E-01 .85193E-01

.29490E-16 -.10875E-02

.58420E+00 .60862E+00

.19418E+00 .21121E+00

.47181E-01 .51768E-01

.25341E+00 .28674E+00

.39592E-01 .50579E-01

.11156E+00 .13161E+00

.14767E-02 -.24347E-02

.62522E+00

.22064E+00

.52666E-01

.30995E+00

.57930E-01

.14543E+00

-.67508E-03

.52378E+00 .62522E+00

.15628E+00 .22064E+00

.32235E-01 .52666E-01

.18337E+00 .30995E+00

.20473E-01 .57930E-01

.72160E-01 .14543E+00

.00000E+00 .67508E-03

.52202E+00

.19586E+00

.72119E-01

.19291E+00

.31777E-01

.85197E-01

-.44309E-02

.53337E+00 .51442E+00

.18441E+00 .16358E+00

.62908E-01 .45956E-01

.19860E+00 .17630E+00

.30625E-01 .22084E-01

.85616E-01 .71395E-01

-.11046E-01 -.49529E-02

.50810E+00 .51442E+00

.15622E+00 .16126E+00

.39571E-01 .42080E-01

.16921E+00 .17600E+00

.19397E-01 .21140E-01

.66833E-01 .70591E-01

-.18258E-02 .86578E-03

.53337E+00

.17977E+00

.54950E-01

.19799E+00

.28444E-01

.83892E-01

.55177E-02

.53086E+00 .48283E+00

.21744E+00 .15582E+00

.92778E-01 .47450E-01

.20744E+00 .14923E+00

.40065E-01 .17536E-01

.96537E-01 .58863E-01

.73211E-02 -.39685E-02

.47651E+00

.14885E+00

.42209E-01

.14280E+00

.15492E-01

.54982E-01

-.19067E-02

.48283E+00 .62847E+00

.15428E+00 .26337E+00

.44980E-01 .10091E+00

.14903E+00 .32927E+00

.17042E-01 .83957E-01

.58390E-01 .16861E+00

-.26208E-03 .69389E-17

.61393E+00 .65086E+00

.25680E+00 .26641E+00

.10224E+00 .93698E-01

.30864E+00 .35959E+00

.76458E-01 .92699E-01

.15620E+00 .18536E+00

.14023E-01 -.11516E-01

.65719E+00

.26760E+00

.90400E-01

.36887E+00

.95025E-01

.19047E+00

.11449E-15

.65086E+00 .61393E+00

.26641E+00 .25680E+00

.93698E-01 .10224E+00

.35959E+00 .30864E+00

.92699E-01 .76458E-01

.18536E+00 .15620E+00

.11516E-01 -.14023E-01

.57919E+00

.22756E+00

.83375E-01

.25914E+00

.53250E-01

.12331E+00

.33343E-02

.57485E+00 .57919E+00

.22683E+00 .22756E+00

.84212E-01 .83375E-01

.25429E+00 .25914E+00

.52010E-01 .53250E-01

.12079E+00 .12331E+00

-.13878E-16 -.33343E-02

.66351E+00 .48203E+00

.30241E+00 .14853E+00

.13187E+00 .56470E-01

.39072E+00 .14491E+00

.12371E+00 .17575E-01

.21448E+00 .56679E-01

.22267E-01 -.54432E-02

.50347E+00

.17243E+00

.79972E-01

.16621E+00

.26751E-01

.70928E-01

-.76641E-02

.51617E+00 .51780E+00

.17274E+00 .16869E+00

.73876E-01 .67529E-01

.17752E+00 .17844E+00

.27607E-01 .25956E-01

.75383E-01 .74514E-01

-.48798E-02 -.43819E-02

.50836E+00

.16004E+00

.60188E-01

.16876E+00

.22144E-01

.68340E-01

-.52380E-02

.48785E+00 .52138E+00

.14820E+00 .18151E+00

.52578E-01 .82787E-01

.14998E+00 .18467E+00

.17321E-01 .31296E-01

.58197E-01 .80326E-01

-.62253E-02 -.10990E-01

.43713E+00 .43599E+00

.12125E+00 .12224E+00

.41214E-01 .42956E-01

.10819E+00 .10762E+00

.96076E-02 .98518E-02

.38110E-01 .38138E-01

-.33964E-02 -.34340E-02

Moment Invariants for Handwritten Characters

173

.44262E+00

.12604E+00

.45504E-01

.11249E+00

.10892E-01

.40548E-01

-.30245E-02

.50895E+00

.17974E+00

.61828E-01

.17705E+00

.25896E-01

.74918E-01

.53088E-02

.50944E+00 .51280E+00

.18643E+00 .16970E+00

.70216E-01 .50571E-01

.17874E+00 .17777E+00

.28055E-01 .23503E-01

.77300E-01 .72961E-01

.10783E-01 .11796E-02

.51852E+00

.17216E+00

.51183E-01

.18324E+00

.24911E-01

.75836E-01

.46476E-02

.51555E+00 .50544E+00

.17372E+00 .18138E+00

.55428E-01 .63731E-01

.18072E+00 .17471E+00

.25417E-01 .25607E-01

.75321E-01 .74179E-01

.79364E-02 .23298E-03

.52286E+00

.19267E+00

.69918E-01

.19351E+00

.31461E-01

.84562E-01

.89060E-04

.48111E+00 .47694E+00

.16464E+00 .16343E+00

.56411E-01 .55886E-01

.15031E+00 .14704E+00

.19333E-01 .18594E-01

.60977E-01 .59362E-01

.55599E-02 .41116E-02

.47911E+00 .45996E+00

.16287E+00 .15485E+00

.54320E-01 .56835E-01

.14861E+00 .13201E+00

.18596E-01 .15851E-01

.59843E-01 .52107E-01

.27183E-02 -.77522E-02

.47325E+00

.16759E+00

.68897E-01

.14383E+00

.20090E-01

.59459E-01

-.11683E-01

.48262E+00 .48331E+00

.16523E+00 .16247E+00

.62782E-01 .58339E-01

.15059E+00 .15083E+00

.20249E-01 .19398E-01

.61612E-01 .60982E-01

-.10459E-01 -.89396E-02

.47531E+00

.15925E+00

.55611E-01

.14439E+00

.17736E-01

.57628E-01

-.71717E-02

.45862E+00 .49086E+00

.15649E+00 .18330E+00

.55303E-01 .77231E-01

.13209E+00 .16176E+00

.15651E-01 .25334E-01

.52176E-01 .69745E-01

-.50996E-02 -.13938E-01

.42411E+00 .42361E+00

.13244E+00 .13341E+00

.44960E-01 .46326E-01

.10389E+00 .10367E+00

.98458E-02 .10015E-01

.37852E-01 .37982E-01

-.47657E-02 -.50714E-02

.42944E+00

.13486E+00

.47115E-01

.10728E+00

.10628E-01

.39594E-01

-.54865E-02

.56671E+00 .55670E+00

.25019E+00 .24339E+00

.11228E+00 .10989E+00

.25333E+00 .24059E+00

.59849E-01 .54822E-01

.12636E+00 .11846E+00

.58981E-16 .10197E-01

.58355E+00

.24986E+00

.10651E+00

.27133E+00

.64407E-01

.13530E+00

-.89732E-02

.58908E+00 .55393E+00

.24949E+00 .26278E+00

.10311E+00 .13233E+00

.27752E+00 .24426E+00

.65536E-01 .61532E-01

.13822E+00 .12482E+00

-.45103E-16 .11280E-01

.56695E+00 .59462E+00

.25180E+00 .28861E+00

.11733E+00 .14597E+00

.25368E+00 .29708E+00

.61168E-01 .86980E-01

.12715E+00 .15923E+00

.15912E-01 -.15018E-01

.52561E+00

.21821E+00

.92689E-01

.20311E+00

.39109E-01

.94601E-01

.24579E-02

.52172E+00 .52561E+00

.21834E+00 .21821E+00

.93786E-01 .92689E-01

.19969E+00 .20311E+00

.38391E-01 .39109E-01

.93026E-01 .94601E-01

.12143E-16 -.24579E-02

.69138E+00

.61179E+00

.56685E+00

.56255E+00

.48245E+00

.43889E+00

.24286E-16

.72222E+00 .76622E+00

.81217E+00 .76521E+00

.82650E+00 .78053E+00

.69957E+00 .76727E+00

.81172E+00 .87998E+00

.63013E+00 .67023E+00

-.14941E+00 -.53558E-01

.78089E+00 .76622E+00

.75149E+00 .76521E+00

.76572E+00 .78053E+00

.79213E+00 .76727E+00

.90267E+00 .87998E+00

.68536E+00 .67023E+00

.16306E-15 .53558E-01

.72222E+00

.81217E+00

.82650E+00

.69957E+00

.81172E+00

.63013E+00

.14941E+00

.62346E+00 .59799E+00

.42809E+00 .44153E+00

.34105E+00 .35485E+00

.38810E+00 .36014E+00

.21280E+00 .19882E+00

.25281E+00 .23842E+00

.13103E-01 -.22009E-02

.82222E+00

.86028E+00

.92925E+00

.93907E+00

.13009E+01

.86933E+00

-.59674E-15

.59799E+00 .69274E+00

.44153E+00 .28223E+00

.35485E+00 .89521E-01

.36014E+00 .42640E+00

.19882E+00 .11664E+00

.23842E+00 .22608E+00

.22009E-02 -.25500E-15

.69031E+00 .66461E+00

.27759E+00 .27335E+00

.92937E-01 .95544E-01

.42003E+00 .38120E+00

.11461E+00 .10139E+00

.22147E+00 .19904E+00

-.19014E-01 .96801E-02

174

Moment Invariants for Handwritten Characters

.65972E+00

.27119E+00

.94573E-01

.37365E+00

.98206E-01

.19425E+00

-.69389E-16

.66461E+00

.27335E+00

.95544E-01

.38120E+00

.10139E+00

.19904E+00

-.96801E-02

.69031E+00 .69274E+00

.27759E+00 .28223E+00

.92937E-01 .89521E-01

.42003E+00 .42640E+00

.11461E+00 .11664E+00

.22147E+00 .22608E+00

.19014E-01 -.25500E-15

.63824E+00

.24422E+00

.75854E-01

.33472E+00

.74420E-01

.16523E+00

.41261E-02

.63336E+00 .63824E+00

.24206E+00 .24422E+00

.75387E-01 .75854E-01

.32769E+00 .33472E+00

.71993E-01 .74420E-01

.16101E+00 .16523E+00

.30358E-16 -.41261E-02

.69184E+00

.28170E+00

.13874E+00

.42520E+00

.11875E+00

.22480E+00

-.11102E-15

.70873E+00 .66484E+00

.29693E+00 .25251E+00

.14913E+00 .91637E-01

.45427E+00 .37301E+00

.14318E+00 .86894E-01

.24891E+00 .18769E+00

-.31433E-02 .55500E-02

.65021E+00 .66484E+00

.23964E+00 .25251E+00

.75628E-01 .91637E-01

.34876E+00 .37301E+00

.73678E-01 .86894E-01

.17063E+00 .18769E+00

-.90206E-16 -.55500E-02

.70873E+00

.29693E+00

.14913E+00

.45427E+00

.14318E+00

.24891E+00

.31433E-02

.69258E+00 .60265E+00

.30106E+00 .20882E+00

.18724E+00 .80215E-01

.42911E+00 .27856E+00

.14674E+00 .48735E-01

.23767E+00 .12729E+00

.25983E-01 -.18186E-02

.58802E+00

.19594E+00

.68790E-01

.25785E+00

.40467E-01

.11411E+00

-.34694E-17

.60265E+00 .57376E+00

.20882E+00 .18627E+00

.80215E-01 .41024E-01

.27856E+00 .24007E+00

.48735E-01 .34786E-01

.12729E+00 .10335E+00

.18186E-02 -.87278E-16

.56467E+00 .55853E+00

.18249E+00 .18213E+00

.45948E-01 .44301E-01

.22900E+00 .22290E+00

.33048E-01 .31785E-01

.97812E-01 .95005E-01

.55292E-03 .22384E-02

.55464E+00

.18015E+00

.43586E-01

.21849E+00

.30624E-01

.92593E-01

-.32960E-16

.55853E+00 .56467E+00

.18213E+00 .18249E+00

.44301E-01 .45948E-01

.22290E+00 .22900E+00

.31785E-01 .33048E-01

.95005E-01 .97812E-01

-.22384E-02 -.55292E-03

.59559E+00

.20179E+00

.48917E-01

.26916E+00

.44443E-01

.12051E+00

-.28337E-02

.53681E+00 .53292E+00

.16488E+00 .16290E+00

.35617E-01 .35093E-01

.19682E+00 .19270E+00

.23921E-01 .23006E-01

.79839E-01 .77694E-01

.86776E-03 .29924E-16

.53681E+00 .83563E+00

.16488E+00 .62965E+00

.35617E-01 .49421E+00

.19682E+00 .85395E+00

.23921E-01 .78583E+00

.79839E-01 .67581E+00

-.86776E-03 -.12212E-14

.81822E+00

.63769E+00

.55466E+00

.81364E+00

.77407E+00

.65061E+00

-.28852E+00

.78858E+00 .78009E+00

.59843E+00 .60056E+00

.47332E+00 .47793E+00

.73532E+00 .71917E+00

.61938E+00 .60307E+00

.56754E+00 .55587E+00

.73632E-01 -.72164E-15

.78858E+00

.59843E+00

.47332E+00

.73532E+00

.61938E+00

.56754E+00

-.73632E-01

.81822E+00 .77160E+00

.63769E+00 .50474E+00

.55466E+00 .34738E+00

.81364E+00 .65770E+00

.77407E+00 .44411E+00

.65061E+00 .46622E+00

.28852E+00 .80602E-01

.93556E+00 .73765E+00

.77702E+00 .51134E+00

.67650E+00 .36880E+00

.11930E+01 .59711E+00

.15222E+01 .39956E+00

.10486E+01 .42575E+00

-.22893E+00 .00000E+00

.93556E+00

.77702E+00

.67650E+00

.11930E+01

.15222E+01

.10486E+01

.22893E+00

.81847E+00 .79861E+00

.74101E+00 .88341E+00

.75519E+00 .97308E+00

.86816E+00 .88258E+00

.97296E+00 .11876E+01

.74592E+00 .83015E+00

-.29143E-15 -.47761E+00

.74889E+00

.63819E+00

.60789E+00

.67433E+00

.61994E+00

.53772E+00

.52100E-01

.73689E+00 .74889E+00

.64413E+00 .63819E+00

.61042E+00 .60789E+00

.65457E+00 .67433E+00

.60074E+00 .61994E+00

.52424E+00 .53772E+00

.12143E-15 -.52100E-01

.79861E+00 .73822E+00

.88341E+00 .51993E+00

.97308E+00 .45629E+00

.88258E+00 .60174E+00

.11876E+01 .43032E+00

.83015E+00 .43289E+00

.47761E+00 .53704E-01

Moment Invariants for Handwritten Characters

175

.69022E+00

.53978E+00

.47905E+00

.52747E+00

.37914E+00

.38663E+00

.96007E+00

.10550E+01

.12551E+01

.14205E+01

.25879E+01

.14567E+01

.21511E-15 .22086E+00

.73822E+00 .52202E+00

.51993E+00 .19586E+00

.45629E+00 .72119E-01

.60174E+00 .19291E+00

.43032E+00 .31777E-01

.43289E+00 .85197E-01

-.53704E-01 -.44309E-02

.53086E+00

.22413E+00

.10541E+00

.20832E+00

.43145E-01

.98963E-01

-.19417E-01

.56096E+00 .57099E+00

.22094E+00 .21967E+00

.86651E-01 .78780E-01

.23735E+00 .24783E+00

.47525E-01 .48433E-01

.11155E+00 .11590E+00

-.15060E-01 -.71637E-02

.56096E+00

.21760E+00

.80117E-01

.23691E+00

.45590E-01

.11019E+00

.12149E-02

.53086E+00 .48283E+00

.21744E+00 .15428E+00

.92778E-01 .44980E-01

.20744E+00 .14903E+00

.40065E-01 .17042E-01

.96537E-01 .58390E-01

.73211E-02 -.26208E-03

.46388E+00 .45756E+00

.15240E+00 .15239E+00

.48457E-01 .50208E-01

.13487E+00 .13048E+00

.15225E-01 .14764E-01

.52502E-01 .50815E-01

-.12101E-02 -.20392E-02

.59105E+00

.25454E+00

.10812E+00

.28150E+00

.68628E-01

.14158E+00

-.40092E-02

.64161E+00 .63238E+00

.24627E+00 .23417E+00

.73690E-01 .70161E-01

.34095E+00 .32413E+00

.76726E-01 .68642E-01

.16862E+00 .15664E+00

.11276E-15 -.12763E-01

.62529E+00

.24791E+00

.85011E-01

.32003E+00

.74103E-01

.15897E+00

.77158E-02

.62140E+00 .62529E+00

.24647E+00 .24791E+00

.84712E-01 .85011E-01

.31473E+00 .32003E+00

.72237E-01 .74103E-01

.15584E+00 .15897E+00

.69389E-17 -.77158E-02

.63238E+00 .64161E+00

.23417E+00 .24627E+00

.70161E-01 .73690E-01

.32413E+00 .34095E+00

.68642E-01 .76726E-01

.15664E+00 .16862E+00

.12763E-01 .11276E-15

.60174E+00

.22223E+00

.67364E-01

.28260E+00

.54976E-01

.13294E+00

.38213E-02

.59785E+00 .60174E+00

.22079E+00 .22223E+00

.67339E-01 .67364E-01

.27766E+00 .28260E+00

.53554E-01 .54976E-01

.13018E+00 .13294E+00

-.17347E-17 -.38213E-02

.40181E+00

.11507E+00

.32060E-01

.87791E-01

.65013E-02

.29707E-01

-.56389E-03

.40045E+00 .39289E+00

.11408E+00 .11448E+00

.32759E-01 .33160E-01

.86694E-01 .83093E-01

.64251E-02 .60953E-02

.29266E-01 .28037E-01

-.18125E-02 -.18648E-16

.41140E+00 .40899E+00

.11380E+00 .11340E+00

.29187E-01 .29235E-01

.92641E-01 .91253E-01

.67265E-02 .65887E-02

.31177E-01 .30648E-01

-.66197E-03 -.56379E-17

.40132E+00

.11292E+00

.30613E-01

.87044E-01

.62749E-02

.29178E-01

.59433E-03

.41425E+00 .38214E+00

.12558E+00 .10535E+00

.38269E-01 .28422E-01

.96936E-01 .75799E-01

.82489E-02 .49122E-02

.34291E-01 .24538E-01

-.18579E-02 -.12829E-03

.42521E+00

.12549E+00

.35322E-01

.10328E+00

.87147E-02

.36487E-01

-.84163E-03

.38180E+00 .41343E+00

.10559E+00 .11596E+00

.28818E-01 .30804E-01

.75661E-01 .94179E-01

.49300E-02 .70553E-02

.24531E-01 .32018E-01

-.63979E-03 .11641E-02

.40824E+00 .40527E+00

.11062E+00 .11727E+00

.27964E-01 .33734E-01

.90177E-01 .89974E-01

.62676E-02 .69122E-02

.29918E-01 .30786E-01

-.36006E-03 .17325E-02

.42206E+00

.11331E+00

.27028E-01

.98423E-01

.70492E-02

.33080E-01

.10847E-02

.40439E+00 .41054E+00

.11619E+00 .11385E+00

.32671E-01 .31136E-01

.89395E-01 .91783E-01

.67130E-02 .68532E-02

.30399E-01 .31039E-01

.55087E-03 .24985E-02

.42494E+00

.12454E+00

.34680E-01

.10292E+00

.85674E-02

.36215E-01

.13497E-03

.39256E+00 .43876E+00

.10650E+00 .12675E+00

.28058E-01 .33636E-01

.80986E-01 .11172E+00

.53727E-02 .95303E-02

.26404E-01 .39690E-01

.12011E-02 .15038E-02

.39168E+00 .37975E+00

.10573E+00 .10366E+00

.27325E-01 .27660E-01

.80465E-01 .74359E-01

.52421E-02 .47099E-02

.26111E-01 .23870E-01

.59224E-03 .22652E-03

176

Moment Invariants for Handwritten Characters

.40824E+00

.11062E+00

.27964E-01

.90177E-01

.62676E-02

.29918E-01

-.36006E-03

.40527E+00

.11727E+00

.33734E-01

.89974E-01

.69122E-02

.30786E-01

.17325E-02

.42206E+00 .40439E+00

.11331E+00 .11619E+00

.27028E-01 .32671E-01

.98423E-01 .89395E-01

.70492E-02 .67130E-02

.33080E-01 .30399E-01

.10847E-02 .55087E-03

.41054E+00

.11385E+00

.31136E-01

.91783E-01

.68532E-02

.31039E-01

.24985E-02

.42494E+00 .39256E+00

.12454E+00 .10650E+00

.34680E-01 .28058E-01

.10292E+00 .80986E-01

.85674E-02 .53727E-02

.36215E-01 .26404E-01

.13497E-03 .12011E-02

.43876E+00

.12675E+00

.33636E-01

.11172E+00

.95303E-02

.39690E-01

.15038E-02

.39168E+00

.10573E+00

.27325E-01

.80465E-01

.52421E-02

.26111E-01

.59224E-03

Appendix F: Pattern.cpp

// pattern1.cpp // Kohonen map for pattern recognition #include “e:\bujji\layerk.cpp" #define INPUT_FILE “e:\\bujji\\input.dat" #define OUTPUT_FILE “e:\\bujji\\kohonen.dat" #define dist_tol 0.001 #define wait_cycles 10000 // creates a pause to // view the character maps void main() { int neighborhood_size, period; float avg_dist_per_cycle=0.0; float dist_last_cycle=0.0; float avg_dist_per_pattern=100.0; // for the latest cycle float dist_last_pattern=0.0; float total_dist; float alpha; unsigned startup; int max_cycles; int patterns_per_cycle=0; int total_cycles, total_patterns; int i; // create a network object Kohonen_network knet; FILE * input_file_ptr, * output_file_ptr; // open input file for reading if ((input_file_ptr=fopen(INPUT_FILE,“r"))==NULL) V.K. David and S. Rajasekaran: Pattern Recog. Using Neural & Funct. Net., SCI 160, pp. 177–181. c Springer-Verlag Berlin Heidelberg 2009 springerlink.com

178

Pattern.cpp

{ cout << “problem opening input file\n"; exit(1); } // open writing file for writing if ((output_file_ptr=fopen(OUTPUT_FILE,“w"))==NULL) { cout << “problem opening output file\n"; exit(1); }

// ----------------------------------------// Read in an initial values for alpha, and the // neighborhood size. // Both of these parameters are decreased with // time. The number of cycles to execute before // decreasing the value of these parameters is // called the period. Read in a value for the // period. // ----------------------------------------cout << “ Please enter initial values for:\n"; cout << “alpha (0.01-1.0),\n"; cout << “and the neighborhood size (integer between 0 and 50)\n"; cout << “separated by spaces, e.g. 0.3 5 \n "; cin >> alpha >> neighborhood_size ; cout << “Now enter the period, which is the\n"; cout << “number of cycles after which the values\n"; cout << “for alpha the neighborhood size are decremented\n"; cout << “choose an integer between 1 and 500 , e.g. 50 \n"; cin >> period; // Read in the maximum number of cycles // each pass through the input data file is a cycle cout << “Please enter the maximum cycles for the simulation\n"; cout << “A cycle is one pass through the data set.\n"; cout << “Try a value of 500 to start with\n"; cin >> max_cycles;

Pattern.cpp

179

// the main loop // // continue looping until the average distance is less than // the tolerance specified at the top of this file // , or the maximum number of // cycles is exceeded; // initialize counters total_cycles=0; // a cycle is once through all the input data total_patterns=0; // a pattern is one entry in the input data // get layer information knet.get_layer_info(); // set up the network connections knet.set_up_network(neighborhood_size); // initialize the weights // randomize weights for the Kohonen layer // note that the randomize function for the // Kohonen simulator generates // weights that are normalized to length = 1 knet.randomize_weights(); // write header to output file fprintf(output_file_ptr, “cycle\tpattern\twin index\tneigh_size\tavg_dist_per_pattern\n"); fprintf(output_file_ptr, “------------------------------------------------\n"); // main loop startup=1; total_dist=0; while ( (avg_dist_per_pattern > dist_tol) && (total_cycles < max_cycles) || (startup==1) )

180

Pattern.cpp

{ startup=0; dist_last_cycle=0; // reset for each cycle patterns_per_cycle=0; // process all the vectors in the datafile while (!feof(input_file_ptr)) { knet.get_next_vector(input_file_ptr); // now apply it to the Kohonen network knet.process_next_pattern(); dist_last_pattern=knet.get_win_dist(); // print result to output file fprintf(output_file_ptr,“%i\t%i\t%i\t\t%i\t\t%f\n", total_cycles,total_patterns,knet.get_win_index(), neighborhood_size,avg_dist_per_pattern); // display the input character and the // weights for the winner to see match knet.display_input_char(); knet.display_winner_weights(); cout << “Press any key ..."; char patrec; cin >> patrec; // pause for a while to view the // character maps for (i=0; i<wait_cycles; i++) {;} total_patterns++; // gradually reduce the neighborhood size // and the gain, alpha if (((total_cycles+1) % period) == 0) { if (neighborhood_size > 0) neighborhood_size –; knet.update_neigh_size(neighborhood_size);

Pattern.cpp

if (alpha>0.1) alpha -= (float)0.1; } patterns_per_cycle++; dist_last_cycle += dist_last_pattern; knet.update_weights(alpha); dist_last_pattern = 0; } avg_dist_per_pattern= dist_last_cycle/patterns_per_cycle; total_dist += dist_last_cycle; total_cycles++; fseek(input_file_ptr, 0L, SEEK_SET); // reset the file pointer // to the beginning of // the file } // end main loop cout << “\n\n\n\n\n\n\n\n\n\n\n"; cout << “---------------------------------------------\n"; cout << “ done \n"; avg_dist_per_cycle= total_dist\total_cycles; cout << “\n"; cout << “---->average dist per cycle = " << avg_dist_per_cycle << " <---\n"; cout << “---->dist last cycle = " << dist_last_cycle << " <---\n"; cout << “->dist last cycle per pattern= " << avg_dist_per_pattern << " <---\n"; cout << “------------>total cycles = " << total_cycles << " <---\n"; cout << “------------>total patterns = " << total_patterns << “ <---\n"; cout << “---------------------------------------------------\n"; // close the input file fclose(input_file_ptr); }

181

Appendix G: handpgm2.m

Refer to the input (INP) and output (OUT) from the program handpgm1.m tic TAG=(‘SALO1’); PAR=[0.5 0.0 0.5 1.0 0 0.08 0.002]’; [Mi Ni]=size(INP); [Mo No]=size(OUT); UnitsA=3; UnitsB=3; WA=ones(UnitsA, 2*Ni); WB=ones(UnitsB, 2*No); WAB=(ones(UnitsA,UnitsB)); MICRO(‘create’, TAG, PAR, WA, WB, WAB) LIST=MICRO(‘list’) MICRO(‘train’, TAG, INP, OUT) OUT=MICRO(‘test’, TAG , INP); INP1=[1 1 0 1 0 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0]; OUT1=MICRO(‘test’, TAG, INP1) toc

V.K. David and S. Rajasekaran: Pattern Recog. Using Neural & Funct. Net., SCI 160, p. 183. c Springer-Verlag Berlin Heidelberg 2009 springerlink.com

Pattern Recognition Using Neural and Functional Networks (Studies in Computational Intelligence)

Pattern recognition and neural networks

Pattern Recognition and Neural Networks

Pattern Recognition and Neural Networks

Pattern Recognition and Neural Networks

Neural Networks and Pattern Recognition

Applied Pattern Recognition (Studies in Computational Intelligence)

Applied Pattern Recognition (Studies in Computational Intelligence)

Neural Networks for Pattern Recognition

Neural Networks for Pattern Recognition

Pattern Recognition with Neural Networks in C++

Pattern Recognition with Neural Networks in C++

Computational Geometry (Machine intelligence and pattern recognition)

Pattern Recognition By Self-organizing Neural Networks

Pattern recognition by self-organizing neural networks

Speech, Audio, Image and Biomedical Signal Processing using Neural Networks (Studies in Computational Intelligence, Volume 83)

Artificial Neural Networks in Pattern Recognition, 3 conf., ANNPR 2008

Neural Networks: Computational Models and Applications (Studies in Computational Intelligence, Volume 53)

A statistical approach to neural networks for pattern recognition

Computer Vision: Detection, Recognition and Reconstruction (Studies in Computational Intelligence)

A Statistical Approach to Neural Networks for Pattern Recognition (Wiley Series in Computational Statistics)

Pattern Recognition and Machine Intelligence - PReMI 2011

Mining and Analyzing Social Networks (Studies in Computational Intelligence, 288)

Computational Intelligence Paradigms in Advanced Pattern Classification

Computational Intelligence in Engineering (Studies in Computational Intelligence)

Computational Intelligence in Engineering (Studies in Computational Intelligence, 313)