The confluence of cloud computing, parallelism and advanced
machine intelligence approaches has created a world in which the
optimum knowledge system will usually be architected from the
combination of two or more knowledge-generating systems. There is a
need, then, to provide a reusable, broadly-applicable set of design
patterns to empower the intelligent system architect to take
advantage of this opportunity.

This book explains how to design and build intelligent systems
that are optimized for changing system requirements (adaptability),
optimized for changing system input (robustness), and optimized for
one or more other important system parameters (e.g.,
accuracy, efficiency, cost). It provides an overview of traditional
parallel processing which is shown to consist primarily of task and
component parallelism; before introducing meta-algorithmic
parallelism which is based on combining two or more algorithms,
classification engines or other systems.

Key features:

* Explains the entire roadmap for the design, testing,
development, refinement, deployment and statistics-driven
optimization of building systems for intelligence

* Offers an accessible yet thorough overview of machine
intelligence, in addition to having a strong image processing
focus

* Contains design patterns for parallelism, especially
meta-algorithmic parallelism - simply conveyed, reusable and
proven effective that can be readily included in the toolbox of
experts in analytics, system architecture, big data, security and
many other science and engineering disciplines

* Connects algorithms and analytics to parallelism, thereby
illustrating a new way of designing intelligent systems compatible
with the tremendous changes in the computing world over the past
decade

* Discusses application of the approaches to a wide number of
fields; primarily, document understanding, image understanding,
biometrics and security printing

* Companion website contains sample code and data sets

Steven J. Simske, Hewlett-Packard Labs, Colorado, USA
Dr Simske is currently Director of the Document Ecosystem Lab, at Hewlett-Packard Labs, Colorado, USA. He has been working in algorithms, imaging, machine learning and classification for the past 20 years. As an engineer at HP Labs, he has designed, developed and shipped products associated with a very broad array of domainsdocument understanding, image segmentation and understanding, speech recognition, medical signal processing and imaging, biometrics, natural language processing, surveillance, optical character recognition, security analytics and security printing. The advantages of systematic meta-algorithmic approaches to the robustness, accuracy, cost and/or other system features which is the focus of the book has been evident across these domains. Dr. Simske is an HP Fellow, IS&T Fellow and IEEE Senior Member. He has published 300 articles and book chapters; and holds 45 US Patents primarily in the areas of classification, machine learning, and large system design and development.

This book explains how to design and build intelligent systems that are optimized for changing system requirements (adaptability), optimized for changing system input (robustness), and optimized for one or more other important system parameters (e.g., accuracy, efficiency, cost). It provides an overview of traditional parallel processing which is shown to consist primarily of task and component parallelism; before introducing meta-algorithmic parallelism which is based on combining two or more algorithms, classification engines or other systems.

Key features:

• Explains the entire roadmap for the design, testing, development, refinement, deployment and statistics-driven optimization of building systems for intelligence
• Offers an accessible yet thorough overview of machine intelligence, in addition to having a strong image processing focus
• Contains design patterns for parallelism, especially meta-algorithmic parallelism simply conveyed, reusable and proven effective that can be readily included in the toolbox of experts in analytics, system architecture, big data, security and many other science and engineering disciplines
• Connects algorithms and analytics to parallelism, thereby illustrating a new way of designing intelligent systems compatible with the tremendous changes in the computing world over the past decade
• Discusses application of the approaches to a wide number of fields; primarily, document understanding, image understanding, biometrics and security printing
• Companion website contains sample code and data sets

Acknowledgments xi

1 Introduction and Overview 1

1.1 Introduction 1

1.2 Why Is This Book Important? 2

1.3 Organization of the Book 3

1.4 Informatics 4

1.5 Ensemble Learning 6

1.6 Machine Learning/Intelligence 7

1.6.1 Regression and Entropy 8

1.6.2 SVMs and Kernels 9

1.6.3 Probability 15

1.6.4 Unsupervised Learning 17

1.6.5 Dimensionality Reduction 18

1.6.6 Optimization and Search 20

1.7 Artificial Intelligence 22

1.7.1 Neural Networks 22

1.7.2 Genetic Algorithms 25

1.7.3 Markov Models 28

1.8 Data Mining/Knowledge Discovery 31

1.9 Classification 32

1.10 Recognition 38

1.11 System-Based Analysis 39

1.12 Summary 39

References 40

2 Parallel Forms of Parallelism 42

2.1 Introduction 42

2.2 Parallelism by Task 43

2.2.1 Definition 43

2.2.2 Application to Algorithms and Architectures 46

2.2.3 Application to Scheduling 51

2.3 Parallelism by Component 52

2.3.1 Definition and Extension to Parallel-Conditional Processing 52

2.3.2 Application to Data Mining, Search, and Other Algorithms 55

2.3.3 Application to Software Development 59

2.4 Parallelism by Meta-algorithm 64

2.4.1 Meta-algorithmics and Algorithms 66

2.4.2 Meta-algorithmics and Systems 67

2.4.3 Meta-algorithmics and Parallel Processing 68

2.4.4 Meta-algorithmics and Data Collection 69

2.4.5 Meta-algorithmics and Software Development 70

2.5 Summary 71

References 72

3 Domain Areas: Where Are These Relevant? 73

3.1 Introduction 73

3.2 Overview of the Domains 74

3.3 Primary Domains 75

3.3.1 Document Understanding 75

3.3.2 Image Understanding 77

3.3.3 Biometrics 78

3.3.4 Security Printing 79

3.4 Secondary Domains 86

3.4.1 Image Segmentation 86

3.4.2 Speech Recognition 90

3.4.3 Medical Signal Processing 90

3.4.4 Medical Imaging 92

3.4.5 Natural Language Processing 95

3.4.6 Surveillance 97

3.4.7 Optical Character Recognition 98

3.4.8 Security Analytics 101

3.5 Summary 101

References 102

4 Applications of Parallelism by Task 104

4.1 Introduction 104

4.2 Primary Domains 105

4.2.1 Document Understanding 112

4.2.2 Image Understanding 118

4.2.3 Biometrics 126

4.2.4 Security Printing 131

4.3 Summary 135

References 136

5 Application of Parallelism by Component…