Multilingual Natural Language Processing Applications is the first comprehensive single-source guide to building robust and accurate multilingual NLP systems. Edited by two leading experts, it integrates cutting-edge advances with practical solutions drawn from extensive field experience.
Part I introduces the core concepts and theoretical foundations of modern multilingual natural language processing, presenting today's best practices for understanding word and document structure, analyzing syntax, modeling language, recognizing entailment, and detecting redundancy.
Part II thoroughly addresses the practical considerations associated with building real-world applications, including information extraction, machine translation, information retrieval/search, summarization, question answering, distillation, processing pipelines, and more.
This book contains important new contributions from leading researchers at IBM, Google, Microsoft, Thomson Reuters, BBN, CMU, University of Edinburgh, University of Washington, University of North Texas, and others.
Coverage includes
Core NLP problems, and today's best algorithms for attacking them
- Processing the diverse morphologies present in the world's languages
- Uncovering syntactical structure, parsing semantics, using semantic role labeling, and scoring grammaticality
- Recognizing inferences, subjectivity, and opinion polarity
- Managing key algorithmic and design tradeoffs in real-world applications
- Extracting information via mention detection, coreference resolution, and events
- Building large-scale systems for machine translation, information retrieval, and summarization
- Answering complex questions through distillation and other advanced techniques
- Creating dialog systems that leverage advances in speech recognition, synthesis, and dialog management
- Constructing common infrastructure for multiple multilingual text processing applications
This book will be invaluable for all engineers, software developers, researchers, and graduate students who want to process large quantities of text in multiple languages, in any environment: government, corporate, or academic.
Autorentext
Daniel M. Bikel is a senior research scientist at Google, developing new methods for NLP and speech recognition. While at IBM, he architected the distillation system for IBM's GALE multilingual information extraction and question-answering system. While pursuing his doctorate at Penn, he built the first extensible multilingual syntactic parsing engine.
Imed Zitouni is a senior research scientist at IBM. He has led IBM's Arabic information extraction and data resources efforts since 2004. He previously led both DIALOCA's Speech/NLP group and Bell Labs/ Alcatel-Lucent's language modeling and call routing activities. His work involves machine translation, NLP, and spoken dialog systems.
Klappentext
Multilingual Natural Language Processing Applications is the first comprehensive single-source guide to building robust and accurate multilingual NLP systems. Edited by two leading experts, it integrates cutting-edge advances with practical solutions drawn from extensive field experience.
Part I introduces the core concepts and theoretical foundations of modern multilingual natural language processing, presenting today's best practices for understanding word and document structure, analyzing syntax, modeling language, recognizing entailment, and detecting redundancy.
Part II thoroughly addresses the practical considerations associated with building real-world applications, including information extraction, machine translation, information retrieval/search, summarization, question answering, distillation, processing pipelines, and more.
This book contains important new contributions from leading researchers at IBM, Google, Microsoft, Thomson Reuters, BBN, CMU, University of Edinburgh, University of Washington, University of North Texas, and others.
Coverage includes
Core NLP problems, and today's best algorithms for attacking them
- Processing the diverse morphologies present in the world's languages
- Uncovering syntactical structure, parsing semantics, using semantic role labeling, and scoring grammaticality
- Recognizing inferences, subjectivity, and opinion polarity
- Managing key algorithmic and design tradeoffs in real-world applications
- Extracting information via mention detection, coreference resolution, and events
- Building large-scale systems for machine translation, information retrieval, and summarization
- Answering complex questions through distillation and other advanced techniques
- Creating dialog systems that leverage advances in speech recognition, synthesis, and dialog management
- Constructing common infrastructure for multiple multilingual text processing applications
This book will be invaluable for all engineers, software developers, researchers, and graduate students who want to process large quantities of text in multiple languages, in any environment: government, corporate, or academic.
Inhalt
Preface xxi
Acknowledgments xxv
About the Authors xxvii
Part I: In Theory 1
Chapter 1: Finding the Structure of Words 3
1.1 Words and Their Components 4
1.2 Issues and Challenges 8
1.3 Morphological Models 15
1.4 Summary 22
Chapter 2: Finding the Structure of Documents 29
2.1 Introduction 29
2.2 Methods 33
2.3 Complexity of the Approaches 40
2.4 Performances of the Approaches 41
2.5 Features 41
2.6 Processing Stages 48
2.7 Discussion 48
2.8 Summary 49
Chapter 3: Syntax 57
3.1 Parsing Natural Language 57
3.2 Treebanks: A Data-Driven Approach to Syntax 59
3.3 Representation of Syntactic Structure 63
3.4 Parsing Algorithms 70
3.5 Models for Ambiguity Resolution in Parsing 80
3.6 Multilingual Issues: What Is a Token? 87
3.7 Summary 92
Chapter 4: Semantic Parsing 97
4.1 Introduction 97
4.2 Semantic Interpretation 98
4.3 System Paradigms 101
4.4 Word Sense 102
4.5 Predicate-Argument Structure 118
4.6 Meaning Representation 147
4.7 Summary 152
Chapter 5: Language Modeling 169
5.1 Introduction 169
5.2 n-Gram Models 170
5.3 Language Model Evaluation 170
5.4 Parameter Estimation 171
5.5 Language Model Adaptation 176
5.6 Types of Language Models 178
5.7 Language-Specific Modeling Problems 188
5.8 Multilingual and Crosslingual Language Modeling 195
5.9 Summary 198
Chapter 6: Recognizing Textual Entailment 209
6.1 Introduction 209
6.2 The Recognizing Textual Entailment Task 210
6.3 A Framework for Recognizing Textual Entailment 219
6.4 Case Studies 238
6.5 Taking RTE Further 248
6.6 Useful Resources 252
6.7 Summary 253
Chapter 7: Multilingual Sentiment and Subjectivity Analysis 259
7.1 Introduction 259
7.2 Definitions 260
7.3 Sentiment and Subjectivity Analysis on English 262
7.4 Word- and Phrase-Level Annotations 264
7.5 Sentence-Level Annotations 270
7.6 Document-Level Annotations 272
7.7 What Works, What Doesn't 274
7.8 Summary 277
Part II: In Practice 283
Chapter 8: Entity Detection and Tracking 285
8.1 Introduction 285
8.2 Mention …