The field of natural language processing (NLP) is one of the most important and useful application areas of artificial intelligence. NLP is now rapidly evolving, as new methods and toolsets converge with an ever-expanding wealth of available data. This state-of-the-art handbook addresses all aspects of formal analysis for natural language processing. Following a review of the field's history, it systematically introduces readers to the rule-based model, statistical model, neural network model, and pre-training model in natural language processing.

At a time characterized by the steady and vigorous growth of natural language processing, this handbook provides a highly accessible introduction and much-needed reference guide to both the theory and method of NLP. It can be used for individual study, as the textbook for courses on natural language processing or computational linguistics, or as a supplement to courses on artificial intelligence, and offers a valuable asset for researchers, practitioners, lecturers, graduate and undergraduate students alike.

Feng Zhiwei is a computational linguist and senior research fellow at the Institute of Applied Linguistics, Ministry of Education, China. He has a broad and extensive background in linguistics, mathematics and computer science, and has been engaged in interdisciplinary research in linguistics, mathematics and computer science for more than 50 years. One of the first natural language processing and computational linguistics scholars in China, he has published more than 30 books and more than 400 papers in China and abroad. He is the winner of the NLPCC (Natural Language Processing & Chinese Computing) Distinguished Achievement Award of the CCF (China Computer Federation) in 2018.

Part One History Review

Chapter One

Past and Present of Natural Language Processing

1.1 What is natural language processing

1.2 History review of natural language processing

1.3 Characteristics of current trends in natural language processing

References

Chapter Two

Pioneers in the Study of Language Computing

2.1 Markov chains

2.2 Zipf's law

2.3 Shannon's work on entropy

2.4 Bar-Hillel's category grammar

2.5 Harris's approach of linguistic string analysis

2.6 . . 's linguistic set theory model

References

Part Two Formal Models

Chapter Three

Formal Models Based on Phrase Structure Grammar

3.1 Chomsky's hierarchy of grammar

3.3 Phrase structure grammar

3.4 Recursive transition networks and augmented transition networks

3.5 Bottom-up and top-down analysis

3.6 General syntactic processor and chart parsing

3.7 Earley algorithm

3.8 Left corner analysis

3.9 Cocke-Younger-Kasami algorithm

3.10 Tomita algorithm

3.11 Government and binding theory and minimalist program

3.12 Joshi's tree-adjoining grammar

3.13 Formal description of the structure of Chinese characters

3.14 Hausser's left-associative grammar

References

Chapter Four

Formal Models Based on Unification

4.1 Multiple branched and multiple labeled tree analysis (MMT)

4.2 Kaplan's lexical functional grammar

4.3 Martin Kay's functional unification grammar

4.4 Gazdar's generalized phrase structure grammar

4.5 Shieber's PATR

4.6 Pollard's head-driven phrase structure grammar

4.7 Pereira &Warren's definite clause grammar

Chapter Five

Formal Models Based on Dependency and Valence

5.1 Origin of valence

5.2 Tesnière's dependency grammar

5.3 Application of dependency grammar in natural language processing

5.4 Valence grammar

5.5 Application of valence grammar in natural language processing

References

Chapter Six

Formal Models Based on Lexicalism

6.1 Gross' lexicon-grammar

6.2 Chain grammar

6.3 Lexical semantics

6.4 Ontology

6.5 WordNet

6.6 HowNet

6.7 Pustejovesky's generative lexicon theory

References

Chapter Seven

Formal Models of Automatic Semantic Processing

7.1 Sememe analysis

7.2 Semantic field

7.3 Semantic network

7.4 Montague's semantics

7.5 Wilks' preference semantics

7.6 Schank's conceptual dependency theory

7.7 Mel'chuk's meaning-text theory

7.8 Fillmore's deep case and frame semantics

7.9 Word sense disambiguation methods

References

Chapter Eight

Formal Models of Automatic Situation and Pragmatic Processing

8.1 Basic concepts of systemic functional grammar

8.2 Application of systemic functional grammar in natural language processing

8.3 Speech act theory and conversation intelligent agent

References

Chapter Nine

9.1 Reference resolution

9.2 Reasoning techniques in text coherence

9.3 Mann & Thompson's rhetorical structure theory

References

Chapter Ten

Formal Models of Probabilistic Grammar

10.1 Probabilistic context-free grammar and sentence ambiguity

10.3 Three assumptions of probabilistic context-free grammar

10.4 Probabilistic lexicalized context-free grammar

References

Chapter Eleven

11.1 Development of neural network

11.2 Brain neural network and artificial neural network

11.4 Word vector and word embedding (CBOW, Skip-gram)

11.5 Dense word vector (Word2vec)

11.6 Perceptron

11.8 Convolutional Neural Network (CNN)

11.9 Recurrent Neural Network (RNN)

11.10 Attention mechanism

11.11 External memory

11.12 Pre-training models (Transformer and BERT)

References

Knowledge Graph

12.1 Knowledge graph and deep learning

12.2 Knowledge representation

12.3 Entity recognition

12.4 Entity disambiguation

12.5 Relation abstraction

12.6 Entity abstraction

12.7 Knowledge storage

References

Conclusion