This book is about constructing models from experimental data. It covers a range of topics, from statistical data prediction to Kalman filtering, from black-box model identification to parameter estimation, from spectral analysis to predictive control.

Written for graduate students, this textbook offers an approach that has proven successful throughout the many years during which its author has taught these topics at his University.

The book:

* Contains accessible methods explained step-by-step in simple terms

* Offers an essential tool useful in a variety of fields, especially engineering, statistics, and mathematics

* Includes an overview on random variables and stationary processes, as well as an introduction to discrete time models and matrix analysis

* Incorporates historical commentaries to put into perspective the developments that have brought the discipline to its current state

* Provides many examples and solved problems to complement the presentation and facilitate comprehension of the techniques presented

SERGIO BITTANTI is Emeritus Professor of Model Identification and Data Analysis (MIDA) at the Politecnico di Milano, Milan, Italy, where his intense activity of research and teaching has attracted the attention of many young researchers.

He started teaching the course of MIDA years ago, with just a few students. Today the course is offered in various sections with about one thousand students.

He has organized a number of workshops and conferences, and has served as member of the Program Committee of more than 70 international congresses.

He has for many years been associated with the National Research Council (CNR) of Italy and is a member of the Academy of Science and Literature of Milan (Istituto Lombardo Accademia di Scienze e Lettere).

He received many awards, in particular the title of Ambassador of the city of Milan and the medal of the President of the Italian Republic for the IFAC World Congress held in Milan in 2011 with a record number of attendees from 73 Countries.

Website: http://home.deib.polimi.it/bittanti/

Inhalt

Introduction xi

Acknowledgments xv

1 Stationary Processes and Time Series 1

1.1 Introduction 1

1.2 The Prediction Problem 1

1.3 Random Variable 4

1.4 Random Vector 5

1.4.1 Covariance Coefficient 7

1.5 Stationary Process 9

1.6 White Process 11

1.7 MA Process 12

1.8 AR Process 16

1.8.1 Study of the AR(1) Process 16

1.9 YuleWalker Equations 20

1.9.1 YuleWalker Equations for the AR(1) Process 20

1.9.2 YuleWalker Equations for the AR(2) and AR(n) Process 21

1.10 ARMA Process 23

1.11 Spectrum of a Stationary Process 24

1.11.1 Spectrum Properties 24

1.11.2 Spectral Diagram 25

1.11.3 Maximum Frequency in Discrete Time 25

1.11.4 White Noise Spectrum 25

1.11.5 Complex Spectrum 26

1.12 ARMA Model: Stability Test and Variance Computation 26

1.12.1 Ruzicka Stability Criterion 28

1.12.2 Variance of an ARMA Process 32

1.13 FundamentalTheorem of Spectral Analysis 35

1.14 Spectrum Drawing 38

1.15 Proof of the FundamentalTheorem of Spectral Analysis 43

1.16 Representations of a Stationary Process 45

2 Estimation of Process Characteristics 47

2.1 Introduction 47

2.2 General Properties of the Covariance Function 47

2.3 Covariance Function of ARMA Processes 49

2.4 Estimation of the Mean 50

2.5 Estimation of the Covariance Function 53

2.6 Estimation of the Spectrum 55

2.7 Whiteness Test 57

3 Prediction 61

3.1 Introduction 61

3.2 Fake Predictor 62

3.2.1 Practical Determination of the Fake Predictor 64

3.3 Spectral Factorization 66

3.4 Whitening Filter 70

3.5 Optimal Predictor from Data 71

3.6 Prediction of an ARMA Process 76

3.7 ARMAX Process 77

3.8 Prediction of an ARMAX Process 78

4 Model Identification 81

4.1 Introduction 81

4.2 Setting the Identification Problem 82

4.2.1 Learning from Maxwell 82

4.2.2 A General Identification Problem 84

4.3 Static Modeling 85

4.3.1 Learning from Gauss 85

4.3.2 Least Squares Made Simple 86

4.3.2.1 Trend Search 86

4.3.2.2 Seasonality Search 86

4.3.2.3 Linear Regression 87

4.3.3 Estimating the Expansion of the Universe 90

4.4 Dynamic Modeling 92

4.5 External RepresentationModels 92

4.5.1 Box and Jenkins Model 92

4.5.2 ARX and AR Models 93

4.5.3 ARMAX and ARMA Models 94

4.5.4 MultivariableModels 96

4.6 Internal RepresentationModels 96

4.7 The Model Identification Process 100

4.8 The Predictive Approach 101

4.9 Models in Predictive Form 102

4.9.1 Box and Jenkins Model 103

4.9.2 ARX and AR Models 103

4.9.3 ARMAX and ARMA Models 104

5 Identification of InputOutput Models 107

5.1 Introduction 107

5.2 Estimating AR and ARX Models: The Least Squares Method 107

5.3 Identifiability 110

5.3.1 The R Matrix for the ARX(1, 1) Model 111

5.3.2 The R Matrix for a General ARX Model 112

5.4 Estimating ARMA and ARMAX Models 115

5.4.1 Computing the Gradient and the Hessian from Data 117

5.5 Asymptotic Analysis 123

5.5.1 Data Generation SystemWithin the Class of Models 125

5.5.2 Data Generation System Outside the Class of Models 127

5.5.2.1 Simulation Trial 132

5.5.3 General Considerations on the Asymptotics of Predictive Identification 132

5.5.4 Estimating the Uncertainty in Parameter Estimation 132

5.5.4.1 Deduction of the Formula of the Estimation Covariance 134

5.6 Recursive Identification 138

5.6.1 Recursive Least Squares 138