A guide to the systematic analytical results for ridge, LASSO, preliminary test, and Stein-type estimators with applications
Theory of Ridge Regression Estimation with Applications offers a comprehensive guide to the theory and methods of estimation. Ridge regression and LASSO are at the center of all penalty estimators in a range of standard models that are used in many applied statistical analyses. Written by noted experts in the field, the book contains a thorough introduction to penalty and shrinkage estimation and explores the role that ridge, LASSO, and logistic regression play in the computer intensive area of neural network and big data analysis.
Designed to be accessible, the book presents detailed coverage of the basic terminology related to various models such as the location and simple linear models, normal and rank theory-based ridge, LASSO, preliminary test and Stein-type estimators.The authors also include problem sets to enhance learning. This book is a volume in the Wiley Series in Probability and Statistics series that provides essential and invaluable reading for all statisticians. This important resource:
* Offers theoretical coverage and computer-intensive applications of the procedures presented
* Contains solutions and alternate methods for prediction accuracy and selecting model procedures
* Presents the first book to focus on ridge regression and unifies past research with current methodology
* Uses R throughout the text and includes a companion website containing convenient data sets
Written for graduate students, practitioners, and researchers in various fields of science, Theory of Ridge Regression Estimation with Applications is an authoritative guide to the theory and methodology of statistical estimation.
Autorentext
A. K. Md. EHSANES SALEH, PhD, is a Professor Emeritus and Distinguished Research Professor in the school of Mathematics and Statistics, Carleton University, Ottawa, Canada.
MOHAMMAD ARASHI, PhD, is an Associate Professor at Shahrood University of Technology, Iran and Extraordinary Professor and C2 rated researcher at University of Pretoria, Pretoria, South Africa.
B. M. GOLAM KIBRIA, PhD, is a Professor in the Department of Mathematics and Statistics at Florida International University, Miami, FL.
Klappentext
Theory of Ridge Regression Estimation with Applications
A guide to the systematic analytical results for ridge, LASSO, preliminary test, and Stein-type estimators with applications
Theory of Ridge Regression Estimation with Applications offers a comprehensive guide to the theory and methods of estimation. Ridge regression and LASSO are at the center of all penalty estimators in a range of standard models that are used in many applied statistical analyses. Written by noted experts in the field, the book contains a thorough introduction to penalty and shrinkage estimation and explores the role that ridge, LASSO, and logistic regression play in the computer intensive area of neural network and big data analysis.
Designed to be accessible, the book presents detailed coverage of the basic terminology related to various models such as the location and simple linear models, normal and rank theory-based ridge, LASSO, preliminary test and Stein-type estimators. The authors also include problem sets to enhance learning. This book is a volume in the Wiley Series in Probability and Statistics series that provides essential and invaluable reading for all statisticians. This important resource:
- Offers theoretical coverage and computer-intensive applications of the procedures presented
- Contains solutions and alternate methods for prediction accuracy and selecting model procedures
- Presents the first book to focus on ridge regression and unifies past research with current methodology
Written for graduate students, practitioners, and researchers in various fields of science, Theory of Ridge Regression Estimation with Applications is an authoritative guide to the theory and methodology of statistical estimation.
Inhalt
List of Figures xvii
List of Tables xxi
Preface xxvii
Abbreviations and Acronyms xxxi
List of Symbols xxxiii
1 Introduction to Ridge Regression 1
1.1 Introduction 1
1.1.1 Multicollinearity Problem 3
1.2 Ridge Regression Estimator: Ridge Notion 5
1.3 LSE vs. RRE 6
1.4 Estimation of Ridge Parameter 7
1.5 Preliminary Test and Stein-Type Ridge Estimators 8
1.6 High-Dimensional Setting 9
1.7 Notes and References 11
1.8 Organization of the Book 12
2 Location and Simple Linear Models 15
2.1 Introduction 15
2.2 Location Model 16
2.2.1 Location Model: Estimation 16
2.2.2 Shrinkage Estimation of Location 17
2.2.3 Ridge RegressionType Estimation of Location Parameter 18
2.2.4 LASSO for Location Parameter 18
2.2.5 Bias and MSE Expression for the LASSO of Location Parameter 19
2.2.6 Preliminary Test Estimator, Bias, and MSE 23
2.2.7 Stein-Type Estimation of Location Parameter 24
2.2.8 Comparison of LSE, PTE, Ridge, SE, and LASSO 24
2.3 Simple Linear Model 26
2.3.1 Estimation of the Intercept and Slope Parameters 26
2.3.2 Test for Slope Parameter 27
2.3.3 PTE of the Intercept and Slope Parameters 27
2.3.4 Comparison of Bias and MSE Functions 29
2.3.5 Alternative PTE 31
2.3.6 Optimum Level of Significance of Preliminary Test 33
2.3.7 Ridge-Type Estimation of Intercept and Slope 34
2.3.7.1 Bias and MSE Expressions 35
2.3.8 LASSO Estimation of Intercept and Slope 36
2.4 Summary and Concluding Remarks 39
3 ANOVA Model 43
3.1 Introduction 43
3.2 Model, Estimation, and Tests 44
3.2.1 Estimation of Treatment Effects 45
3.2.2 Test of Significance 45
3.2.3 Penalty Estimators 46
3.2.4 Preliminary Test and Stein-Type Estimators 47
3.3 Bias and Weighted L2 Risks of Estimators 48
3.3.1 Hard Threshold Estimator (Subset Selection Rule) 48
3.3.2 LASSO Estimator 49
3.3.3 Ridge Regression Estimator 51
3.4 Comparison of Estimators 52
3.4.1 Comparison of LSE with RLSE 52
3.4.2 Comparison of LSE with PTE 52
3.4.3 Comparison of LSE with SE and PRSE 53
3.4.4 Comparison of LSE and RLSE with RRE 54
3.4.5 Comparison of RRE with PTE, SE, and PRSE 56
3.4.5.1 Comparison Between 𝜽n^RR (kopt) and 𝜽n^PT (𝛼) 56
3.4.5.2 Comparison Between 𝜽n^RR (kopt) and 𝜽n^ s 56
3.4.5.3 Comparison of 𝜽n^RR (kopt) with 𝜽n^S+ 57
3.4.6 Comparison of LASSO with LSE and RLSE 58
3.4.7 Comparison of LASSO with PTE, SE, and PRSE 59
3.4.8 Comparison of LASSO with RRE 60
3.5 Application 60
3.6 Efficiency in Terms of Unweighted L2 Risk 63
3.7 Summary and Concluding Remarks 72
3A. Appendix 74
4 Seemingly Unrelated Simple Linear Models 79
4.1 Model, Estimation, and Test of Hypothesis 79
4.1.1 LSE of 𝜃 and 𝛽 80
4.1.2 Penalty Estimation of 𝛽 and 𝜃 80
4.1.3 PTE and Stein-Type Estimators of 𝛽 and 𝜃 81
4.2 Bias and MSE Expressions of the Estimators 82
4.3 Comparison of Estimators 86
4.3.1 Comparison of LSE with RLSE 86
4.3.2 Comparison of LSE with PTE 86
4.3.3 Comparison of LSE with SE and PRSE 87
4.3.4 Co…