ROBUST STATISTICS - Peter J, Huber

ROBUST STATISTICS - Peter J, Huber

The present monograph is the first systematic, book - length exposition of robust statistics. The technical term “robust” was coined only in 1953 (by G. E. P. Box), and the subject matter acquired recognition as a legitimate topic for investigation only in the mid - sixties, but it certainly never was a revolutionary new concept. Among the leading scientists of the late nineteenth and early twentieth century, there were several practicing statisticians (to name but a few: the astronomer S. Newcomb, the astrophysicist A. S. Eddington, and the geophysicist H. Jeffreys), who had a perfectly clear, operational understanding of the idea; they knew the dangers of long - tailed error distributions, they proposed probability models for gross errors, and they even invented excellent robust alternatives to the standard estimates, which were rediscovered only recently. But for a long time theoretical statisticians tended to shun the subject as being inexact and “dirty.” My 1964 paper may have helped to dispel such prejudices. Amusingly (and disturbingly), it seems that lately a kind of bandwagon effect has evolved, that the pendulum has swung to the other extreme, and that “robust” has now become a magic word, which is invoked in order to add respectability.

This book gives a solid foundation in robustness to both the theoretical and the applied statistician. The treatment is theoretical, but the stress is on concepts, rather than on mathematical completeness. The level of presentation is deliberately uneven:

in some chapters simple cases are treated with mathematical rigor; in others the results obtained in the simple cases are transferred by analogy to more complicated situations (like multiparameter regression and covariance matrix estimation), where proofs are not always available (or are available only under unrealistically severe assumptions). Also selected numerical algorithms for computing robust estimates are described and, where possible, convergence proofs are given.

Chapter 1 gives a general introduction and overview; it is a must for every reader.

Chapter 2 contains an account of the formal mathematical background behind qual - itative and quantitative robustness, which can be skipped (or skimmed) if the reader is willing to accept certain results on faith. Chapter 3 introduces and discusses the three basic types of estimates (M - , L - , and R - estimates), and Chapter 4 treats the asymptotic minimax theory for location estimates; both chapters again are musts. The remaining chapters branch out in different directions and are fairly independent and self - contained; they can be read or taught in more or less any order.

The book does not contain exercises - I found it hard to invent a sufficient number of problems in this area that were neither trivial nor too hard - so it does not satisfy some of the formal criteria for a textbook. Nevertheless I have successfully used various stages of the manuscript as such in graduate courses. The book also has no pretensions of being encyclopedic. I wanted to cover only those aspects and tools that I personally considered to be the most important ones.

Some omissions and gaps are simply due to the fact that I currently lack time to fill them in, but do not want to procrastinate any longer (the first draft for this book goes back to 1972). Others are intentional. For instance, adaptive estimates were excluded because I would now prefer to classify them with nonparametric rather than with robust statistics, under the heading of nonparametric efficient estimation. The so - called Bayesian approach to robustness confounds the subject with admissible estimation in an ad hoc parametric supermodel, and still lacks reliable guidelines on how to select the supermodel and the prior so that we end up with something robust.

The coverage of L - and R - estimates was cut back from earlier plans because they do not generalize well and get awkward to compute and to handle in multiparameter situations a large part of the final draft was written when I was visiting Harvard University in the fall of 1977; my thanks go to the students, in particular to P. Rosenbaum and Y. Yoshizoe, who then sat in my seminar course and provided many helpful comments.

PETER J. HUBER.

update bibliographical references, so the manuscript of the second edition could be based on a re - keyed version of the first. Other aspects deserved a more extended discussion. I was fortunate to persuade Elvezio Ronchetti, who had been one of the prime researchers working in the two last mentioned areas (robust tests and small sample asymptotics), to collaborate and add the corresponding Chapters 13 and 14.

Also, I extended the discussion of regression, and I decided to add a chapter on Bayesian robustness - even though, or perhaps because, I am not a Bayesian (or only rarely so). Among other minor changes, since most readers of the first edition had appreciated the General Remarks at the beginning of the chapters, I have expanded some of them and also elsewhere devoted more space to an informal discussion of motivations.

The new edition still has no pretensions of being encyclopedic. Like the first, it is centered on a robustness concept based on minimax asymptotic variance and on M - estimation, complemented by some exact finite sample results. Much of the material of the first edition is just as valid as it was in 1980. Deliberately, such parts were left intact, except that bibliographical references had to be added. Also, I hope that my own perspective has improved with an increased temporal and professional distance. Although this improved perspective has not affected the mathematical facts, it has sometimes sharpened their interpretation.

Special thanks go to Amy Hendrickson for her patient help with the Wiley LA# - macros and the various quirks of T#.

..

Content of this e-book

Generalities

Why Robust Procedures?

What Should a Robust Procedure Achieve?

Robust, Nonparametric, and Distribution-Free

Adaptive Procedures

Resistant Procedures

Robustness versus Diagnostics

Breakdown point

Qualitative Robustness

Quantitative Robustness

Infinitesimal Aspects

Optimal Robustness

Performance Comparisons

Computation of Robust Estimates

Limitations  to Robustness Theory

The Weak Topology and its Metrization

General Remarks

The Weak Topology

LCvy and Prohorov Metrics

The Bounded Lipschitz Metric

FrCchet and GQeaux Derivatives

Hampel’s Theorem

The Basic Types of Estimates

General Remarks

Maximum Likelihood Type Estimates  (M-Estimates)

Influence Function of M-Estimates

Asymptotic Properties of M-Estimates

Linear Combinations of Order Statistics (L-Estimates)

Influence Function of L-Estimates

Estimates Derived from Rank Tests (R-Estimates)

Influence Function of R-Estimates

Asymptotically Efficient M-,  L-, and R-Estimates

Quantitative and Qualitative Robustness  of  M- Estimates

Quantitative and Qualitative Robustness of L-Estimates

Quantitative and Qualitative Robustness of R-Estimates

Asymptotic Minimax Theory for Estimating Location

General Remarks

Minimax Bias

Minimax Variance: Preliminaries

Distributions Minimizing Fisher Information

Determination of FO  by Variational Methods

Asymptotically Minimax M-Estimates

On the Minimax Property for L- and R-Estimates

Redescending M-Estimates

Questions of Asymmetric Contamination

Scale Estimates

General Remarks

M-Estimates of Scale

L-Estimates of Scale

R-Estimates of Scale

Asymptotically Efficient Scale Estimates

Minimax Properties

Distributions Minimizing Fisher Information for Scale

Multiparameter  Problems-in  Particular  Joint  Estimation

of Location and Scale

General Remarks

Consistency of M-Estimates

Asymptotic Normality of M-Estimates

Simultaneous M-Estimates of Location and Scale

M-Estimates with Preliminary Estimates of Scale

Quantitative Robustness of Joint Estimates of Location and Scale

The Computation of M-Estimates of Scale

Studentizing

Regression

General Remarks

The Classical Linear Least Squares Case

Residuals and Outliers

Robustizing the Least Squares Approach

Asymptotics of Robust Regression Estimates

The Cases hp +   and hp -+

Conjectures and Empirical Results

Symmetric Error Distributions

The Question of Bias

Asymptotic Covariances and Their Estimation

Concomitant Scale Estimates

Computation of Regression M-Estimates

The Scale Step

The Location Step with Modified Residuals

The Location Step with Modified Weights

Analysis of Variance

The Fixed Carrier Case: What Size hi?

L-estimates and Median Polish

Other Approaches to Robust Regression

Robust Covariance and Correlation Matrices

General Remarks

Estimation of Matrix Elements Through Robust Variances

Estimation of Matrix Elements Through Robust Correlation

An Affinely Equivariant Approach

Estimates Determined by Implicit Equations

Existence and Uniqueness of Solutions

The Scatter Estimate V

The Location Estimate t

Influence Functions and Qualitative Robustness

Consistency and Asymptotic Normality

Breakdown Point

Least Informative Distributions

Location

Covariance

Some Notes on Computation

Joint Estimation oft  and V

Robustness of Design

General Remarks

Minimax Global Fit

Minimax Slope

Exact Finite Sample Results

General Remarks

Lower and Upper Probabilities and Capacities

-Monotone and -Alternating Capacities

Robust Tests

Monotone and Alternating Capacities of Infinite Order

Particular Cases   Sequential Tests

Estimates Derived From Tests

Minimax Interval Estimates

The Neyman-Pearson Lemma for -Alternating Capacities

Finite Sample Breakdown Point

General Remarks

Definition and Examples

Variances and Covariances

One-dimensional M-estimators of Location

Multidimensional Estimators of Location

Structured Problems: Linear Models

Infinitesimal Robustness and Breakdown

Malicious versus Stochastic Breakdown

Infinitesimal Robustness

General Remarks

Hampel’s Infinitesimal Approach

Shrinking Neighborhoods

Robust Tests

General Remarks

Local Stability of a Test

Tests for General Parametric Models in the Multivariate Case

Robust Tests for Regression and Generalized Linear Models

Small Sample Asymptotics

General Remarks

Tail Probabilities

Marginal Distributions

Saddlepoint Test

Relationship with Nonparametric Techniques

Appendix

Saddlepoint Approximation for the Mean

Saddlepoint Approximation of the Density of M-estimators

Bayesian Robustness

General Remarks

Some Asymptotic Theory

Minimax Asymptotic Robustness Aspects

Nuisance Parameters

Disparate Data and Problems with the Prior

Maximum Likelihood and Bayes Estimates

Why there is no Finite Sample Bayesian Robustness Theory

References

Index