Applied Linear Statistical Models

Applied Linear Statistical Models

Michael H. Kutner
EmOlY University
Christopher J.  Nachtsheim
University of  Minnesota
John Neter
University of  Georgia
William Li
Universlty of  Minnesota

 Michael H. Kutner Christopher J. Nachtsheim John Neter Williamli  Linear statistical models for regression, analysis of variance, and experimental design are widely used today in business administration, economics, engineering, and the social, health, and biological sciences. Successful applications of these models require a sound understand- ing of both the underlying theory and the practical problems that are encountered in using the models in real-life situations. While Applied linear Statistical Models, Fifth Edition, is basically an applied book, it seeks to blend theory and applications effectively, avoiding the extremes of presenting theory in isolation and of giving elements of applications without the needed understanding of the theoretical foundations.

The fifth edition differs from the fourth in a number of important respects.

In the area of regression analysis (Parts I-III):

1. We have reorganized the chapters for better clarity and flow of topics. Material from the old Chapter 15 on normal correlation models has been integrated throughout the text where appropriate. Much of the material is now found in an expanded Chapter

2, which focuses on inference in regression analysis. Material from the old Chapter 7 pertaining to polynomial and interaction regression models and from old Chapter 11 on quantitative predictors has been integrated into a new Chapter 8 called, "Models for Quantitative and Qualitative Predictors." Material on model validation from old Chapter lOis now fully integrated with updated material on model selection in a new Chapter 9 entitled, "Building the Regression Model I: Model Selection and Validation."

2. We have added material on important techniques for data mining, including regression trees and neural network models in Chapters 11 and 13, respectively.

3. The chapter on logistic regression (Chapter 14) has been extensively revised and expanded to include a more thorough treatment of logistic, probit, and complemen- tary log-log models, logistic regression residuals, model selection, model assessment, logistic regression diagnostics, and goodness of fit tests. We have also developed new material on polytomous (multicategory) nominal logistic regression models and poly- tomous ordinal logistic regression models.

4. We have expanded the discussion of model selection methods and criteria. The Akaike information criterion and Schwarz Bayesian criterion have been added, and a greater emphasis is placed on the use of cross-validation for model selection and validation.

In the areas pertaining to the design and analysis of experimental and observational studies (Parts IV-VI):

5. In the previous edition, Chapters 16 through 25 emphasized the analysis of variance, and the design of experiments was not encountered formally until Chapter 26. We have completely reorganized Parts IV-VI, emphasizing the design of experimental and observational studies from the start. In a new Chapter 15, we provide an overview of the basic concepts and planning approaches used in the design of experimental and observational studies, drawing in part from material from old Chapters 16, 26, and

27. Fundamental concepts of experimental design, including the basic types of factors, treatments, experimental units, randomization, and blocking are described in detail.

This is followed by an overview of standard experimental designs, as well as the basic types of observational studies, including cross-sectional, retrospective, and prospective studies. Each of the design topics introduced in Chapter 15 is then covered in greater detail in the chapters that follow. We emphasize the importance of good statistical design of scientific studies, and make the point that proper design often leads to a simple analYSIS. We note that the statistical analysis techniques used for observational and experimental studies are often the same, but the ability to "prove" cause-and-effect requires a carefully designed experimental study.

6. Previously, the planning of sample sizes was covered -in Chapter 26. We now present material on planning of sample sizes in the relevant chapter, rather than devoting a single, general discussion to this issue.

7. We have expanded and updated our coverage (Section 24.2) on the interpretation of interaction plots for multi-factor studies.

8. We have reorganized and expanded the material on repeated measures designs in Chap- ter 27. In particular, we introduce methods for handling the analysis of factor effects when interactions between subjects and treatments are important, and when interactions between factors are important.

9. We have added material on the design and analysis of balanced incomplete block experiments in Section 28.1, including the planning of sample sizes. A new appendix (B.15) has been added that provides standard balanced incomplete block designs.

10. We have added new material on robust product and process design experiments in Chapter 29, and illustrate its use with a case study from the automotive industry. These experiments are frequently used in industrial studies to identify product or process designs that exhibit low levels of variation.

The remaining changes pertain to both regression analysis (Parts I-III) and the design and analysis of experimental and observational studies (Parts IV-VI):

11. We have made extensive revisions to the problem material. Problem data sets are generally larger and more challenging, and we have included a large number of new case data sets in Appendix C. In addition, we have added a new category of chapter exercises, called Case Studies. These are open-ended problems that require students, given an overall objective, to carry out complete analyses of the various case data sets in Appendix C. They are distinct from the material in the Problems and Projects sections, which frequently ask students to simply carry out specific .analytical procedures.

12. We have substantially expanded the amount of graphic presentation, including much greater use of scatter plot matrices, three-dimensional rotating plots, three-dimensional response surface and contour plots, conditional effects plots, and main effects and interaction plots.

13. Throughout the text, we have made extensive revisions in the exposition on the basis of classroom experience to improve the clarity of the presentation.

We have included in this book not only the more conventional topics in regression and design, but also topics that are frequently slighted, though important in practice. We devote three chapters (Chapters 9-11) to the model-building process for regression, including computer-assisted selection procedures for identifying good subsets of predictor variables The Student Solutions Manual and all of the data files on the compact disk can also be downloaded from the book's website at: www.mhhe.com/kutnerALSM5e.Alist of errata for the book as well as some useful, related links will also be maintained at this address.

a. book such as this cannot be written without substantial assistance from numerous persons. We are indebted to the many contributors who have developed the theory and practice discussed in this book. We also would like to acknowledge appreciation to our stu- dents, who helped us in a variety of ways to fashion the method of presentation contained herein. We are grateful to the many users of Applied Linear Statistical Models and Applied Linear Regression Models, who have provided us with comments and suggestions based on their teaching with these texts. We are also indebted to Professors James E. Holstein, University of Missouri, and David L. Sherry, University of West Florida, for their review of Applied Linear Statistical Models, First Edition; to Professors Samuel Kotz, University of Maryland at College Park, Ralph P. Russo, University ofIowa, and Peter F. Thall, The George Washington University, for theirreview of Applied Linear Regression Models, First Edition;

to Professors John S. Y Chiu, University of Washington, James A. Calvin, University of Iowa, and Michael F. Driscoll, Arizona State University, for their review of Applied Linear Statistical Models, Second Edition; to Professor Richard Anderson-Sprecher, University of Wyoming, for his review of Applied Linear Regression Models, Second Edition; and to Professors Alexander von Eye, The Pennsylvania State University, Samuel Kotz, University of Maryland at College Park, and John B. Willett, Harvard University, for their review of Applied Linear Statistical Models, Third Edition; to Professors Jason Abrevaya, Univer- sity of Chicago, Frank Alt, University of Maryland, Vitoria Chen, Georgia Tech, Rebecca Doerge, Purdue University, Mark Henry, Clemson University, Jim Hobert, University of Florida, Ken Koehler, Iowa State University, Chii-Dean Lin, University of Massachussets Amherst, Mark Reiser, Arizona State University, Lawrence Ries, University of Missouri Columbia, and Ehsan Soofi, University of Wisconsin Milwaukee, for their reviews of Applied Linear Regression Models, Third Edition, or Applied Linear Statistical Models, Fourth Edition. These reviews provided many important suggestions, for which we are most grateful.

In addition, valuable assistance was provided by Professors Richard K. Burdick, Arizona State University, R. Dennis Cook, University of Minnesota. W. J. Conover, Texas Tech University, Mark E. Johnson, University of Central Florida. Dick DeVeaux, Williams College, and by Drs. Richard I. Beckman, Los Alamos National Laboratory, Ronald L.

Iman, Sandia National Laboratories, Lexin Li, University of California Davis, and Brad Jones, SAS Institute. We are most appreciative of their willing help. We are also indebted to the 88 participants in a survey concerning Applied Linear Regression Models, Second Edition, the 76 participants in a survey concerning Applied Linear Statistical Models, Third Edition, and the 73 participants in a survey concerning Applied Linear Regression Models, Third Edition, or Applied Linear Statistical Models, Fourth Edition. Helpful suggestions were received in these surveys, for which we are thankful.

Weiyong Zhang and Vincent Agboto assisted us diligently in the development of new problem material, and Lexin Li and Yingwen Dong helped prepare the revised Instructor Solutions Manual and Student Solutions Manual under considerable time pressure. Amy Hendrickson provided much-needed LaTeX expertise. George Cotsonis assisted us dili- gently in preparing computer-generated plots and in checking analysis results. We are most grateful to these persons for their invaluable help and assistance. We also wish to thank the various members of the Carlson Executive MBA Program classes of 2003 and 2004;

notably Mike Ohmes, Trevor Bynum, Baxter Stephenson, Zakir Salyani, Sanders Marvin, Trent Spurgeon, Nate Ogzawalla, David Mott, Preston McKenzie, Bruce Dejong, and TIm Kensok, for their contributions of interesting and relevant case study data and materials.

Finally, our families bore patiently the pressures caused by our commitment to complete this revision. We are appreciative of their understanding.

Content

 PART ONE SIMPLE LINEAR REGRESSION

Chapter 1 Linear Regression with One Predictor Variable

Chapter 2 Inferences in Regression and Correlation Analysis

Chapter 3 Diagnostics and Remedial Measures

Chapter 4 Simultaneous Inferences and Other Topics in Regression Analysis

Chapter 5 Matrix Approach to Simple Linear Regression Analysis

 PART TWO MULTIPLE LINEAR REGRESSION

Chapter 6 Multiple Regression I

Chapter 7 Multiple Regression II

Chapter 8 Regression Models for Quantitative and Qualitative Predictors

Chapter 9 Building the Regression Model I: Model Selection and Validation

Chapter 10 Building the Regression Model II: Diagnostics

Chapter 11 Building the Regression Model III: Remedial Measures

Chapter 12 Autocorrelation in Time Series Data

 PART THREE NONLINEAR REGRESSION

Chapter 13 Introduction to Nonlinear Regression and Neural Networks

Chapter 14 Logistic Regression, Poisson Regression, and Generalized Linear Models

 PART FOUR DESIGN AND ANALYSIS OF SINGLE-FACTOR STUDIES

Chapter 15 Introduction to the Design of Experimental and Observational Studies

Chapter 16 Single-Factor Studies

Chapter 17 Analysis of Factor Level Means

Chapter 18 ANOVA Diagnostics and Remedi!l Measures

 PART FIVE MULTI-FACTOR STUDIES

Chapter 19 Two-Factor Studies with Equal Sample Sizes

Chapter 20 Two-Factor Studies-One Case per Treatment

Chapter 21 Randomized Complete Block Designs

Chapter 22 Analysis of Covariance

Chapter 23 Two-Factor Studies with Unequal Sample Sizes

Chapter 24 Multi-Factor Studies

Chapter 25 Random and Mixed Effects Models

 PART SIX SPECIALIZED STUDY DESIGNS

Chapter 26 Nested Designs, SubsampJing, and Partially Nested Designs

Chapter 27 Repeated Measures and Related Designs

Chapter 28 Balanced Incomplete Block, Latin Square, and Related Designs

Chapter 29 Exploratory Experiments: Two-Lev Factorial and Fractional Factorial Designs

Chapter 30 Response Surface Methodology

Appendix A Some Basic Results in Probab and Statistics

Appendix B Tables

Appendix C Data Sets

Appendix D Rules for Developing ANOVA Tables for Balanced Designs

Appendix E Selected Bibliography

Index