Least Squares Regressions with the Bootstrap

A Survey of their Performance

Diploma Thesis , 2009 , 51 Pages , Grade: 1,6

Mathematics - Statistics

The statistical technique called bootstrap is usable with a lot of inferential problems and it is the main topic of this paper. Since the bootstrap provides material for a whole series of books
it is essential to pick one special aspect of the bootstrap and investigate it in depth, otherwise the analysis would inevitably become too general. This aspect is the topic of regression. Hence, this paper will introduce the bootstrap and compare the performance of the new inference methods which it provides with some classical methods of judging a regression which were used in the years before the bootstrap.
Therefore the remainder of this paper is as follows: First there will be a description of the basic model in which all of the following investigations will be done, chapter two. The next chapter will describe the different regression techniques which try to solve the model.
The fourth chapter is going to show the behavior of these regression techniques in large samples, i.e. shows some classical methods of statistical inference. Following chapter five will give an introduction to the bootstrap which will be succeeded by a description of the bootstrap in regression problems, chapter six. The seventh chapter will show how inference is done with the help of the bootstrap. The eighth chapter is going to compare the
performances of classical and bootstrap inference in regressions. Before the concluding remarks of chapter ten, there will be a practical application in chapter nine which tries to prove some observations of the preceeding chapters.

Inhaltsverzeichnis (Table of Contents)

• Introduction
• The model
  • The basic form
  • The disturbance term
• Regression techniques
  • The method of least squares
    • Ordinary Least Squares
    • Generalized Least Squares
  • Alternative regression methods
• Classical measures of performance
  • Bias
  • Variances
    • The variance of OLS
    • The variance of GLS
    • A remark on the variances
  • Confidence intervals
    • A remark on the critical values
    • A confidence interval for OLS
    • A confidence interval for GLS
  • Rate of convergence
• The bootstrap
  • How does the bootstrap work?
  • When does the bootstrap work?
  • The non-parametric bootstrap
  • The parametric bootstrap
  • Why does the bootstrap work?
  • How many bootstrap repetitions?
  • On the size of each repetition
• Regressions with the bootstrap
  • Case resampling
  • Residual resampling
  • Wild bootstrap
  • When to use which method?
• Inference with the bootstrap
  • Variances with the bootstrap
  • Confidence intervals with the bootstrap
    • The percentile interval
    • The bootstrap-t interval
    • Other bootstrap intervals
  • Convergence with the bootstrap
• Classical or bootstrap inference?
• A practical test for the bootstrap
  • The datasets
    • The homoscedastic data
    • The heteroscedatic data
  • The simulations
    • Results simulation one
    • Results simulation two
  • Resumé of simulations
• Concluding remarks

Zielsetzung und Themenschwerpunkte (Objectives and Key Themes)

This paper introduces the bootstrap method and compares its performance in regression problems with classical methods of statistical inference. The paper aims to provide a comprehensive overview of the bootstrap and its applicability in regressions, demonstrating its effectiveness in situations where traditional methods may fall short. Key themes explored in the paper include:

• The application of the bootstrap in regression problems
• A comparative analysis of the bootstrap's performance against classical inference methods
• The limitations and effectiveness of the bootstrap in different regression settings
• The impact of the bootstrap on the estimation of variances and confidence intervals
• A practical demonstration of the bootstrap's performance through simulations

Zusammenfassung der Kapitel (Chapter Summaries)

The paper begins with a brief introduction to the concept of regression in statistics and highlights the challenges associated with selecting the most appropriate regression technique. The second chapter describes the basic model used throughout the paper, outlining the underlying assumptions and concepts. Chapter 3 introduces various regression techniques, focusing on the method of least squares, including Ordinary Least Squares (OLS) and Generalized Least Squares (GLS), as well as alternative regression methods. The fourth chapter delves into classical measures of performance for regression techniques, examining bias, variance, confidence intervals, and the rate of convergence in large samples. Chapter 5 provides an introduction to the bootstrap method, explaining its principles, applications, and advantages. The next chapter, chapter 6, specifically addresses the application of the bootstrap in regression problems, detailing methods like case resampling, residual resampling, and wild bootstrap. Chapter 7 explores how inference is conducted using the bootstrap, analyzing the estimation of variances, confidence intervals, and convergence. Chapter 8 compares the performances of classical and bootstrap inference in regressions, highlighting strengths and weaknesses of each approach. Finally, chapter 9 presents a practical test of the bootstrap method using simulations, evaluating its performance in different scenarios.

Schlüsselwörter (Keywords)

The paper focuses on the key concepts of least squares regressions, the bootstrap method, and their respective strengths and limitations in statistical inference. This includes exploring topics such as:

• Ordinary Least Squares (OLS)
• Generalized Least Squares (GLS)
• Bootstrap inference
• Classical inference
• Variances and confidence intervals
• Regression techniques
• Simulations

• College: University of Bonn (Statistische Abteilung der Rechts- und Staatswissenschaftlichen Fakultät)
• Course: Diplomarbeit bei Prof.Dr. Alois Kneip
• Grade: 1,6
• Author: Diplom Volkswirt Jonas Böhmer (Author)
• Publication Year: 2009
• Pages: 51
• Catalog Number: V135688
• ISBN (eBook): 9783640422418
• ISBN (Book): 9783640421831
• Language: English
• Tags: Least, Squares, Regressions, Bootstrap, Survey, Performance