This thesis deals with the development of an "alpha"-quantile estimate based on a surrogate model with the use of artificial neural
networks. Using artificial neural networks as an estimate is considered a nonparametric approach.
The estimation of a specific quantile of a data population is a widely used statistical task and a comprehensive way to discover the true relationship among variables. It can be classified as nonparametric regression, where it is one of the standard tasks. The most common selected levels for estimation are the first, second and third quartile (25, 50 and 75 percent). The quantile level is given by "alpha". A 25 percent quantile for example has 25 percent of the data distribution below the named quantile and 75 percent of the data distribution above it. Sometimes the tail regions of a population characteristic are of interest rather than the core of the distribution.
Quantile estimation is applied in many different contexts - financial economics, survival analysis and environmental modelling are
only a few of them.
Table of Contents
1 Introduction and Overview
2 Nonparametric regression
3 Nonparametric quantile estimation based on surrogate models
3.1 Introduction in Surrogate Models
3.2 Order Statistics
3.2.1 Asymptotic distribution of a central order statistic
3.3 A general error bound
3.3.1 Theorem
4 Neural Networks
4.1 Introduction
4.2 Biological neural networks
4.3 Historical Background
4.3.1 McCulloch-Pitts Model
4.3.2 Perceptron
4.4 Elements of an artificial neural network
4.5 Definitions
4.5.1 Sigmoid function
4.5.2 Squashing function
4.5.3 Artificial neuron
4.5.4 Feedforward neural network with hidden layers
4.5.5 A recursive definition of multilayered feedforward neural networks
4.6 Approximation characteristics of neural networks
4.6.1 Idea
4.6.2 Lemma 3 (An approximation result)
4.6.3 Lemma 4
5 Implementation
5.1 The Quantile estimates
5.2 Test Settings
5.3 Application on Simulated Data
5.4 Backpropagation algorithm
5.4.1 Gradient descent method
5.4.2 Phases of the Backpropagation algorithm
5.4.3 Training and Test Phase
5.4.4 Implementation
5.4.5 Initialising of the weights
5.4.6 Structure of the Parameters
5.4.7 Partial Derivatives
5.5 Comparison
5.6 Discussion
6 Conclusion and Outlook
Research Objectives and Core Topics
This thesis focuses on the development of an alpha-quantile estimate for costly-to-compute functions by utilizing surrogate models based on artificial neural networks. The research aims to establish a nonparametric approach to estimate quantiles efficiently by substituting the original complex function with a neural network approximation and subsequently applying Monte Carlo methods to handle larger sample sizes.
- Nonparametric quantile estimation techniques
- Development of surrogate models using artificial neural networks
- Application of the backpropagation algorithm for function approximation
- Comparative analysis of order statistic estimates versus Monte Carlo surrogate estimates
- Empirical investigation of estimator behavior on simulated data
Excerpt from the Book
3.3.1 Theorem
Theorem 2. Let X be an d-valued random variable, let m : d → be a measurable function and let α ∈ (0,1). The Monte Carlo surrogate quantile estimate qˆmn(X),Nn,α of qm(X),αis given as in the definition above. Let βn,δn > 0 be sequences such that the estimate mn satisfies the following for n ∈ :
|mn(x)−m(x)| ≤ δn 2 + 1 2 · |qm(X),α −m(x)| for PX -almost all x ∈ [−βn,βn] d. (10)
Furthermore, assume that
Nn ·P X ∈/ [−βn,βn] d → 0 for n → ∞. (11)
Then we get
|qˆmn(X),Nn,α −qm(X),α| = OP δn + 1 √Nn .
Remark 1. A possible choice for δn, which satisfies condition (10) is given by:
δn = 2 · ||mn −m||∞,supp(PX )∩[−βn,βn] d .
This is because with this δn we get
δn 2 + 1 2 |qm(x),α −m(x)| ≥ δn 2 = ||mn −m||∞,supp(PX )∩[−βn,βn] d .
Summary of Chapters
1 Introduction and Overview: Introduces the necessity of nonparametric quantile estimation for costly functions and outlines the structure of the thesis.
2 Nonparametric regression: Provides the theoretical foundations of nonparametric regression and discusses the use of L2 risk for evaluating estimation quality.
3 Nonparametric quantile estimation based on surrogate models: Defines the core problem of surrogate modeling, introduces order statistics, and presents error bounds for these estimations.
4 Neural Networks: Details the biological inspiration, history, and structural elements of artificial neural networks, including key approximation lemmas.
5 Implementation: Describes the practical application of the proposed methods using MATLAB, including the backpropagation algorithm and simulation settings.
6 Conclusion and Outlook: Reflects on the findings, discusses limitations of the current implementation, and suggests future improvements.
Keywords
Nonparametric regression, Quantile estimation, Artificial neural networks, Surrogate models, Monte Carlo estimate, Order statistics, Backpropagation, Error bounds, Simulation models, Function approximation, Hölder continuous functions, Statistical learning, Optimization, MATLAB, Computational complexity.
Frequently Asked Questions
What is the fundamental goal of this thesis?
The thesis aims to develop an alpha-quantile estimate for complex, costly functions by using artificial neural networks as surrogate models, which are faster to evaluate.
What are the primary thematic fields covered?
The work integrates nonparametric regression, neural network theory, statistical estimation, and Monte Carlo simulation techniques.
What is the core research question?
The research investigates how to effectively estimate quantiles of costly functions by approximating them with artificial neural networks and how the error of these estimates behaves.
Which scientific methods are applied?
The study employs nonparametric statistical estimation, neural network approximation theory, gradient descent-based backpropagation, and simulation-based validation.
What is covered in the main part of the work?
The main body covers theoretical error bounds for surrogate models, the architecture of feedforward neural networks, the implementation of backpropagation for weight optimization, and a comparative performance study on simulated data.
Which keywords characterize this work?
Key terms include Nonparametric regression, Quantile estimation, Artificial neural networks, Surrogate models, and Monte Carlo methods.
How does the backpropagation algorithm specifically optimize the neural network?
It uses a gradient descent method to iteratively minimize the cost function (MSE) by calculating partial derivatives of the weights and adjusting them in the direction of the negative gradient.
Why are "surrogate models" used in the context of costly computations?
They allow for the construction of an approximate function (mn) that is significantly cheaper to evaluate, enabling the generation of large samples that would be impossible with the original function (m).
What role do "order statistics" play in this research?
They provide the basis for the plug-in estimate of the alpha-quantile, which is then compared against the neural network-based Monte Carlo surrogate estimate.
- Quote paper
- Sevda Alaca (Author), 2017, Estimation of quantiles in a simulation model based on artificial neural networks, Munich, GRIN Verlag, https://www.grin.com/document/368611
