Ranking Analysis for Expectation of Binary Outcomes. A Bayesian Approach

Table of Contents

Abstract

Introduction

Probability Distributions of the Initial Beliefs

Uninformed Uniform Prior Distribution

Pre-calibrated Prior Distribution

Probability Distributions with Observed Data

Posterior Distribution from Un-informed Prior

Posterior Distribution from Informed Beta Prior

Probability Distributions with Multi-period Observations

Through-The-Cycle Posterior Distributions from Un-informed Prior

Through-The-Cycle Posterior Distributions from Informed Beta Prior

Conclusion

References

Ranking Analysis for Expectation of Binary Outcomes: A Bayesian Approach

WORKING DRAFT: May 16, 2017

Yang Liu¹

Abstract

The estimation and grading of customer Probability of Default (PD) using a firm- wide master scale for internal portfolios is a well established procedure in many financial institu- tions. However, there are often discussions within the institutions about the comparison between the grade PDs, the portfolio average PD calculated across grades and the observed Default Rate

(DR). The Bayesian estimator for the grade PD based on different prior distribution assumption is often referred to as an alternative over the simple default rate. In this paper, we apply and compare the approaches to derive the posterior distribution of grade PDs that is closely linked to the portfolio performance assumptions made according to the Master Scale. We then detail the numerical analysis of both informed and uninformed Bayesian estimators for the Grade PDs with simulated single and multi-period data sets. We compare the results in term of prior and posterior Density-in-Range (DiR) and Direction-of-Movement (DoM) for each Grade. With data analysis and preference of criteria, the approach and metrics can be used for both model development and validation, or other risk-awareness reporting and information update exercises to help understand the difference between expectation and observed data. More importantly, this approach provides a methodology to quantify such differences according to knowledge and understanding of the observer, which was not possible for simple approaches such as direct nu- merical comparison of Master Scale defined PD and the simple rate.

Keywords: Portfolio Default Rate, PD Master Scale, Grade Mean PD, Bayesian PD Estimate, Posterior Density in Range, Posterior Direction of Movement.

Introduction

It is common practice for banks or financial institutions to establish and maintain a Probability of Default Master Scale to measure and manage the default risk and the relative allocated exposure (See 1 and 2). The Master Scale itself could be derived in several ways including: a numerical scale with boundaries derived from pre-assumed probability distribution, a direct mapping between external ratings and derived default probabilities, or a combination of both.

Table 1: Example of a 9-grade Master Scale

illustration not visible in this excerpt

Table 1 is an example of a 9-grade Master Scale that is derived from a predefined PD density. A typical Master Scale includes Grade names, representative PD for the Grade, and the PD boundaries for each Grade. Some Master Scales include a score mapping for each Grade to cross reference an internal scoring system. Other Master Scales may choose to include a rating-Grade mapping to benchmark external ratings (See 3 and 4 for more details).

From the modelling perspective, a firm’s internal model could estimate a score that is mapped into a grade using the lower and higher score boundaries. Or, the internal model could estimate a PD² and the grade is then mapped using the model estimated PD and the associated lower and higher PD boundaries. Either way, the model estimate is converted into a grade which has a single representative PD value as the probability of default for all customers in the same grade.

Assume that the internal model categorised 100 customer into Grade 4 shown in the Table 1, this is equivalent in saying that the estimated probability of default for each of the 100 customers is the same, 1 . 30% according to the table.

Because the real outcome of a default/non-default event is binary, one may wish to consider the simplified example where we look at 100 biased coins where the initial guess for the bias in terms of probability to get a Head (H) is 1 . 30%. At the time when outcomes are observed, the binary outcome here is either obtaining a Head (H) or Tail (T) from each one of the 100 coins. (See 5)

In the next few sections, we demonstrate in details how to establish the initial expectation given a set of boundaries and a fixed representative PD, and then discuss how to link the observed results and with one’s prior beliefs to adjust and assess the post-observation estimated PD for each population that was graded according to the Master Scale.

Probability Distributions of the Initial Beliefs

A model is expected to be able to classify the customers into different Grades. Back to the example discussed in the previous section, bucketing 100 coins together and assigning a homogeneous representative PD of 1 . 30% between the boundaries of 0 . 66% and 1 . 94%, is to assume that all the true individual PDs are seen equivalent within this range. (See 5)

As with all Bayesian analysis, it is common to have two natural candidates for the prior assump- tions:

Uninformed Prior Belief: The choice of this kind prior usually leads to the analysis using the Uniform Distribution, as the uninformed prior knowledge impose a negative review of the model as this is equivalent to say that ”the model has equal probability to categorise the customer in this grade as any other grades.”

Pre-calibrated Prior Belief: On the contrary, one can impose a strong belief in the internal model by choose to use a distribution of which the majority of the probability density is located in the target PD range. A 95% probability density coverage within each Grade is assumed in the rest of this paper for informed prior assumptions.

For example, we can assume a probability distribution over the range between 0 and 1, where at least 95% of the density is skewed into the range from 0 . 66% to 1 . 94% while the mean of the distribution is 1 . 30%. Here the range between the boundaries is defined as the range of hypothetical equivalence as defined by the Master Scale, and 95% of the distribution locating within these boundaries ensure that the mean of samples from the distribution is located in the centre of this range.

Uninformed Uniform Prior Distribution

We start with the scenario where no expectation is known to the portfolio reviewer. Obviously, neither the 0 nor 1 event is included in the PD range of (0 , 1). So an uninformed distribution is equivalent to assuming “every probability in the grade is equally possible” for a model graded customer, e.g. all probabilities in Grade 7 is equally possible and NOT peaked or centred at the mid point of the Grade.

The Uniform prior is often regarded as the improper or vague prior (See 6) as it does not fully satisfy some of the statistical properties for a distribution. This is acceptable in case for prior distributions as it will adjust to the latest observation at the time of information update.

A typical Uniform distribution (more in 7) is mathematically defined as following: {

illustration not visible in this excerpt

Here in Figure 1 is the density plot of expectations according to the uninformed assumption for Grade 7 at this stage before observing the outcome. The lines and markings for Figure 1 and through out this paper is defined as following:

- The green dotted line is the mean of the post observation distribution. Later in section for Multi-period analsis, this indicates the mean of the distribution calibrated for the whole observation period.
- The blue dotted line is the boundaries for each Grade. In Figure 1 it is Grade 7, where the lower bound is 10 . 75% and the higher bound is 20 . 43%.
- The solid blue line marks the mid point representative probability of the Grade. Here for Grade 7, this is 15 . 59%.
- The dark red line marks the simple Default Rate observed in a single year.

Figure 1: Uninformed density for Grade 7.

illustration not visible in this excerpt

Figure 2 is an overview of all Grades under the uninformed assumption. The percentage in each Grade reported under the sub plots is the probability covered by the prior distributions calibrated for each Grade, the percentage is exactly 100 . 00% in this case.

Figure 2: Uninformed distribution for all Grades.

illustration not visible in this excerpt

Pre-calibrated Prior Distribution

It is well-known that the observed outcome of credit default is binary, thus the distribution function of the number of defaults can be written in terms of a Beta-distribution (see 9 for more details). Define p as the default rate, the probability distribution (more in 8) function can be

written as:

illustration not visible in this excerpt

Similar to the uninformed assumption discussed in the previous section, one may wish to establish an alternative prior assumption that is informed with different initial beliefs. To illustrate the idea, we assume the model estimations are as described below:

1) Each Grade is different, as customers in different Grades are expected to follow different behaviour. Therefore the probability distribution, although from the same distribution family, is different from Grade to Grade.
2) Majority of the distribution density is located within the boundaries of each Grade. For example, one may assume that there are exactly 95% of the density between the boundaries. Or alternatively the assumption could be that there are at least 95% of the density located between the boundaries.
3) The mean or median of the distribution is located at the representative probability (mid point) of the Grade.

Figure 3 demonstrate the calibrated prior distribution according to the prior beliefs as described above, the lines and markers in the Figure is as described in earlier section before Figure 1.

[...]

¹ Yang Liu is a quantitative specialist at the HSBC Bank. Before joining the HSBC, he worked for the Barclays Bank. Yang holds a doctorate in quantitative finance from Cass Business School, City University of London. He has published a number of papers on quantitative methods in risk and finance. The opinions expressed in this paper are those of the author and do not necessarily reflect views of the HSBC. HSBC, 8CS Canary Wharf, London E14 5HQ, UK

² In case the PD is used directly without mapping into a Grade, the Grade analysis in this paper is equivalent to analysis on the portfolio Central Tendency.