Computing highest density regions for continuous univariate distributions with known probability functions

Abstract

We examine the problem of computing the highest density region (HDR) in a computational context where the user has access to a density function and quantile function for the distribution (e.g., in the statistical language R). We examine several common classes of continuous univariate distributions based on the shape of the density function; this includes monotone densities, quasi-concave and quasi-convex densities, and general multimodal densities. In each case we show how the user can compute the HDR from the quantile and density functions by framing the problem as a nonlinear optimisation problem. We implement these methods in R to obtain general functions to compute HDRs for classes of distributions, and for commonly used families of distributions. We compare our method to existing R packages for computing HDRs and we show that our method performs favourably in terms of both accuracy and average speed.

Appendix: Proof of Theorems

In this appendix we give proofs of the lemmas and theorems shown in the main body of the paper. For ease of reference, we repeat these lemmas and theorems here.

Lemma 1

If ${\mathcal{H}}$ is a highest density region then for any set ${\mathcal{A}}$ we have:

$$ \begin{gathered} \left| {\mathcal{A}} \right| < \left| {\mathcal{H}} \right| \Rightarrow {\mathbb{P}}\left( {X \in {\mathcal{A}}} \right) < {\mathbb{P}}\left( {X \in {\mathcal{H}}} \right){\text{,}} \hfill \\ \left| {\mathcal{A}} \right| \le \left| {\mathcal{H}} \right| \Rightarrow {\mathbb{P}}\left( {X \in {\mathcal{A}}} \right) \le {\mathbb{P}}\left( {X \in {\mathcal{H}}} \right){\text{.}} \hfill \\ \end{gathered} $$

Proof of Lemma 1

: For any set ${\mathcal{A}}$ we have:

$$ \begin{aligned} {\mathbb{P}}\left( {X \in {\mathcal{H}}} \right) - {\mathbb{P}}\left( {X \in {\mathcal{A}}} \right) &= \mathop \smallint \limits_{{\mathcal{H}}} f\left( x \right)dx - \mathop \smallint \limits_{{\mathcal{A}}} f\left( x \right)dx\\&= \mathop \smallint \limits_{{{\mathcal{H}} - {\mathcal{A}}}} f\left( x \right)dx - \mathop \smallint \limits_{{{\mathcal{A}} - {\mathcal{H}}}} f\left( x \right)dx{.} \end{aligned}$$

If $\left| {\mathcal{A}} \right| < \left| {\mathcal{H}} \right|$ then we must have $\left| {{\mathcal{A}} - {\mathcal{H}}} \right| < \left| {{\mathcal{H}} - {\mathcal{A}}} \right|$, so the range of integration in the first integral is larger than in the second integral. Moreover, since ${\mathcal{H}}$ is a highest-density region it encompasses all points with $f\left( x \right) \ge f_{*}$, so we must have $f\left( x \right) < f_{*} \le f\left( y \right)$ for all $x \in {\mathcal{A}} - {\mathcal{H}}$ and $y \in {\mathcal{H}} - {\mathcal{A}}$. This means that the integrand in the first integral is strictly higher than in the second integral.²⁵ Since the first integral has a higher integrand and larger region of integration, this implies that it is larger than the second integral, so ${\mathbb{P}}\left( {X \in {\mathcal{H}}} \right) > {\mathbb{P}}\left( {X \in {\mathcal{A}}} \right)$. This gives us the first inequality result in the lemma; the second result follows analogously. □

Theorem 1

A highest density region ${\mathcal{H}}$ with actual coverage probability $1 - \alpha$ is a smallest closed covering region with minimal coverage probability $1 - \alpha$.

Proof of Theorem 1

Consider a HDR ${\mathcal{H}}$ with actual coverage probability $1 - \alpha$. To show that ${\mathcal{H}}$ is a smallest closed covering region with minimal coverage probability $1 - \alpha$, we have to show that there is no closed set ${\mathcal{A}}$ with $\left| {\mathcal{A}} \right| < \left| {\mathcal{H}} \right|$ and ${\mathbb{P}}\left( {X \in {\mathcal{A}}} \right) \ge 1 - \alpha$. This follows directly from the first inequality result in Lemma 1. □

Theorem 2

If the intensity function for $X$ is continuous then a highest density region ${\mathcal{H}}$ formed with minimal coverage probability $1 - \alpha$ has actual coverage probability $1 - \alpha$.

Proof of Theorem 2

Consider a HDR ${\mathcal{H}}$ with minimum coverage probability $1 - \alpha$. Since the intensity function $H$ is continuous, the random variable $X$ is also continuous, so we have:

$$ \begin{array}{*{20}c} {{\mathbb{P}}\left( {X \in {\mathcal{S}}} \right) = {\mathbb{P}}\left( {X \in {\text{closure}}{\mathcal{S}}} \right)} & {} & {{\text{for any set }}{\mathcal{S}}{.}} \\ \end{array} $$

Since the intensity function $H$ is continuous we have $H( {f_{*} } ) = 1 - \alpha$, it follows that:

$$ \begin{aligned}{\mathbb{P}}\left( {X \in {\mathcal{H}}} \right) &= {\mathbb{P}}\left( {X \in {\text{closure}}\left\{ {x \in {\mathbb{R}}{|}f\left( x \right) \ge f_{*} } \right\}} \right)\\&= {\mathbb{P}}\left( {X \in \left\{ {x \in {\mathbb{R}}{|}f\left( x \right) \ge f_{*} } \right\}} \right)\\&= {\mathbb{P}}\left( {f\left( X \right) \ge f_{*} } \right) \\ &= H( {f_{*} }) = 1 - \alpha {,} \end{aligned} $$

which establishes that the actual coverage probability is $1 - \alpha$. □

Table 1 Accuracy of HDRs computed by different R packages