Draft:Sub-exponential distribution

• Comment: This article would need more references to establish notability - mention in text or reference books for example. It also needs to establish why this is more notable than
  the sub-exponential distribution in heavy-tailed distribution mentioned at the top. It needs an indication of which statements in the article are supported by the list of references in the References section. Newystats (talk) 02:48, 21 March 2023 (UTC)

---

This article generalize the space of random variables generated by exponential Orlicz function, which in turn can be regarded as generalization of $L^{p}$ space for random variables. The sub-exponential distribution discussed here is heavily related to sub-Gaussian distribution. Do not be confused with the sub-exponential distribution in heavy-tailed distribution.

In probability theory, a sub-exponential distribution is a probability distribution with exponential tail decay. Informally, the tails of a sub-exponential distribution decay at a rate similar to those of the tails of a exponential random variable. This property gives sub-exponential distributions their name.

Formally, the probability distribution of a random variable $X$ is called sub-exponential if there are positive constant C such that for every $t\geq 0$,

${\textstyle \operatorname {P} (|X|\geq t)\leq 2\exp {(-t/C)}}$.

The sub-exponential distribution is heavily related to sub-Gaussian distribution. In fact, the square of a sub-exponential is sub-Gaussian , which has an even stronger tail decay.

Sub-Exponential properties

Let $X$ be a random variable. The following conditions are equivalent:

1. $\operatorname {P} (|X|\geq t)\leq 2\exp {(-t/K_{1})}$ for all $t\geq 0$, where $K_{1}$ is a positive constant;
2. $\operatorname {E} [\exp {(X/K_{2})}]\leq 2$, where $K_{2}$ is a positive constant;
3. $\operatorname {E} |X|^{p}\leq 2K_{3}^{p}\Gamma \left(p+1\right)$ for all $p\geq 1$, where $K_{3}$ is a positive constant.

Proof. $(1)\implies (3)$ By the layer cake representation,

$${\begin{aligned}\operatorname {E} |X|^{p}&=\int _{0}^{\infty }\operatorname {P} (|X|^{p}\geq t)dt\\&=\int _{0}^{\infty }pt^{p-1}\operatorname {P} (|X|\geq t)dt\\&\leq 2\int _{0}^{\infty }pt^{p-1}\exp \left(-{\frac {t}{K_{1}}}\right)dt\\\end{aligned}}$$

After a change of variables $u=t/K_{1}$, we find that

$${\begin{aligned}\operatorname {E} |X|^{p}&\leq 2K_{1}^{p}p\int _{0}^{\infty }u^{p-1}e^{-u}du\\&=2K_{1}^{p}p\Gamma \left(p\right)\\&=2K_{1}^{p}\Gamma \left(p+1\right).\end{aligned}}$$

$(3)\implies (2)$ Using the Taylor series for $e^{x}$:

$$e^{x}=1+\sum _{p=1}^{\infty }{\frac {x^{p}}{p!}},$$

and monotone convergence theorem, we obtain that

$${\begin{aligned}\operatorname {E} [\exp {(\lambda X)}]&=1+\sum _{p=1}^{\infty }{\frac {\lambda ^{p}\operatorname {E} {[X^{p}]}}{p!}}\\&\leq 1+\sum _{p=1}^{\infty }{\frac {2\lambda ^{p}K_{3}^{p}\Gamma \left(p+1\right)}{p!}}\\&=1+2\sum _{p=1}^{\infty }\lambda ^{p}K_{3}^{p}\\&=2\sum _{p=0}^{\infty }\lambda ^{p}K_{3}^{p}-1\\&={\frac {2}{1-\lambda K_{3}}}-1\quad {\text{for }}\lambda K_{3}<1,\end{aligned}}$$

which is less than or equal to $2$ for $\lambda \leq {\frac {1}{3K_{3}}}$. Take $K_{2}\geq 3K_{3}$, then

$$\operatorname {E} [\exp {(X/K_{2})}]\leq 2.$$

$(2)\implies (1)$ By Markov's inequality,

$$\operatorname {P} (|X|\geq t)=\operatorname {P} \left(\exp \left({\frac {X}{K_{2}}}\right)\geq \exp \left({\frac {t}{K_{2}}}\right)\right)\leq {\frac {\operatorname {E} [\exp {(X/K_{2})}]}{\exp \left({\frac {t}{K_{2}}}\right)}}\leq 2\exp \left(-{\frac {t}{K_{2}}}\right).$$

Definitions

A random variable $X$ is called a sub-exponential random variable if either one of the equivalent conditions above holds.

The sub-exponential norm of $X$, denoted as $||X||_{exp}$, is defined by

$$||X||_{exp}=\inf \left\{c>0:\operatorname {E} \left[\exp {\left({\frac {X}{c}}\right)}\right]\leq 2\right\},$$

which is the Orlicz norm of $X$ generated by the Orlicz function $\Phi (u)=e^{u}-1.$ By condition $(2)$ above, sub-exponential random variables can be characterized as those random variables with finite sub-exponential norm.

Relation with sub-Gaussian distributions

A random variable $X$ is sub-Gaussian if and only if $X^{2}$is sub-exponential. Moreover, $||X^{2}||_{exp}=||X||_{gauss}^{2}$.

Proof. This follows easily from the characterization of the random variables by the sub-exponential norm and sub-Gaussian norm. Indeed, by definition,

$${\begin{aligned}||X^{2}||_{exp}&=\inf \left\{c>0:\operatorname {E} \left[\exp {\left({\frac {X^{2}}{c}}\right)}\right]\leq 2\right\},\\||X||_{gauss}&=\inf \left\{d>0:\operatorname {E} \left[\exp {\left({\frac {X^{2}}{d^{2}}}\right)}\right]\leq 2\right\}.\end{aligned}}$$

Hence, we find that $||X^{2}||_{exp}=||X||_{gauss}^{2}$. Therefore, one of the norm is finite if and only if another one is. This shows that $X$ is sub-Gaussian if and only if $X^{2}$is sub-exponential.

More equivalent definitions

The following properties are equivalent:

• The distribution of $X$ is sub-exponential.
• Laplace transform condition: for some $B,b>0$, $\operatorname {E} e^{\lambda (X-\operatorname {E} [X])}\leq Be^{\lambda b}$ holds for all $\lambda$.
• Moment condition: for some $K>0$, $\operatorname {E} |X|^{p}\leq K^{p}p^{p}$ for all $p\geq 1$.
• Moment generating function condition: for some $L>0$, $\operatorname {E} [\exp(\lambda |X|)]\leq \exp(L\lambda )$ for all $\lambda$ such that $0\leq \lambda \leq {\frac {1}{L}}$.

Example

If $X$ has exponential distribution with rate $\lambda >0$, i.e. $X\sim Exp(\lambda )$, then

$$\operatorname {P} (X\geq t)=e^{-\lambda t}.$$

Then $X$ is sub-exponential since it satisfy condition $(1)$ with $K_{1}={\frac {1}{\lambda }}$.

See also

• Sub-Gaussian distribution
• Orlicz space

References

• Kahane, J.P. (1960). "Propriétés locales des fonctions à séries de Fourier aléatoires". Studia Mathematica. 19: 1–25. doi:10.4064/sm-19-1-1-25.
• Ledoux, Michel; Talagrand, Michel (1991). Probability in Banach Spaces. Springer-Verlag.
• Stromberg, K.R. (1994). Probability for Analysts. Chapman & Hall/CRC.
• Litvak, A.E.; Pajor, A.; Rudelson, M.; Tomczak-Jaegermann, N. (2005). "Smallest singular value of random matrices and geometry of random polytopes" (PDF). Advances in Mathematics. 195 (2): 491–523. doi:10.1016/j.aim.2004.08.004.
• Rudelson, Mark; Vershynin, Roman (2010). "Non-asymptotic theory of random matrices: extreme singular values". Proceedings of the International Congress of Mathematicians 2010. pp. 1576–1602. arXiv:1003.2990. doi:10.1142/9789814324359_0111.
• Vershynin, R. (2018). "High-dimensional probability: An introduction with applications in data science" (PDF). Volume 47 of Cambridge Series in Statistical and Probabilistic Mathematics. pp. 32-36. Cambridge University Press, Cambridge.
• Zajkowskim, K. (2020). "On norms in some class of exponential type Orlicz spaces of random variables". Positivity. An International Mathematics Journal Devoted to Theory and Applications of Positivity. 24(5): 1231--1240. arXiv:1709.02970. doi.org/10.1007/s11117-019-00729-6.