normal-inverse-gamma Probability density function ![Probability density function of normal-inverse-gamma distribution for α = 1.0, 2.0 and 4.0, plotted in shifted and scaled coordinates.](//upload.wikimedia.org/wikipedia/commons/thumb/4/48/Normal-inverse-gamma.svg/565px-Normal-inverse-gamma.svg.png) |
Parameters | location (real)
(real)
(real)
(real) |
---|
Support | ![{\displaystyle x\in (-\infty ,\infty )\,\!,\;\sigma ^{2}\in (0,\infty )}](https://wikimedia.org/api/rest_v1/media/math/render/svg/9752767a3db2a43c037cfefbe059a0409c416c18) |
---|
PDF | ![{\displaystyle {\frac {\sqrt {\lambda }}{\sqrt {2\pi \sigma ^{2}}}}{\frac {\beta ^{\alpha }}{\Gamma (\alpha )}}\left({\frac {1}{\sigma ^{2}}}\right)^{\alpha +1}\exp \left(-{\frac {2\beta +\lambda (x-\mu )^{2}}{2\sigma ^{2}}}\right)}](https://wikimedia.org/api/rest_v1/media/math/render/svg/d3d4040d8090127f083255889c6cfb8f3073cbc1) |
---|
Mean | ![{\displaystyle \operatorname {E} [x]=\mu }](https://wikimedia.org/api/rest_v1/media/math/render/svg/d60f5921cca1c75d673eb70db395bf3a88f9170f) , for ![{\displaystyle \alpha >1}](https://wikimedia.org/api/rest_v1/media/math/render/svg/17d81dbbc4786493c7b8548cc324a978d7cf5dbd)
|
---|
Mode | ![{\displaystyle \sigma ^{2}={\frac {\beta }{\alpha +1+1/2}}\;{\textrm {(univariate)}},\sigma ^{2}={\frac {\beta }{\alpha +1+k/2}}\;{\textrm {(multivariate)}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/a2d8570284c452dabd6b54e3081c45cad00113e6) |
---|
Variance | , for ![{\displaystyle \alpha >1}](https://wikimedia.org/api/rest_v1/media/math/render/svg/17d81dbbc4786493c7b8548cc324a978d7cf5dbd) , for ![{\displaystyle \alpha >2}](https://wikimedia.org/api/rest_v1/media/math/render/svg/432334d220d6e1b0340cc2a37531d0327494a8e2) , for ![{\displaystyle \alpha >1}](https://wikimedia.org/api/rest_v1/media/math/render/svg/17d81dbbc4786493c7b8548cc324a978d7cf5dbd) |
---|
In probability theory and statistics, the normal-inverse-gamma distribution (or Gaussian-inverse-gamma distribution) is a four-parameter family of multivariate continuous probability distributions. It is the conjugate prior of a normal distribution with unknown mean and variance.
Definition
Suppose
![{\displaystyle x\mid \sigma ^{2},\mu ,\lambda \sim \mathrm {N} (\mu ,\sigma ^{2}/\lambda )\,\!}](https://wikimedia.org/api/rest_v1/media/math/render/svg/3b162c00ab95501eaa5456a88cc88d3d1bf1e0b0)
has a normal distribution with mean
and variance
, where
![{\displaystyle \sigma ^{2}\mid \alpha ,\beta \sim \Gamma ^{-1}(\alpha ,\beta )\!}](https://wikimedia.org/api/rest_v1/media/math/render/svg/af5b94a5e6f2c94a56701a998db475359603184a)
has an inverse-gamma distribution. Then
has a normal-inverse-gamma distribution, denoted as
![{\displaystyle (x,\sigma ^{2})\sim {\text{N-}}\Gamma ^{-1}(\mu ,\lambda ,\alpha ,\beta )\!.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/6054e26b057d59500fd046adfccd537653859fc7)
(
is also used instead of
)
The normal-inverse-Wishart distribution is a generalization of the normal-inverse-gamma distribution that is defined over multivariate random variables.
Characterization
Probability density function
![{\displaystyle f(x,\sigma ^{2}\mid \mu ,\lambda ,\alpha ,\beta )={\frac {\sqrt {\lambda }}{\sigma {\sqrt {2\pi }}}}\,{\frac {\beta ^{\alpha }}{\Gamma (\alpha )}}\,\left({\frac {1}{\sigma ^{2}}}\right)^{\alpha +1}\exp \left(-{\frac {2\beta +\lambda (x-\mu )^{2}}{2\sigma ^{2}}}\right)}](https://wikimedia.org/api/rest_v1/media/math/render/svg/9c6f2ff09b67f308da375fb0a3343d97b6775b6c)
For the multivariate form where
is a
random vector,
![{\displaystyle f(\mathbf {x} ,\sigma ^{2}\mid \mu ,\mathbf {V} ^{-1},\alpha ,\beta )=|\mathbf {V} |^{-1/2}{(2\pi )^{-k/2}}\,{\frac {\beta ^{\alpha }}{\Gamma (\alpha )}}\,\left({\frac {1}{\sigma ^{2}}}\right)^{\alpha +1+k/2}\exp \left(-{\frac {2\beta +(\mathbf {x} -{\boldsymbol {\mu }})^{T}\mathbf {V} ^{-1}(\mathbf {x} -{\boldsymbol {\mu }})}{2\sigma ^{2}}}\right).}](https://wikimedia.org/api/rest_v1/media/math/render/svg/e80c5ad03fdc717e7f02ecf13da886be056144d4)
where
is the determinant of the
matrix
. Note how this last equation reduces to the first form if
so that
are scalars.
Alternative parameterization
It is also possible to let
in which case the pdf becomes
![{\displaystyle f(x,\sigma ^{2}\mid \mu ,\gamma ,\alpha ,\beta )={\frac {1}{\sigma {\sqrt {2\pi \gamma }}}}\,{\frac {\beta ^{\alpha }}{\Gamma (\alpha )}}\,\left({\frac {1}{\sigma ^{2}}}\right)^{\alpha +1}\exp \left(-{\frac {2\gamma \beta +(x-\mu )^{2}}{2\gamma \sigma ^{2}}}\right)}](https://wikimedia.org/api/rest_v1/media/math/render/svg/0de2757f81e02769e20fd7a68df05ad5b19c8855)
In the multivariate form, the corresponding change would be to regard the covariance matrix
instead of its inverse
as a parameter.
Cumulative distribution function
![{\displaystyle F(x,\sigma ^{2}\mid \mu ,\lambda ,\alpha ,\beta )={\frac {e^{-{\frac {\beta }{\sigma ^{2}}}}\left({\frac {\beta }{\sigma ^{2}}}\right)^{\alpha }\left(\operatorname {erf} \left({\frac {{\sqrt {\lambda }}(x-\mu )}{{\sqrt {2}}\sigma }}\right)+1\right)}{2\sigma ^{2}\Gamma (\alpha )}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/f2ab3974a8acdcd68d24c21ba5e47c501a2865d8)
Properties
Marginal distributions
Given
as above,
by itself follows an inverse gamma distribution:
![{\displaystyle \sigma ^{2}\sim \Gamma ^{-1}(\alpha ,\beta )\!}](https://wikimedia.org/api/rest_v1/media/math/render/svg/025cafc6781cc617befdff8f81790e077c44def6)
while
follows a t distribution with
degrees of freedom.[1]
Proof for
For
probability density function is
Marginal distribution over
is
Except for normalization factor, expression under the integral coincides with Inverse-gamma distribution
with
,
,
.
Since
, and
Substituting this expression and factoring dependence on
,
Shape of generalized Student's t-distribution is
.
Marginal distribution
follows t-distribution with
degrees of freedom
.
In the multivariate case, the marginal distribution of
is a multivariate t distribution:
![{\displaystyle \mathbf {x} \sim t_{2\alpha }({\boldsymbol {\mu }},{\frac {\beta }{\alpha }}\mathbf {V} )\!}](https://wikimedia.org/api/rest_v1/media/math/render/svg/14295263bb612daab16a8469f43add6b2dbdc259)
Summation
Scaling
Suppose
![{\displaystyle (x,\sigma ^{2})\sim {\text{N-}}\Gamma ^{-1}(\mu ,\lambda ,\alpha ,\beta )\!.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/6054e26b057d59500fd046adfccd537653859fc7)
Then for
,
![{\displaystyle (cx,c\sigma ^{2})\sim {\text{N-}}\Gamma ^{-1}(c\mu ,\lambda /c,\alpha ,c\beta )\!.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/828dd1ea701d026836b77bbe8a5379b878d1f977)
Proof: To prove this let
and fix
. Defining
, observe that the PDF of the random variable
evaluated at
is given by
times the PDF of a
random variable evaluated at
. Hence the PDF of
evaluated at
is given by :
The right hand expression is the PDF for a
random variable evaluated at
, which completes the proof.
Exponential family
Normal-inverse-gamma distributions form an exponential family with natural parameters
,
,
, and
and sufficient statistics
,
,
, and
.
Kullback–Leibler divergence
Measures difference between two distributions.
Maximum likelihood estimation
Posterior distribution of the parameters
See the articles on normal-gamma distribution and conjugate prior.
Interpretation of the parameters
See the articles on normal-gamma distribution and conjugate prior.
Generating normal-inverse-gamma random variates
Generation of random variates is straightforward:
- Sample
from an inverse gamma distribution with parameters
and ![{\displaystyle \beta }](https://wikimedia.org/api/rest_v1/media/math/render/svg/7ed48a5e36207156fb792fa79d29925d2f7901e8)
- Sample
from a normal distribution with mean
and variance ![{\displaystyle \sigma ^{2}/\lambda }](https://wikimedia.org/api/rest_v1/media/math/render/svg/833ade0b9c62ddf0e9eb8ad07f605c1f77e09acc)
- The normal-gamma distribution is the same distribution parameterized by precision rather than variance
- A generalization of this distribution which allows for a multivariate mean and a completely unknown positive-definite covariance matrix
(whereas in the multivariate inverse-gamma distribution the covariance matrix is regarded as known up to the scale factor
) is the normal-inverse-Wishart distribution
See also
References
- ^ Ramírez-Hassan, Andrés. 4.2 Conjugate prior to exponential family | Introduction to Bayesian Econometrics.
- Denison, David G. T.; Holmes, Christopher C.; Mallick, Bani K.; Smith, Adrian F. M. (2002) Bayesian Methods for Nonlinear Classification and Regression, Wiley. ISBN 0471490369
- Koch, Karl-Rudolf (2007) Introduction to Bayesian Statistics (2nd Edition), Springer. ISBN 354072723X
Discrete univariate | with finite support | |
---|
with infinite support | |
---|
|
---|
Continuous univariate | supported on a bounded interval | |
---|
supported on a semi-infinite interval | |
---|
supported on the whole real line | |
---|
with support whose type varies | |
---|
|
---|
Mixed univariate | |
---|
Multivariate (joint) | |
---|
Directional | |
---|
Degenerate and singular | |
---|
Families | |
---|
Category Commons |