In probability theory and statistics, two real-valued random variables,
,
, are said to be uncorrelated if their covariance,
, is zero. If two variables are uncorrelated, there is no linear relationship between them.
Uncorrelated random variables have a Pearson correlation coefficient, when it exists, of zero, except in the trivial case when either variable has zero variance (is a constant). In this case the correlation is undefined.
In general, uncorrelatedness is not the same as orthogonality, except in the special case where at least one of the two random variables has an expected value of 0. In this case, the covariance is the expectation of the product, and
and
are uncorrelated if and only if
.
If
and
are independent, with finite second moments, then they are uncorrelated. However, not all uncorrelated variables are independent.[1]: p. 155
Definition
Definition for two real random variables
Two random variables
are called uncorrelated if their covariance
is zero.[1]: p. 153 [2]: p. 121 Formally:
Definition for two complex random variables
Two complex random variables
are called uncorrelated if their covariance
and their pseudo-covariance
is zero, i.e.
Definition for more than two random variables
A set of two or more random variables
is called uncorrelated if each pair of them is uncorrelated. This is equivalent to the requirement that the non-diagonal elements of the autocovariance matrix
of the random vector
are all zero. The autocovariance matrix is defined as:
![{\displaystyle \operatorname {K} _{\mathbf {X} \mathbf {X} }=\operatorname {cov} [\mathbf {X} ,\mathbf {X} ]=\operatorname {E} [(\mathbf {X} -\operatorname {E} [\mathbf {X} ])(\mathbf {X} -\operatorname {E} [\mathbf {X} ])^{\rm {T}}]=\operatorname {E} [\mathbf {X} \mathbf {X} ^{T}]-\operatorname {E} [\mathbf {X} ]\operatorname {E} [\mathbf {X} ]^{T}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/f2920d62cbc5b8edeea646abcf040e83aae2baac)
Examples of dependence without correlation
Example 1
- Let
be a random variable that takes the value 0 with probability 1/2, and takes the value 1 with probability 1/2. - Let
be a random variable, independent of
, that takes the value −1 with probability 1/2, and takes the value 1 with probability 1/2. - Let
be a random variable constructed as
.
The claim is that
and
have zero covariance (and thus are uncorrelated), but are not independent.
Proof:
Taking into account that
![{\displaystyle \operatorname {E} [U]=\operatorname {E} [XY]=\operatorname {E} [X]\operatorname {E} [Y]=\operatorname {E} [X]\cdot 0=0,}](https://wikimedia.org/api/rest_v1/media/math/render/svg/bd4da767a2ab5ae8e62c4e76439b4a30a4ba624a)
where the second equality holds because
and
are independent, one gets
![{\displaystyle {\begin{aligned}\operatorname {cov} [U,X]&=\operatorname {E} [(U-\operatorname {E} [U])(X-\operatorname {E} [X])]=\operatorname {E} [U(X-{\tfrac {1}{2}})]\\&=\operatorname {E} [X^{2}Y-{\tfrac {1}{2}}XY]=\operatorname {E} [(X^{2}-{\tfrac {1}{2}}X)Y]=\operatorname {E} [(X^{2}-{\tfrac {1}{2}}X)]\operatorname {E} [Y]=0\end{aligned}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/a6d8e5b797ac2f8316c71571d441696d0bfbe8f8)
Therefore,
and
are uncorrelated.
Independence of
and
means that for all
and
,
. This is not true, in particular, for
and
.
![{\displaystyle \Pr(U=1\mid X=0)=\Pr(XY=1\mid X=0)=0}](https://wikimedia.org/api/rest_v1/media/math/render/svg/e6629af22dd27eaa22f130a77d24e6a2c35feac4)
![{\displaystyle \Pr(U=1)=\Pr(XY=1)=1/4}](https://wikimedia.org/api/rest_v1/media/math/render/svg/c0314fc956d299fa201d2ca63dbec892209349ff)
Thus
so
and
are not independent.
Q.E.D.
Example 2
If
is a continuous random variable uniformly distributed on
and
, then
and
are uncorrelated even though
determines
and a particular value of
can be produced by only one or two values of
:
on the other hand,
is 0 on the triangle defined by
although
is not null on this domain. Therefore
and the variables are not independent.
Therefore the variables are uncorrelated.
There are cases in which uncorrelatedness does imply independence. One of these cases is the one in which both random variables are two-valued (so each can be linearly transformed to have a Bernoulli distribution).[3] Further, two jointly normally distributed random variables are independent if they are uncorrelated,[4] although this does not hold for variables whose marginal distributions are normal and uncorrelated but whose joint distribution is not joint normal (see Normally distributed and uncorrelated does not imply independent).
Generalizations
Two random vectors
and
are called uncorrelated if
.
They are uncorrelated if and only if their cross-covariance matrix
is zero.[5]: p.337
Two complex random vectors
and
are called uncorrelated if their cross-covariance matrix and their pseudo-cross-covariance matrix is zero, i.e. if
![{\displaystyle \operatorname {K} _{\mathbf {Z} \mathbf {W} }=\operatorname {J} _{\mathbf {Z} \mathbf {W} }=0}](https://wikimedia.org/api/rest_v1/media/math/render/svg/a2d70907b7fe279a5603a4ddd097ec01863023fd)
where
![{\displaystyle \operatorname {K} _{\mathbf {Z} \mathbf {W} }=\operatorname {E} [(\mathbf {Z} -\operatorname {E} [\mathbf {Z} ]){(\mathbf {W} -\operatorname {E} [\mathbf {W} ])}^{\mathrm {H} }]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/686b575db50d74be88fa21e2778665659fcbc712)
and
.
Two stochastic processes
and
are called uncorrelated if their cross-covariance
is zero for all times.[2]: p. 142 Formally:
.
See also
References
- ^ a b Papoulis, Athanasios (1991). Probability, Random Variables and Stochastic Processes. MCGraw Hill. ISBN 0-07-048477-5.
- ^ a b Kun Il Park, Fundamentals of Probability and Stochastic Processes with Applications to Communications, Springer, 2018, 978-3-319-68074-3
- ^ Virtual Laboratories in Probability and Statistics: Covariance and Correlation, item 17.
- ^ Bain, Lee; Engelhardt, Max (1992). "Chapter 5.5 Conditional Expectation". Introduction to Probability and Mathematical Statistics (2nd ed.). pp. 185–186. ISBN 0534929303.
- ^ Gubner, John A. (2006). Probability and Random Processes for Electrical and Computer Engineers. Cambridge University Press. ISBN 978-0-521-86470-1.
Further reading
- Probability for Statisticians, Galen R. Shorack, Springer (c2000) ISBN 0-387-98953-6