logo

Various Properties of Covariance 📂Mathematical Statistics

Various Properties of Covariance

Definitions and Properties

The covariance of probability variables XX and YY, whose means are μX\mu_{X} and μY\mu_{Y} respectively, is defined as Cov⁡(X,Y):=E[(X−μX)(Y−μY)]\operatorname{Cov} (X ,Y) : = E \left[ ( X - \mu_{X} ) ( Y - \mu_{Y} ) \right]. Covariance has the following properties:

  • [1]: Var⁡(X)=Cov⁡(X,X)\Var (X) = \operatorname{Cov} (X,X)
  • [2]: Cov⁡(X,Y)=Cov⁡(Y,X)\operatorname{Cov} (X,Y) = \operatorname{Cov} (Y, X)
  • [3]: Var⁡(X+Y)=Var⁡(X)+Var⁡(Y)+2Cov⁡(X,Y)\Var (X + Y) = \Var (X) + \Var (Y) + 2 \operatorname{Cov} (X,Y)
  • [4]: Cov⁡(X+Y,Z)=Cov⁡(X,Z)+Cov⁡(Y,Z)\operatorname{Cov} (X + Y , Z ) = \operatorname{Cov}(X,Z) + \operatorname{Cov}(Y,Z)
  • [5]: Cov⁡(aX+b,cY+d)=acCov⁡(X,Y)\operatorname{Cov} (aX + b , cY + d ) = ac \operatorname{Cov}(X,Y)

Explanation

Covariance indicates the linear correlation between two variables and, unlike variance, can also be negative as well as 00.

Proof

[1]

Cov⁡(X,X)=E[(X−μX)(X−μX)]=E[(X−μX)2]=Var⁡(X) \begin{align*} \operatorname{Cov} (X ,X) =& E[ ( X - \mu_{X} ) ( X - \mu_{X} ) ] \\ =& E[ ( X - \mu_{X} )^2 ] \\ =& \Var (X) \end{align*}

■

[2]

Cov⁡(X,Y)=E[(X−μX)(Y−μY)]=E[(Y−μY)(X−μX)]=Cov⁡(X,Y) \begin{align*} \operatorname{Cov} (X ,Y) =& E[ ( X - \mu_{X} ) ( Y - \mu_{Y} ) ] \\ =& E[ ( Y - \mu_{Y} ) ( X - \mu_{X} ) ] \\ =& \operatorname{Cov} (X ,Y) \end{align*}

■

[3]

Var⁡(X+Y)=E[(X+Y−μX−μY)2]=E[{(X−μX)+(Y−μY)}2]=E[(X−μX)2+2(X−μX)(Y−μY)+(Y−μY)2]=E[(X−μX)2]+2E[(X−μX)(Y−μY)]+E[(Y−μY)2]=Var⁡(X)+2Cov⁡(X,Y)+Var⁡(Y) \begin{align*} \Var (X + Y) =& E [ ( X + Y - \mu_{X} - \mu_{Y} )^2 ] \\ =& E \left[ \left\{ ( X - \mu_{X} ) + (Y - \mu_{Y} ) \right\} ^2 \right] \\ =& E \left[ ( X - \mu_{X} )^2 + 2 ( X - \mu_{X} ) (Y - \mu_{Y} )+ (Y - \mu_{Y} )^2 \right] \\ =& E[ ( X - \mu_{X} )^2] + 2 E [ ( X - \mu_{X} ) (Y - \mu_{Y} ) ] + E [ (Y - \mu_{Y} )^2 ] \\ =& \Var (X) + 2 \operatorname{Cov} (X,Y) + \Var (Y) \end{align*}

■

[4]

Cov⁡(X+Y,Z)=E[(X+Y−μX−μY)(Z−μZ)]=E[{(X−μX)+(Y−μY)}(Z−μZ)]=E[(X−μX)(Z−μZ)]+E[(Y−μY)(Z−μZ)]=Cov⁡(X,Z)+Cov⁡(Y,Z) \begin{align*} \operatorname{Cov} (X + Y , Z ) =& E \left[ ( X + Y - \mu_{X} - \mu_{Y} ) ( Z - \mu_{Z} ) \right] \\ =& E \left[ \left\{ ( X - \mu_{X} ) + ( Y - \mu_{Y} ) \right\} ( Z - \mu_{Z} ) \right] \\ =& E \left[ ( X - \mu_{X} ) ( Z - \mu_{Z} ) \right] + E \left[ ( Y - \mu_{Y} ) ( Z - \mu_{Z} ) \right] \\ =& \operatorname{Cov}(X,Z) + \operatorname{Cov}(Y,Z) \end{align*}

■

[5]

Cov⁡(aX+b,cY+d)=E[(aX+b−aμX−b)(cY+d−cμY−d)]=E[(aX−aμX)(cY−cμY)]=E[ac(X−μX)(Y−μY)]=acE[(X−μX)(Y−μY)]=acCov⁡(X,Y) \begin{align*} \operatorname{Cov} (aX + b , cY + d ) =& E \left[ ( aX + b - a \mu_{X} - b ) ( cY + d - c \mu_{Y} - d ) \right] \\ =& E \left[ ( aX - a \mu_{X} ) ( cY - c \mu_{Y} ) \right] \\ =& E \left[ a c ( X - \mu_{X} ) ( Y - \mu_{Y} ) \right] \\ =& ac E \left[( X - \mu_{X} ) ( Y - \mu_{Y} ) \right] \\ =& ac \operatorname{Cov}(X,Y) \end{align*}

■

See Also