다변량정규분포의 조건부 평균과 분산
📂확률분포론 다변량정규분포의 조건부 평균과 분산 공식 이변량정규분포 ( X , Y ) ∼ N 2 ( [ μ 1 μ n ] , [ σ X 2 ρ σ X σ Y ρ σ X σ Y σ Y 2 ] )
\left( X, Y \right) \sim N_{2} \left( \begin{bmatrix} \mu_{1} \\ \mu_{n} \end{bmatrix} , \begin{bmatrix} \sigma_{X}^{2} & \rho \sigma_{X} \sigma_{Y} \\ \rho \sigma_{X} \sigma_{Y} & \sigma_{Y}^{2} \end{bmatrix} \right)
( X , Y ) ∼ N 2 ( [ μ 1 μ n ] , [ σ X 2 ρ σ X σ Y ρ σ X σ Y σ Y 2 ] )
랜덤벡터 ( X , Y ) \left( X,Y \right) ( X , Y ) 이변량정규분포 를 따른다고 할 때, X ∣ Y X | Y X ∣ Y 일변량정규분포 를 따르며 조건부 평균과 분산 은 다음과 같다.
E ( X ∣ Y ) = μ X + ρ σ X σ Y ( Y − μ Y ) Var ( X ∣ Y ) = ( 1 − ρ 2 ) σ X 2
\begin{align*}
E \left( X | Y \right) =& \mu_{X} + \rho {{ \sigma_{X} } \over { \sigma_{Y} }} \left( Y - \mu_{Y} \right)
\\ \Var \left( X | Y \right) =& \left( 1 - \rho^{2} \right) \sigma_{X}^{2}
\end{align*}
E ( X ∣ Y ) = Var ( X ∣ Y ) = μ X + ρ σ Y σ X ( Y − μ Y ) ( 1 − ρ 2 ) σ X 2
다변량정규분포 X = [ X 1 X 2 ] : Ω → R n μ = [ μ 1 μ 2 ] ∈ R n Σ = [ Σ 11 Σ 12 Σ 21 Σ 22 ] ∈ R n × n
\begin{align*}
\mathbf{X} =& \begin{bmatrix} \mathbf{X}_{1} \\ \mathbf{X}_{2} \end{bmatrix} & : \Omega \to \mathbb{R}^{n}
\\ \mu =& \begin{bmatrix} \mu_{1} \\ \mu_{2} \end{bmatrix} & \in \mathbb{R}^{n}
\\ \Sigma =& \begin{bmatrix} \Sigma_{11} & \Sigma_{12} \\ \Sigma_{21} & \Sigma_{22} \end{bmatrix} & \in \mathbb{R}^{n \times n}
\end{align*}
X = μ = Σ = [ X 1 X 2 ] [ μ 1 μ 2 ] [ Σ 11 Σ 21 Σ 12 Σ 22 ] : Ω → R n ∈ R n ∈ R n × n 조던블럭폼 으로 나타낸 X \mathbf{X} X μ \mu μ Σ \Sigma Σ 다변량정규분포 를 따르는 랜덤벡터 X ∼ N n ( μ , Σ ) \mathbf{X} \sim N_{n} \left( \mu , \Sigma \right) X ∼ N n ( μ , Σ ) 조건부확률벡터 X 1 ∣ X 2 : Ω → R m \mathbf{X}_{1} | \mathbf{X}_{2} : \Omega \to \mathbb{R}^{m} X 1 ∣ X 2 : Ω → R m 모평균 벡터와 모공분산행렬 을 가진다.
X 1 ∣ X 2 ∼ N m ( μ 1 + Σ 12 Σ 22 − 1 ( X 2 − μ 2 ) , Σ 11 − Σ 12 Σ 22 − 1 Σ 21 )
\mathbf{X}_{1} | \mathbf{X}_{2} \sim N_{m} \left( \mu_{1} + \Sigma_{12} \Sigma_{22}^{-1} \left( \mathbf{X}_{2} - \mu_{2} \right) , \Sigma_{11} - \Sigma_{12} \Sigma_{22}^{-1} \Sigma_{21} \right)
X 1 ∣ X 2 ∼ N m ( μ 1 + Σ 12 Σ 22 − 1 ( X 2 − μ 2 ) , Σ 11 − Σ 12 Σ 22 − 1 Σ 21 )
유도 쉽고 복잡하고 특수한 유도 행렬대수 에 익숙하지 않은 어린이들은 애초에 이변량정규분포 정도도 버거울 것이다. 다음의 유도과정은 고등학교만 졸업해도 따라할 수 있는 수준으로 쉽지만, 전개 자체는 복잡하며 이변량정규분포의 평균분산에 그친다.
f ( x , y ) = 1 2 π σ X σ Y 1 − ρ 2 exp [ − 1 2 ( 1 − ρ 2 ) [ ( x − μ X σ X ) 2 + ( y − μ Y σ Y ) 2 − 2 ρ ( x − μ X ) ( y − μ Y ) σ X σ Y ] ]
f(x,y) = {{ 1 } \over { 2 \pi \sigma_{X} \sigma_{Y} \sqrt{1 - \rho^{2}} }} \exp \left[ - {{ 1 } \over { 2 \left( 1 - \rho^{2} \right) }} \left[ \left( {{ x - \mu_{X} } \over { \sigma_{X} }} \right)^{2} + \left( {{ y - \mu_{Y} } \over { \sigma_{Y} }} \right)^{2}- 2 \rho {{ \left( x - \mu_{X} \right) \left( y - \mu_{Y} \right) } \over { \sigma_{X} \sigma_{Y} }} \right] \right]
f ( x , y ) = 2 π σ X σ Y 1 − ρ 2 1 exp [ − 2 ( 1 − ρ 2 ) 1 [ ( σ X x − μ X ) 2 + ( σ Y y − μ Y ) 2 − 2 ρ σ X σ Y ( x − μ X ) ( y − μ Y ) ] ]
f ( x ∣ y ) = f ( x , y ) f Y ( y ) = 1 2 π σ X σ Y 1 − ρ 2 e [ − 1 2 ( 1 − ρ 2 ) [ ( x − μ X σ X ) 2 + ( y − μ Y σ Y ) 2 − 2 ρ ( x − μ X ) ( y − μ Y ) σ X σ Y ] ] 1 2 π σ Y e [ − 1 2 ( y − μ Y σ Y ) 2 ] = 1 2 π σ X 1 − ρ 2 e [ − 1 2 ( 1 − ρ 2 ) [ ( x − μ X σ X ) 2 + ( y − μ Y σ Y ) 2 − 2 ρ ( x − μ X ) ( y − μ Y ) σ X σ Y ] ] e [ − 1 − ρ 2 2 ( 1 − ρ 2 ) ( y − μ Y σ Y ) 2 ] = 1 2 π σ X 1 − ρ 2 exp [ − 1 2 ( 1 − ρ 2 ) [ ( x − μ X σ X ) 2 + ρ 2 ( y − μ Y σ Y ) 2 − 2 ρ ( x − μ X ) ( y − μ Y ) σ X σ Y ] ] = 1 2 π σ X 1 − ρ 2 exp [ − 1 2 ( 1 − ρ 2 ) [ ( x − μ X σ X ) − ρ ( y − μ Y σ Y ) ] 2 ] = 1 2 π σ X 1 − ρ 2 exp [ − 1 2 σ X 2 ( 1 − ρ 2 ) [ x − μ X − ρ σ X ( y − μ Y σ Y ) ] 2 ] = 1 2 π σ X 1 − ρ 2 exp [ − 1 2 σ X 2 ( 1 − ρ 2 ) [ x − μ X − ρ σ X σ Y ( y − μ Y ) ] 2 ]
\begin{align*}
& f \left( x | y \right)
\\ =& {{ f \left( x , y \right) } \over { f_{Y}(y) }}
\\ =& {{ {{ 1 } \over { 2 \pi \sigma_{X} \sigma_{Y} \sqrt{1 - \rho^{2}} }} e^{\left[ - {{ 1 } \over { 2 \left( 1 - \rho^{2} \right) }} \left[ \left( {{ x - \mu_{X} } \over { \sigma_{X} }} \right)^{2} + \left( {{ y - \mu_{Y} } \over { \sigma_{Y} }} \right)^{2}- 2 \rho {{ \left( x - \mu_{X} \right) \left( y - \mu_{Y} \right) } \over { \sigma_{X} \sigma_{Y} }} \right] \right]} } \over { {{ 1 } \over { \sqrt{2 \pi} \sigma_{Y} }} e^{ \left[ - {{ 1 } \over { 2 }} \left( {{ y - \mu_{Y} } \over { \sigma_{Y} }} \right)^{2} \right] } }}
\\ =& {{ 1 } \over { \sqrt{2 \pi} \sigma_{X} \sqrt{1 - \rho^{2}} }} {{ e^{\left[ - {{ 1 } \over { 2 \left( 1 - \rho^{2} \right) }} \left[ \left( {{ x - \mu_{X} } \over { \sigma_{X} }} \right)^{2} + \left( {{ y - \mu_{Y} } \over { \sigma_{Y} }} \right)^{2}- 2 \rho {{ \left( x - \mu_{X} \right) \left( y - \mu_{Y} \right) } \over { \sigma_{X} \sigma_{Y} }} \right] \right]} } \over { e^{\left[ - {{ 1 - \rho^{2} } \over { 2 \left( 1 - \rho^{2} \right) }} \left( {{ y - \mu_{Y} } \over { \sigma_{Y} }} \right)^{2} \right]} }}
\\ =& {{ 1 } \over { \sqrt{2 \pi} \sigma_{X} \sqrt{1 - \rho^{2}} }} \exp \left[ - {{ 1 } \over { 2 \left( 1 - \rho^{2} \right) }} \left[ \left( {{ x - \mu_{X} } \over { \sigma_{X} }} \right)^{2} + \rho^{2} \left( {{ y - \mu_{Y} } \over { \sigma_{Y} }} \right)^{2}- 2 \rho {{ \left( x - \mu_{X} \right) \left( y - \mu_{Y} \right) } \over { \sigma_{X} \sigma_{Y} }} \right] \right]
\\ =& {{ 1 } \over { \sqrt{2 \pi} \sigma_{X} \sqrt{1 - \rho^{2}} }} \exp \left[ - {{ 1 } \over { 2 \left( 1 - \rho^{2} \right) }} \left[ \left( {{ x - \mu_{X} } \over { \sigma_{X} }} \right) - \rho \left( {{ y - \mu_{Y} } \over { \sigma_{Y} }} \right) \right]^{2} \right]
\\ =& {{ 1 } \over { \sqrt{2 \pi} \sigma_{X} \sqrt{1 - \rho^{2}} }} \exp \left[ - {{ 1 } \over { 2 \sigma_{X}^{2} \left( 1 - \rho^{2} \right) }} \left[ x - \mu_{X} - \rho \sigma_{X} \left( {{ y - \mu_{Y} } \over { \sigma_{Y} }} \right) \right]^{2} \right]
\\ =& {{ 1 } \over { \sqrt{2 \pi} \sigma_{X} \sqrt{1 - \rho^{2}} }} \exp \left[ - {{ 1 } \over { 2 \sigma_{X}^{2} \left( 1 - \rho^{2} \right) }} \left[ x - \mu_{X} - \rho {{ \sigma_{X} } \over { \sigma_{Y} }} \left( y - \mu_{Y} \right) \right]^{2} \right]
\end{align*}
= = = = = = = f ( x ∣ y ) f Y ( y ) f ( x , y ) 2 π σ Y 1 e [ − 2 1 ( σ Y y − μ Y ) 2 ] 2 π σ X σ Y 1 − ρ 2 1 e [ − 2 ( 1 − ρ 2 ) 1 [ ( σ X x − μ X ) 2 + ( σ Y y − μ Y ) 2 − 2 ρ σ X σ Y ( x − μ X ) ( y − μ Y ) ] ] 2 π σ X 1 − ρ 2 1 e [ − 2 ( 1 − ρ 2 ) 1 − ρ 2 ( σ Y y − μ Y ) 2 ] e [ − 2 ( 1 − ρ 2 ) 1 [ ( σ X x − μ X ) 2 + ( σ Y y − μ Y ) 2 − 2 ρ σ X σ Y ( x − μ X ) ( y − μ Y ) ] ] 2 π σ X 1 − ρ 2 1 exp [ − 2 ( 1 − ρ 2 ) 1 [ ( σ X x − μ X ) 2 + ρ 2 ( σ Y y − μ Y ) 2 − 2 ρ σ X σ Y ( x − μ X ) ( y − μ Y ) ] ] 2 π σ X 1 − ρ 2 1 exp [ − 2 ( 1 − ρ 2 ) 1 [ ( σ X x − μ X ) − ρ ( σ Y y − μ Y ) ] 2 ] 2 π σ X 1 − ρ 2 1 exp [ − 2 σ X 2 ( 1 − ρ 2 ) 1 [ x − μ X − ρ σ X ( σ Y y − μ Y ) ] 2 ] 2 π σ X 1 − ρ 2 1 exp [ − 2 σ X 2 ( 1 − ρ 2 ) 1 [ x − μ X − ρ σ Y σ X ( y − μ Y ) ] 2 ]
이는 모평균이 μ X + ρ σ X σ Y ( Y − μ Y ) \mu_{X} + \rho {{ \sigma_{X} } \over { \sigma_{Y} }} \left( Y - \mu_{Y} \right) μ X + ρ σ Y σ X ( Y − μ Y ) ( 1 − ρ 2 ) σ X 2 \left( 1 - \rho^{2} \right) \sigma_{X}^{2} ( 1 − ρ 2 ) σ X 2 일변량정규분포 의 확률밀도함수 와 같다.
어렵고 간단하고 일반적인 유도 W : = X 1 − Σ 12 Σ 22 − 1 X 2 \mathbf{W} := \mathbf{X}_{1} - \Sigma_{12} \Sigma_{22}^{-1} \mathbf{X}_{2} W := X 1 − Σ 12 Σ 22 − 1 X 2 p : = ( m − n ) ∈ N p := (m - n) \in \mathbb{N} p := ( m − n ) ∈ N 항등행렬 I k ∈ R k × k I_{k} \in \mathbb{R}^{k \times k} I k ∈ R k × k 영행렬 O ∈ R p × m O \in \mathbb{R}^{p \times m} O ∈ R p × m [ W X 2 ] = [ I m − Σ 12 Σ 22 − 1 O I p ] [ X 1 X 2 ]
\begin{bmatrix} \mathbf{W} \\ \mathbf{X}_{2} \end{bmatrix} = \begin{bmatrix}
I_{m} & - \Sigma_{12} \Sigma_{22}^{-1}
\\ O & I_{p}
\end{bmatrix} \begin{bmatrix} \mathbf{X}_{1} \\ \mathbf{X}_{2} \end{bmatrix}
[ W X 2 ] = [ I m O − Σ 12 Σ 22 − 1 I p ] [ X 1 X 2 ]
선형변환의 정규성 : 행렬 A ∈ R m × n A \in \mathbb{R}^{m \times n} A ∈ R m × n 벡터 b ∈ R m \mathbf{b} \in \mathbb{R}^{m} b ∈ R m 다변량정규분포 를 따르는 랜덤벡터 X ∼ N n ( μ , Σ ) \mathbf{X} \sim N_{n} \left( \mu , \Sigma \right) X ∼ N n ( μ , Σ ) 선형변환 Y = A X + b \mathbf{Y} = A \mathbf{X} + \mathbf{b} Y = A X + b N m ( A μ + b , A Σ A T ) N_{m} \left( A \mu + \mathbf{b} , A \Sigma A^{T} \right) N m ( A μ + b , A Σ A T )
W \mathbf{W} W E W = μ 1 − Σ 12 Σ 22 − 1 μ 2
E \mathbf{W} = \mu_{1} - \Sigma_{12} \Sigma_{22}^{-1} \mu_{2}
E W = μ 1 − Σ 12 Σ 22 − 1 μ 2 A = Σ 12 Σ 22 − 1 A = \Sigma_{12} \Sigma_{22}^{-1} A = Σ 12 Σ 22 − 1 공분산행렬 을 가진다.
Cov X = A Σ A T = [ I m − Σ 12 Σ 22 − 1 O I p ] [ Σ 11 Σ 12 Σ 21 Σ 22 ] [ I m O T − Σ 22 − 1 Σ 21 I p ] = [ Σ 11 − Σ 12 Σ 22 − 1 Σ 21 O T O Σ 22 ] ⟹ Cov W = Σ 11 − Σ 12 Σ 22 − 1 Σ 21
\begin{align*}
\operatorname{Cov} \mathbf{X} =& A \Sigma A^{T}
\\ =& \begin{bmatrix}
I_{m} & - \Sigma_{12} \Sigma_{22}^{-1}
\\ O & I_{p}
\end{bmatrix} \begin{bmatrix}
\Sigma_{11} & \Sigma_{12}
\\ \Sigma_{21} & \Sigma_{22}
\end{bmatrix} \begin{bmatrix}
I_{m} & O^{T}
\\ - \Sigma_{22}^{-1} \Sigma_{21} & I_{p}
\end{bmatrix}
\\ =& \begin{bmatrix}
\Sigma_{11} - \Sigma_{12} \Sigma_{22}^{-1} \Sigma_{21} & O^{T}
\\ O & \Sigma_{22}
\end{bmatrix}
\\ \implies \operatorname{Cov} \mathbf{W} =& \Sigma_{11} - \Sigma_{12} \Sigma_{22}^{-1} \Sigma_{21}
\end{align*}
Cov X = = = ⟹ Cov W = A Σ A T [ I m O − Σ 12 Σ 22 − 1 I p ] [ Σ 11 Σ 21 Σ 12 Σ 22 ] [ I m − Σ 22 − 1 Σ 21 O T I p ] [ Σ 11 − Σ 12 Σ 22 − 1 Σ 21 O O T Σ 22 ] Σ 11 − Σ 12 Σ 22 − 1 Σ 21 A T A^{T} A T A A A 트랜스포즈 다.
독립과 제로 공분산의 동치 : 다변량 정규분포를 따르는 ( X 1 , X 2 ) ∼ N n ( μ , Σ ) \left( \mathbf{X}_{1} , \mathbf{X}_{2} \right) \sim N_{n} \left( \mu , \Sigma \right) ( X 1 , X 2 ) ∼ N n ( μ , Σ ) X 1 ⊥ X 2 ⟺ Σ 12 = Σ 21 = O
\mathbf{X}_{1} \perp \mathbf{X}_{2} \iff \Sigma_{12} = \Sigma_{21} = O
X 1 ⊥ X 2 ⟺ Σ 12 = Σ 21 = O
X \mathbf{X} X Cov ( W , X 2 ) = O \operatorname{Cov} \left( \mathbf{W} , \mathbf{X}_{2} \right) = O Cov ( W , X 2 ) = O W \mathbf{W} W X 2 \mathbf{X}_{2} X 2 W ∣ X 2 \mathbf{W} | \mathbf{X}_{2} W ∣ X 2 W \mathbf{W} W W ∣ X 2 ∼ N m ( μ 1 − Σ 12 Σ 22 − 1 μ 2 , Σ 11 − Σ 12 Σ 22 − 1 Σ 21 )
\mathbf{W} | \mathbf{X}_{2} \sim N_{m} \left( \mu_{1} - \Sigma_{12} \Sigma_{22}^{-1} \mu_{2} , \Sigma_{11} - \Sigma_{12} \Sigma_{22}^{-1} \Sigma_{21} \right)
W ∣ X 2 ∼ N m ( μ 1 − Σ 12 Σ 22 − 1 μ 2 , Σ 11 − Σ 12 Σ 22 − 1 Σ 21 ) W \mathbf{W} W X 1 = W + Σ 12 Σ 22 − 1 X 2 \mathbf{X}_{1} = \mathbf{W} + \Sigma_{12} \Sigma_{22}^{-1} \mathbf{X}_{2} X 1 = W + Σ 12 Σ 22 − 1 X 2 X 1 ∣ X 2 ∼ N m ( μ 1 + Σ 12 Σ 22 − 1 ( X 2 − μ 2 ) , Σ 11 − Σ 12 Σ 22 − 1 Σ 21 )
\mathbf{X}_{1} | \mathbf{X}_{2} \sim N_{m} \left( \mu_{1} + \Sigma_{12} \Sigma_{22}^{-1} \left( \mathbf{X}_{2} - \mu_{2} \right) , \Sigma_{11} - \Sigma_{12} \Sigma_{22}^{-1} \Sigma_{21} \right)
X 1 ∣ X 2 ∼ N m ( μ 1 + Σ 12 Σ 22 − 1 ( X 2 − μ 2 ) , Σ 11 − Σ 12 Σ 22 − 1 Σ 21 )
■
