Bingo, Computer Graphics & Game Developer

Mathematical Foundations of Monte Carlo Methods 2

本文为在Scratchapixel上学习的翻译读后感与部分个人解读。这里不会将全篇的内容系数翻译，保留原文以便后期自行理解，笔者只精炼一些文章中关键的点出来便于记录。

Variance and Standard Deviation

方差与标准差(Variance and Standard Deviation):Standard deviation is simply the square root of variance, and variance is defined as the expected value of the square difference between the outcome of the experiment.

Var(X)=σ2=E[(X−E[X])2]=∑i(xi−E[X])2pi.Var(X) = \sigma^2 = E[(X - E[X])^2] = \sum_i (x_i - E[X])^2p_i.

Standard Deviation=σ2.\text{Standard Deviation} = \sqrt{\sigma^2}.

如若注意到的话，可以看到方差的符号标示上方有平方，这是为了避免潜在的符号干扰，本质上声明了方差和标准差是不可能为负数的。

由于期望的可加性性质(上一篇笔记中有推导过程)，令μ=E[X]\mu = E[X]，若随机变量XX为常数，那么E[c]=cE[c] = c。

E[X−E[X]2]=E[(X−μ)2]=E[X2−2μX+μ2]=E[X2]−2μE[X]+E[μ2]=E[X2]−2μ2+μ2=E[X2]−μ2=∑ixi2pi−μ2=∑ixi2pi−(∑ixipi)2
\begin{array}{l}
E[X - E[X]^2] & = & E[(X - \mu)^2] \\
& = & E[X^2 - 2 \mu X + \mu^2] \\
& = & E[X^2] - 2 \mu E[X] + E[\mu^2] \\
& = & E[X^2] - 2\mu^2 + \mu^2 \\
& = & E[X^2] - \mu^2 \\
& = & \sum_i x_i^2 p_i - \mu^2 \\
& = & \sum_i x_i^2 p_i - (\sum_i x_i p_i)^2
\end{array}

若当前随机变量表示的是一个等概率随机事件，那么方差可以直接根据其样本均值Xˉ=E[X]\bar{X} = E[X]构建计算

Var(X)=∑i(xi−E[X])2pi=∑i=1n(xi−Xˉ)2n
\begin{array}{l}
Var(X) & = & \sum_i(x_i - E[X])^2 p_i\\
& = & \sum_{i=1}^n \frac{(x_i - \bar{X})^2}{n}
\end{array}

Properies of Variance

  1. 若Pr(X=c)=1Pr(X = c) = 1，那么其方差Var(X)=∑ixi2pi−μ2=c2∗1−c2=0Var(X) = \sum_i x_i^2p_i - \mu^2 = c^2*1 - c^2 = 0。换言之就是，一个必然事件的方差为0。

  2. 若有事件Y=aX+bY = aX + b, 那么其方差

    Var(Y)=E[(Y−E[Y])2]=E[(aX+b−E[aX+b])2]=E[(aX+b−aE[X]−b)2]=a2E[(X−E[X])2]=a2Var(X)
    \begin{array}{l}
    Var(Y) & = & E[(Y - E[Y])^2] \\
    & = & E[(aX + b - E[aX + b])^2] \\
    & = & E[(aX + b - aE[X] - b)^2] \\
    & = & a^2 E[(X - E[X])^2] \\
    & = & a^2Var(X)
    \end{array}

  3. 若X1,...,XnX_1, …, X_n为独立随机变量，那么其方差Var(X1+...+Xn)=Var(X1)+...+Var(Xn).Var(X_1 + … + X_n) = Var(X_1) + … + Var(X_n).

这里只推导两个随机变量之间的相加，多项式可递推。令μ1=E[X1],μ2=E[X2]\mu_1 = E[X_1], \mu_2 = E[X_2]

，而E[X1+X2]=E[X1]+E[X2]=μ1+μ2E[X_1 + X_2] = E[X_1] + E[X_2] = \mu_1 + \mu_2，因此:

Var(X1+X2)=E[(X1+X2−E[X1+X2])2]=E[(X1+X2−μ1−μ2)2]=E[(X1−μ1)2]+E[(X2−μ2)2]−E[2(X1−μ1)(X2−μ2)]=E[(X1−μ1)2]+E[(X2−μ2)2]−2(E[(X1−μ1)]∗E[X2−μ2])=E[(X1−μ1)2]+E[(X2−μ2)2]−2((μ1−μ1)(μ2−μ2))=E[(X1−μ1)2]+E[(X2−μ2)2]=Var(X1)+Var(X2)
\begin{array}{l}
Var(X_1 + X_2) & = & E[(X_1 + X_2 - E[X_1 + X_2])^2] \\
& = & E[(X_1 + X_2 - \mu_1 - \mu_2)^2] \\
& = & E[(X_1 - \mu_1)^2] + E[(X_2 - \mu_2)^2] - E[2(X_1 - \mu_1)(X_2 - \mu_2)] \\
& = & E[(X_1 - \mu_1)^2] + E[(X_2 - \mu_2)^2] - 2(E[(X_1 - \mu_1)]*E[X_2 - \mu_2]) \\
& = & E[(X_1 - \mu_1)^2] + E[(X_2 - \mu_2)^2] - 2((\mu_1 - \mu_1)(\mu_2 - \mu_2)) \\
& = & E[(X_1 - \mu_1)^2] + E[(X_2 - \mu_2)^2] \\
& = & Var(X_1) + Var(X_2)
\end{array}

Probability Distribution: Part 2

正态分布(Normal Distribution):p(x)=N(μ,σ)=1σ2πe−(x−μ)22σ2.p(x) = \mathcal{N}(\mu, \sigma) = {\dfrac{1}{\sigma \sqrt {2 \pi} } } e^{-{\dfrac{(x -\mu)^2}{2\sigma^2}}}.

其中μ\mu代表正态分布的期望，σ\sigma代表正态分布的标准差，整个曲线根据μ\mu对称。





见图所示

Sampling Distribution

样本分布(Sample Distribution):Each sample on its own, is a random variable, but because now they represent the mean of certain number n of items in the population, we label them with the upper letter XX. We can repeat this experiment NN times which gives as series of samples: X1,X2,...XNX_1,X_2,…X_N. This collection of samples is what we call a sampling distribution.

样本均值的期望(Expected value of the distribution of mean):We can apply to samples or statistics the same method for computing a mean than the method we used to calculate the mean of random variables.





注意到样本分布和普通的集群分布的区别，样本分布中，假定每个样本对集群取三次观察值，由于观察值本身是随机的缘故，因此观察值就是一个随机变量xx。那么这样的一个样本分布的样本大小为3，所以样本均值X1ˉ=E[x]=∑i=1nxin\bar{X_1} = E[x] = \frac{\sum^n_{i=1}x_i}{n}，样本方差Var(X1ˉ)=∑i=1n(xi−X1ˉ)Var(\bar{X_1}) = \sum^n_{i=1}(x_i - \bar{X_1})。

上文说到的样本均值的期望的计算，也就是将最基本的观察值事件求取均值作为随机变量的期望计算，是讲样本这个群作为一个随机变量XX，那么重复这样在总群中做采样，可以得到一系列X1,X2,...XNX_1, X_2, … X_N，此时样本均值的期望μXˉ=E[Xˉ]=∑i=1NXiˉN\mu_{\bar{X}} = E[\bar{X}] = \frac{\sum^N_{i=1}\bar{X_i}}{N}，样本均值的方差Var(X1ˉ)=∑i−1N(Xiˉ−μXˉ)2NVar(\bar{X_1}) = \frac{\sum^N_{i-1}(\bar{X_i} - \mu_{\bar{X}})^2}{N}。

所以务必要明确原文当中Expected value of the distribution of mean的含义才可得以进一步的计算。


中心极限定理(Central Limit Theorem, CLT): The mean of the sampling distribution of the mean μXˉ\mu_{\bar{X}} equals the mean of the population μ\mu and that the standard error of the distribution of means μXˉ\mu_{\bar{X}} is equal to the standard deviation of the population σ \sigma divided by the square root of nn. In addition, the sampling distribution of the mean will approach a normal distribution N(μ,σn)N(\mu, {\frac{\sigma}{\sqrt{n}}}). These relationships may be summarized as follows:

μXˉ=μσXˉ=σn\mu_{\bar{X}} = \mu_{ \sigma \bar{X}}=\frac{ \sigma }{\sqrt{n}}

Properties of the Sample Mean

  1. Xˉ=1n(X1+...+Xn)\bar X = \dfrac{1}{n} (X_1 + … + X_n)

  2. 样本均值等于总体平均值

    E[Xˉn]=1n∑i=1nE[Xi]=1n⋅nμ=μ.E[\bar X_n] = \dfrac{1}{n} \sum_{i=1}^n E[X_i] = \dfrac{1}{n} \cdot { n \mu } = \mu.

  3. 遵循与基本事件xx一样的性质(样本均值的期望本身就是随机变量xx的期望所计算而来的均值，2已经证明样本期望就是总体期望本身)

    E[aX+b]=aE[X]+bE[X1+...+Xn]=E[X1]+...+E[Xn]\begin{array}{l}
    E[aX+b] = aE[X] + b\\
    E[X_1 + … + X_n] = E[X_1] + … + E[X_n]
    \end{array}

  4. 也是样本期望的定义

    E[Xˉ]=E[1n(X1+...+Xn)]=1nE[X1+...+Xn]=1n∑i=1NE[Xi].\begin{array}{l}
    E[\bar X]&=&E[\dfrac{1}{n}(X_1 + … + X_n)]\\
    &=&\dfrac{1}{n}E[X_1 + … + X_n]\\
    &=&\dfrac{1}{n} \sum_{i=1}^N E[X_i].
    \end{array}

  5. 样本方差

    Var(Xˉn)=1n2Var(∑i=1nXi)=1n2∑i=1nVar(Xi)=1n2⋅nσ2=σ2n.\begin{array}{l}
    Var(\bar X_n)&=&\dfrac{1}{n^2} Var \left( \sum_{i=1}^n X_i \right) \\
    &=&\dfrac{1}{n^2} \sum_{i=1}^n Var(X_i) = \dfrac{1}{n^2} \cdot n \sigma^2 = \dfrac{\sigma^2}{n}.
    \end{array}

6.正如之前方差的定义中讲述的以下性质样本方差也都继承

Var(aX+b)=a2Var(X)Var(X1+...+Xn)=Var(X1)+...+Var(Xn).\begin{array}{l}
Var(aX + b) = a^2Var(X)\\Var(X_1+…+X_n) = Var(X_1) + … + Var(X_n).
\end{array}

因此样本方差σ2\sigma^2为:

Var(Xˉ)=Var(1n(X1+...Xn))=1n2Var(X1+...Xn)=1n2∑i=1nVar(Xi).\begin{array}{l}
Var(\bar X)&=&Var(\dfrac{1}{n}(X_1 + … X_n))\\
&=&\dfrac{1}{n^2 } Var(X_1 + … X_n)\\
&=&\dfrac{1}{n^2 } \sum_{i=1}^n Var(X_i).
\end{array}

7.因为样本方差为σ2n\frac{\sigma^2}{n}比总体方差σ2\sigma^2要更小的关系(换言之样本标准差σn\frac{\sigma}{\sqrt{n}})，样本均值Xˉ\bar{X}会比单一观察量XiX_i所计算得到的期望μ\mu更接近