7-2 GMM Application: PDF Modeling

[chinese][english]
(請注意：中文版本並未隨英文版本同步更新！)
In the previous section, we have covered the mathematics of EM (expectation maximization) which, under the framework of MLE, can be employed to identify the optimum parameters for a GMM. In this section, we shall demonstrate the use of GMM for PDF modeling.
以下將就 GMM 的使用範例進行說明。
For the first example, we shall use GMM for modeling the probability density function of a 1D data, as follows.
在下列範例中，我們將 GMM 用於一維的資料：
Example 1: gmm1d01.m

In the previous example, the data is generated via three Gaussian PDFs centered at -2, 0, 2 (please refer to the contents of dcData.m). The first plot is the histogram of the dataset; the second plot is the curve of log probability w.r.t. the number of iterations. From the above example, we have the following observations:
在上述範例中，原來的資料是由中心點為 -2, 0, 2 的三個高斯機率密度函數所產生（請見 dcData.m 的內容），第一個圖是資料的質方圖（Histogram），第二個圖是 log probability 在訓練過程中的變化。由上述範例可以看出：

The identified centers are very close to the means of the three Gaussian PDF.
Log probability is monotonically nondecreasing throughout the training iterations.

所計算出來的中心點和原先資料的理想中心點很接近。
對數機率在訓練過程中，會一路爬升或持平，不會下降。

We can use the following example to plot the PDF after training:
若要看到訓練後的機率密度函數，可以使用下列範例：
Example 2: gmm1d02.m

From the above example, the identified GMM PDF can match the data histogram closely. This is based on the following three conditions:
在上述範例中，可以很明顯的看出來，資料的質方圖和 GMM 所算出來的機率密度函數，非常接近。這是由於下列三個條件都成立：

The size of data is large enough. (The above example has 600 data entries.)
We are able to guess the number of Gaussians correctly.
The data is indeed governed by GMM.

資料量夠大。（上述範例有 600 筆資料！）
事先能夠猜對高斯機率密度函數的個數。
資料的確是由高斯機率密度函數所產生。

In practice, the above three conditions do not always hold. The basic remedies are:

Try to collect as much data as possible.
Use some heuristic search to find the optimum number of Gaussian PDFs.
Increase the number of mixtures so we can approximate any PDF using the training data.

但是在實際世界中，這三種條件未必都成立，因此我們的對應方案是：

資料蒐集是越多越好。
執行 GMM 訓練多次，以便找到高斯機率密度函數的最佳個數。
可用事後驗證，看看資料分佈是否符合我們的假設。

In the following example, we should use GMM to model the 2D donut dataset, as follows:
下面這個範例，我們將 GMM 用於二維的「甜甜圈」資料，如下：
Example 3: gmm2d01.m

In the above example, you should be able to see the flashy animation during the training process. Moreover, since we have set gmmPrm.useKmeans=0, the training process will randomly select several data points as the initial centers instead of using k-means for determining a set of better centers. Since the initial centers are randomly selected, the program will need more time to adjust these 6 Gaussians.
在執行上述範例時，可以看到動畫展示，非常有趣，請各位讀者自己試試看！
Not every dataset modeled by GMM will generate satisfactory result. An example follows.
GMM 的訓練，並不是每次都會產生理想的結果，請見下列範例：
Example 4: gmm2d02.m

Judging from the scatter plot of the data set, we should have three Gaussians to cover the three clusters. The first two at the upper left corner should be sharper while the third one at the center should be flatter. In practice, it is likely to have the situation with "big circle surrounds small one", indicating the training process was trapped in a local maximum. (Since the data is randomly generated, you should try the program several times to obtain several possible results.)
在上述範例中，由資料分佈來看，理想中應該有三個高斯機率密度函數，分別負責三組資料的產生，其中在左上角的兩組很密集，而在中間的一組則比較分散。但實際進行訓練時，常常會發生「大圓包小圓」的情況，代表 GMM 的訓練，掉入了局部最大點而無法跳出。（由於資料是亂數產生，所以每次訓練的結果都不相同。如果讀者們跑出的結果和此範例不同，請多試幾次，就可以看到這種「大圓包小圓」的情況。）
Here is another example of GMM modeling (using 4 Gaussians) over 2D data:
以下是另一個範例，使用GMM（包含4個高斯PDF）來對2D的資料進行建模：
Example 5: gmm2d03.m

Data Clustering and Pattern Recognition (資料分群與樣式辨認)