7-3 GMM Application: Classification

[english][all]
(請注意：中文版本並未隨英文版本同步更新！)
若要將 GMM 用於資料分類或是樣式辨認，其流程如下：

訓練時：對於每一個類別的資料，訓練一個 GMM。
測試時：將某一筆資料送到每一個類別的 GMM，機率最大者，即代表此資料屬於此類別的可能性較大。

以下這個範例，將使用 GMM 於 IRIS 資料的分類：
Example 1: gmmcIris01.m

在上述範例中，我們將原先的資料集分成 DS 及 TS，並使用 DS 做為訓練資料、TS 做為測試資料，每一個類別用了具有兩個高斯機率密度函數的 GMM 來代表，所得到的辨識率都是 100%（內部測試）以及 96%（外部測試）。
Since the above procedure of training a GMM classifier and evaluating the classifier is used quite often, we can simplify the procedure by using the following two functions:

gmmcTrain.m: For training a GMM classifier.
gmmcEval.m: For evaluating a GMM classifier.
In particular, gmmcTrain.m will genearte a set of parameters for GMM classifier (including the prior information), and gmmcEval.m can take the parameters and evaluate the classifier based on a given new set of data. The use of these two functions is shown in the next example.
Example 2: gmmcIris02.m

If we want to visualize the decision boundary imposed by a GMM classifier, we can try the following example which apply GMMC to a nonlinear separable dataset:
Example 3: gmmcNonlinearSeparable01.m

In the above example, we set the number of Gaussians to be 3 since we already know the data distribution and 3 is a good choice since class 1 is segmented into 3 disjoint regions. In practice, we are usually dealing with high-dimensional data and such information does not come by easily. As a result, we usually have to resort to trials and errors, as explained later.
The next example demonstrates the PDFs of GMMs associated with these two classes. Due to the combination of several Gaussian PDFs, we are able to model more complicated PDFs of this problem.
Example 4: gmmcNonlinearSeparable02.m

Hint
If we set the number of Gaussians to 2 in the above two examples, what would you expect to see? Guess it before you try it. Then try it, you might be amazed at how versatile & flexible the GMMC is!

如前所述，GMM 的效能和高斯機率密度函數的個數有密切的關係，因此我們可以使用下列範例，來畫出辨識率對高斯機率密度函數之個數的關係，如下：
Example 5: gmmcIris03.m

從上述範例所產生的圖形，我們可以觀測到一般分類系統的共同特性：

當分類器的可調參數越來越多時，訓練資料的辨識率會隨之水漲船高，但測試資料的辨識率則會先爬高，然後再降低。
一般分類器的最佳組態，就是取用在測試資料的辨識率最高時，所對應的結構和參數。

Since it is a common practice to observe the training and test recognition rates with respective to the number of Gaussians, we have create a function gmmcGaussianNumEstimate.m to accomplish the task. In the following example, we use the wine dataset for such task:
Example 6: gmmcWine02.m

In order to perform more analysis, we plot the range of all features:
Example 7: rangePlotWine.m

Obviously the last feature has a much wider range than the others. We can perform input normalization before GMM training, as follows:
Example 8: gmmcWine03.m

The above plot demonstrates that input normalization can sometimes lead to a better accuracy.
Data Clustering and Pattern Recognition (資料分群與樣式辨認)