## 5-5 Naive Bayes Classifiers (單純貝氏分類器)

[english][all]

(請注意：中文版本並未隨英文版本同步更新！)

p(X|C) = P(X1|C)P(X2|C) ... P(Xd|C)

1. 假設每一個類別的資料均是由 d 維的高斯機率密度函數（Gaussian probability density function）所產生：：
gi(x, m, S) = (2p)-d/2*det(S)-0.5*exp[-(x-m)TS-1(x-m)/2]
其中 m 是此高斯機率密度函數的平均向量（Mean vector），S 則是其共變異矩陣（Covariance matrix），我們可以根據 MLE，產生最佳的平均向量 m 和共變異矩陣 S
2. 若有需要，可以對每一個高斯機率密度函數乘上一個權重 wi
3. 在實際進行分類時，wi*gi(x, m, S) 越大，則資料 x 隸屬於類別 i 的可能性就越高。

log[p(ci)g(x, mi, Si)] = log(p(ci)) - (d*log(2p) + log|Si|)/2 - (x-mi)TSi-1(x-mi)/2
The decision boundary between class i and j is represented by the following trajectory:
p(ci)g(x, mi, Si) = p(cj)g(x, mj, Sj).
Taking the logrithm of both sides, we have
log(p(ci)) - (d*log(2p) + log|Si|)/2 - (x-mi)TSi-1(x-mi)/2 = log(p(cj)) - (d*log(2p) + log|S|j)/2 - (x-mj)TSj-1(x-mj)/2
After simplification, we have the decision boundary as the following equation:
(x-mj)TSj-1(x-mj) - (x-mi)TSi-1(x-mi) = log{[|S|i p2(ci)]/[|S|j p2(cj)]}
where the right-hand side is a constant. Since both (x-mj)TSj-1(x-mj) and (x-mi)TSi-1(x-mi) are quadratic, the above equation represents a decision boundary of the quadratic form in the d-dimensional feature space.

Example 1: nbc01dataPlot.mDS = prData('iris'); DS.input=DS.input(3:4, :); % Only take dimensions 3 and 4 for 2d visualization plotOpt=1; % Plotting [nbcPrm, logLike, recogRate, hitIndex]=nbcTrain(DS, [], plotOpt); fprintf('Recog. rate = %f%%\n', recogRate*100); Recog. rate = 96.000000%

• 如果要滿足貝氏分類的原理（請見後續章節），此權重可以設定是每一個類別的資料個數。（計算每個類別的資料個數，可由 dsClassSize.m 來達成。）
• 如果每個類別的資料出線的機率相差不大，我們可將每一個類別的權重都設定成 1。

Example 2: nbcPlot00.mDS=prData('iris'); DS.input=DS.input(3:4, :); [nbcPrm, logLike, recogRate, hitIndex]=nbcTrain(DS); nbcPlot(DS, nbcPrm, '1dPdf');

Example 3: nbcPlot01.mDS=prData('iris'); DS.input=DS.input(3:4, :); [nbcPrm, logLike, recogRate, hitIndex]=nbcTrain(DS); nbcPlot(DS, nbcPrm, '2dPdf');

Example 4: nbcPlot02.mDS=prData('iris'); DS.input=DS.input(3:4, :); [nbcPrm, logLike, recogRate, hitIndex]=nbcTrain(DS); DS.hitIndex=hitIndex; % Attach hitIndex to DS for plotting nbcPlot(DS, nbcPrm, 'decBoundary');

Data Clustering and Pattern Recognition (資料分群與樣式辨認)