10-2 線性迴歸??面??

@uhJBXvuʰjkƾǼҫig

$$ y = f(\mathbf{x}) = \theta_1f_1(\mathbf{x}) + \theta_2f_2(\mathbf{x}) + \cdots + \theta_nf_n(\mathbf{x}) $$

䤤 $\mathbf{x}$ J]׬ $m$ Vq^Ay X]¶q^A$\theta_1$B$\theta_2$B$\cdots$B$\theta_n$ iܪѼơA$f_i(\mathbf{x}), i=1$ to $n$ hOwơA٬򩳨ơ]Basis Functions^C]ҵI $(\mathbf{x}_i, y_i), i=1 \cdots m$AoǸI٬˸ơ]Sample Data^ΰVmơ]Training Data^ANoǸIaJҫioG $$ \left\{ \begin{matrix} y_1 & = & f(\mathbf{x}_1) & = & \theta_1f_1(\mathbf{x}_1) + \theta_2f_2(\mathbf{x}_1) + \cdots + \theta_nf_n(\mathbf{x}_1) \\ \vdots & = & \vdots & = & \vdots \\ y_m & = & f(\mathbf{x}_m) & = & \theta_1f_1(\mathbf{x}_m) + \theta_2f_2(\mathbf{x}_m) + \cdots + \theta_nf_n(\mathbf{x}_m) \\ \end{matrix} \right. $$

Υiܦx}榡G

$$ \underbrace{ \left[ \begin{matrix} f_1(\mathbf{x}_1) & \cdots & f_n(\mathbf{x}_1) \\ f_1(\mathbf{x}_2) & \cdots & f_n(\mathbf{x}_2) \\ \vdots & \vdots & \vdots\\ f_1(\mathbf{x}_m) & \cdots & f_n(\mathbf{x}_m) \\ \end{matrix} \right] }_\mathbf{A} \underbrace{ \left[ \begin{matrix} \theta_1\\ \vdots\\ \theta_n\\ \end{matrix} \right] }_\mathbf{\theta} = \underbrace{ \left[ \begin{matrix} y_1\\ y_2\\ \vdots\\ y_m\\ \end{matrix} \right] }_\mathbf{y} $$

ѩb@뱡pUA$m>n$]YIӼƻjiܰѼƭӼơ^A]WLTѡAϤWߡA[W@~tVq $\mathbf{e}$G $$ \mathbf{A}\mathbf{\theta}=\mathbf{y}+\mathbf{e} $$ ~thig

$$ E(\mathbf{\theta})=\|\mathbf{e}\|^2=\mathbf{e}^T\mathbf{e}= (\mathbf{A}\mathbf{\theta}-\mathbf{y})^T (\mathbf{A}\mathbf{\theta}-\mathbf{y}) $$

$E(\mathbf{\theta})$ $\mathbf{\theta}$ LAåO䵥sAYio@ $n$ @upߤ{AYϥίx}BӪܡA$\mathbf{\theta}$ ̨έȥiHܦ $$ \hat{\mathbf{\theta}} = (\mathbf{A}^T\mathbf{A})^{-1}\mathbf{A}^T\mathbf{y} $$ ]WɡAi̫@p`C^

b@WAڭ̥iHϥ MATLAB uvӺX $\mathbf{\theta}$ ̨έȡAY $\hat{\mathbf{\theta}} = \mathbf{A}$\$\mathbf{y}$C

Hint
zפWĄΪ $\mathbf{\theta}$ Ȭ $(\mathbf{A}^T\mathbf{A})^{-1}\mathbf{A}^T\mathbf{y}$AO $\mathbf{A}^T\mathbf{A}$ p|ͥ趵A]bϯx}eyqƭȻ~tAMATLAB ڦbpuvɡA|̷ӯx} $\mathbf{A}$ Sʦӿγ̨ΪkA]iHoíwBTƭȸѡC

HUH peaks ƬҡAӻ@몺uʰjkCYb MATLAB UJ peaksAiHeX@ӥWYPApUG

ƪ{pUG

$$ z = 3(1-x)^2 e^{-x^2-(y+1)^2}-10\left(\frac{x}{5}-x^3-y^5\right) e^{-x^2-y^2}- \frac{1}{3} e^{-(x+1)^2-y^2} $$

bUCAڭ̰]G

]WzƥigG

$$ \begin{array}{rcl} z & = & 3(1-x)^2 e^{-x^2-(y+1)^2}-10\left(\frac{x}{5}-x^3-y^5\right) e^{-x^2-y^2}- \frac{1}{3} e^{-(x+1)^2-y^2} + noise\\ & = & 3 f_1(x, y) - 10 f_2(x, y) - \frac{1}{3} f_3(x, y) + noise\\ & = & \theta_1 f_1(x, y) + \theta_2 f_2(x, y) + \theta_3 f_3(x, y) + noise\\ \end{array} $$

䤤ڭ̰] $\theta_1$B$\theta_2$ M $\theta_3$ OѼơA$noise$ hOsBܲ 1 WGTCҦpGpGno 100 VmơAiϥΤUCdҡG

Example 1: 10-uXPjkR/peaks01.mpointNum = 10; [xx, yy, zz] = peaks(pointNum); zz = zz + randn(size(zz)); % [JT surf(xx, yy, zz); axis tight

bWҤArandn OϥΧYb[JWGTCWϬڭ̦쪺VmơAѩTܤjAҥHMaTϧήtܤjC{bڭ̭nΤw򩳨ơAӧX̨Ϊ $\theta_1$B$\theta_2$ M $\theta_3$AdҦpUG

Example 2: 10-uXPjkR/peaks02.mpointNum = 10; [xx, yy, zz] = peaks(pointNum); zz = zz + randn(size(zz))/10; % [JT x = xx(:); % ରVq y = yy(:); % ରVq z = zz(:); % ରVq A = [(1-x).^2.*exp(-(x.^2)-(y+1).^2), (x/5-x.^3-y.^5).*exp(-x.^2-y.^2), exp(-(x+1).^2-y.^2)]; theta = A\z % ̨Ϊ theta theta = 3.0392 -10.0407 -0.4093

ѦX $\mathbf{\theta}$ ȩM̨έ $\left(3, -10, -\frac{1}{3} \right)$ ۷CھڦѼơAڭ̥iHJKIAojk᪺AШUCdҡG

Example 3: 10-uXPjkR/peaks03.mpointNum = 10; [xx, yy, zz] = peaks(pointNum); zz = zz + randn(size(zz))/10; % [JT x = xx(:); y = yy(:); z = zz(:); % ରVq A = [(1-x).^2.*exp(-(x.^2)-(y+1).^2), (x/5-x.^3-y.^5).*exp(-x.^2-y.^2), exp(-(x+1).^2-y.^2)]; theta = A\z; % ̨Ϊ theta % eXw pointNum = 31; [xx, yy] = meshgrid(linspace(-3, 3, pointNum), linspace(-3, 3, pointNum)); x = xx(:); y = yy(:); % ରVq A = [(1-x).^2.*exp(-(x.^2)-(y+1).^2), (x/5-x.^3-y.^5).*exp(-x.^2-y.^2), exp(-(x+1).^2-y.^2)]; zz = reshape(A*theta, pointNum, pointNum); surf(xx, yy, zz); axis tight

bWϤAijk᪺M۷C̥Dn]OGڭ̲qF򩳨ơ]άO󥿽TAڭ̰ݤFT򩳨ơ^A]oD`nXC@ӨAYT򩳨ƦӭJÿΡA 3 ӥiܨƹF 100 ӸI}nXC

bWҤڭ̴bI[JWG]Normal Distributed^TCƹWAun򩳨ƥTAӥBTOWGAIVӶVhAWz̤pkNiHGѼƪuƭȡC


MATLAB{]pGig