吴恩达公开课-04

1. Motivations¶

对于non-linear classification，尽管可以在logistic regression中添加多项式项来解决，但当feature较多时，很难列举全所有的多项式项，因此需要non-linear hypothesis

"one learning algorithm"

Dendrite -> cell body -> axon

$a_i^{(j)}$ : activation of unit $i$ in layer $j$

$\theta^{(j)}$ : layer $j$ to $j+1$

$a_1^{(2)}=g(\theta_{10}^{(1)}x_0+\theta_{11}^{(1)}x_1+\theta_{12}^{(1)}x_2+\theta_{13}^{(1)}x_3)$

$a_2^{(2)}=g(\theta_{20}^{(1)}x_0+\theta_{21}^{(1)}x_1+\theta_{22}^{(1)}x_2+\theta_{23}^{(1)}x_3)$

$a_3^{(2)}=g(\theta_{30}^{(1)}x_0+\theta_{31}^{(1)}x_1+\theta_{32}^{(1)}x_2+\theta_{33}^{(1)}x_3)$

$h_\theta(x)=a_1^3=g(\theta_{10}^{(2)}a_0^{(2)}+\theta_{11}^{(2)}a_1^{(2)}+\theta_{12}^2a_{(2)}^{(2)}+\theta_{13}^{(2)}a_3^{(2)})$

$size(\theta^{(j)})=len(j+1)*[len(j)+1]$

添加 $x_0=a_0^2=1$ ，这里的 $a_0^2$ : bias unit

vectorize

$z^{(j+1)}=\theta^{(j)}a^{(j)}$$ $$a^{(j+1)}=g(\theta^{(j+1)})$

等于拿 $a_{(2)}$ 作为输入，进行logistic regression的求解

$AND: h_\theta(x)=g(20x_1+20x_2-30)$

$OR: h_\theta(x)=g(20x_1+20x_2-10)$

$NOT: h_\theta(x)=g(10-20x)$

$((NOT\ x_1)\ AND\ (NOT\ x_2)): h_\theta(x)=g(10-20x_1-20x_2)$

可以用 $AND$ , $OR$ , $NOT\ AND\ NOT$ 组合 $XNOR$

越多层网络可以表达越复杂的网络