无监督学习 (Unsupervised Learning)
K-means
Randomly initialize $K$ cluster centroids $\mu_1,\mu_2,\dots,\mu_K \in \mathbb{R}^n$
Repeat {
$\space\space\space\space$for i = 1 to m
$\space\space\space\space$$c^{(i)}$ := index (from 1 to $K$ ) of cluster centroid closest to $x^{(i)}$
$\space\space\space\space$for k = 1 to $K$
$\space\space\space\space\space\space\space\space$$\mu_k$ := average (mean) of points assigned to cluster
}
Cost-Function：
$$ \begin{equation}J(c^{(1)},\dots,c^{(m)},\mu_1,\dots,\mu_K)={1\over m}\sum_{i=1}^m\lVert x^{(i)}-\mu_{c^{(i)}}\rVert \end{equation} $$
- 在一次K-means训练过程中，代价函数一定是单调递减的
- 不同次K-means训练结果不是k越大效果越好，训练结果也与centroid初始化情况有关
主成分分析法PCA(Priciple Component Analysis)
After mean normalization(ensure every feature has zero mean) and optionally feature scaling:
- $\Sigma={1\over m}\sum_{i=1}^m(x^{(i)})(x^{(i)})^T$
- $[U,S,V]=svd(\Sigma)$
- $U_{reduce} = U(:,1:k)$
- $z=U_{reduce}^Tx$
Choosing $k$，number of principle components
$$ \begin{equation}{\sum_{i=1}^m \lVert x^{(i)}-x^{(i)}{approx} \rVert^2 \over\sum{i=1}^m \lVert x^{(i)} \rVert^2}={\sum_{i=1}^m \lVert x^{(i)}-U_{reduce}z^{(i)} \rVert^2 \over \sum_{i=1}^m \lVert x^{(i)} \rVert^2}= 1-{\sum_{i=1}^kS_{ii}\over\sum_{i=1}^nS_{ii}}\end{equation} $$
选择$k$的原则是使得上述式子小于一个比较小的$\epsilon \in (0,1)$
异常检测(Anomaly Detection)
正态分布
$$ \begin{equation} p(x) = {1\over \sqrt{2\pi}\sigma}e^{-{(x-\mu)^2\over2\sigma^2}} \end{equation} $$
其中$\mu={1\over m }\sum_{i=1}^mx^{(i)}$，$\sigma^2={1\over m}\sum_{i=1}^m{(x^{(i)}-\mu)^2}$
根据$p(x)$判断$x$ normaly or abnormally
多元正态分布
$$ \begin{equation} p(x) = {1\over {(2\pi)}^{n\over 2}|\Sigma|^{1\over 2}}e^{-{1\over 2}(x-\mu)^T\Sigma^{-1}(x-\mu)} \end{equation} $$
其中$\mu={1\over m }\sum_{i=1}^mx^{(i)}$，$\Sigma={1\over m}\sum_{i=1}^m{(x^{(i)}-\mu)(x^{(i)}-\mu)^T}$
根据$p(x)$判断$x$ normaly or abnormally