ccc-台大林轩田机器学习基石-hw0

news2024/11/24 12:49:09

文章目录

- - 1 Probability and Statistics
  - 2 Linear Algebra
  - 3.Calculus
  - 一览图
  - hw0题目链接

1 Probability and Statistics

不妨假设 $\le K\le N$ 成立，只需证明 $K)=\frac {(n+1)!}{K!(n-K+1)!} ，0 \le K\le N$ 即可

K=0， $C (n + 1, K) = 1$ ，符合假设
$1\le K\le n$ ，如下

$\begin{aligned}C(n+1, K)&=C(n, K)+C(n,K-1)\\&=\frac {n!}{K!(n-K)!}+ \frac {n!}{(K-1)!(n-K+1)!}\\&=\frac {n!(n-K+1 + K)}{K!(n-K+1)!}\\&=\frac {n!(n+1)}{K!(n-K+1)!}\\&=\frac {(n+1)!}{K!(n-K+1)!}\end{aligned}$

10次硬币4次正面 $\frac{C_{10}^{4}}{2^{10}}=\frac {105}{512}$
13组，选2组进行组合 $\frac {13\times 12 \times C_4^3 \times C_4^2}{C_{52}^5}=\frac {6}{4165}$
$P(三次投掷有一次正面）=1-\frac{1}{8}=\frac{7}{8}$
$P(三次投掷有三次次正面）=\frac{1}{8}$
$\mathbb p=\frac{\frac{1}{8}}{\frac{7}{8}}=\frac{1}{7}$

随机位为0应该题目写错了，理解为大于小于0即可
令 $\begin{aligned}A&=\{|X| =1 \}\\B&=\{X<0 \}\end{aligned}$

则 $\begin{aligned}p&= P(B |A)\\&= \frac{ P(AB)}{P(A)}\\&=\frac{ P(X=-1)}{ P(|X| =1 )}\\&=\frac{\frac 12 \times \frac 1 4}{\frac 12 \times \frac 1 8 +\frac 12 \times \frac 1 4}\\&=\frac 2 3 \end{aligned}$

$\cap B)=min\{P(A),P(B)\}=0.3$
$\cap B)=0$
$\cup B)=P(A)+P(B)-minP(AB)=0.7$
$\cup B)=P(A)+P(B)-maxP(AB)=0.4$

2 Linear Algebra

初等行变换即可，r为2
$\left( \begin{matrix} 1 & 2 & 1 \\ 1 & 0& 3 \\ 1 & 1& 2 \end{matrix} \right) \overset{(2)-(1)} {\longrightarrow} \left( \begin{matrix} 1 & 2 & 1 \\ 0 & -2& 2 \\ 1 & 1& 2 \end{matrix} \right)\overset{(3)-(1)} {\longrightarrow} \\ \left( \begin{matrix} 1 & 2 & 1 \\ 0 & -2& 2 \\ 0 & -1& 1 \end{matrix} \right)\overset{(2)/2} {\longrightarrow}\left( \begin{matrix} 1 & 2 & 1 \\ 0 & -1& 1 \\ 0 & -1& 1 \end{matrix} \right)\overset{(3)+(2)} {\longrightarrow} \left( \begin{matrix} 1 & 2 & 1 \\ 0 & -1& 1 \\ 0 & 0& 0 \end{matrix} \right)$

$\left( \begin{matrix} 0 & 2 & 4 \\ 2 & 4& 2 \\ 3 & 3& 1 \end{matrix} \right)^{-1}=\frac{\left( \begin{matrix} 0 & 2 & 4 \\ 2 & 4& 2 \\ 3 & 3& 1 \end{matrix} \right)^*}{|\left( \begin{matrix} 0 & 2 & 4 \\ 2 & 4& 2 \\ 3 & 3& 1 \end{matrix} \right)|}=\frac{\left( \begin{matrix} -2 & 10 & -12 \\ 4 & -12& 8 \\ -6 & 6& -4 \end{matrix} \right)}{-16}$

$(A-\lambda E)=\left( \begin{matrix} 3-\lambda & 1 & 1 \\ 2 & 4-\lambda & 2 \\ -1 & -1 & 1-\lambda \end{matrix} \right) \longrightarrow \left( \begin{matrix} 4-\lambda & 4-\lambda & 4-\lambda \\ 0 & 2-\lambda & 0 \\ 0 & 0 & 2-\lambda \end{matrix} \right)\\ =(4-\lambda)(2-\lambda)^2 \\$

当特征值为4时，有：
$A-4E=\left( \begin{matrix} -1 & 1 & 1 \\ 2 & 0 & 2 \\ -1 & -1 & -3 \end{matrix} \right)\longrightarrow \left( \begin{matrix} 1 & 0 & 1 \\ 0 & 1 & 2 \\ 0 & 0 & 0 \end{matrix} \right)$

求得 $\vec x_1=\left( \begin{matrix} 1 \\ 2 \\ -1 \end{matrix} \right)$

特征值为2时，同理可以求得 $\vec x_2=\left( \begin{matrix} 1 \\ 0 \\ -1 \end{matrix} \right)$ 和 $\vec x_3=\left( \begin{matrix} 1 \\ -1 \\ 0 \end{matrix} \right)$

(a)
不妨认为 $M\in\mathbb R^{m\times n},U\in \mathbb R^{m\times m},\Sigma\in \mathbb R^{m\times n}, V\in \mathbb R^{n\times n}$ 易得 $\Sigma^{\dagger}\in \mathbb R^{n\times m}$
所以有 $\begin{aligned} MM^{\dagger}M &=U\Sigma V^TV\Sigma^{\dagger} U^TU\Sigma V^T\\ &=U(\Sigma\Sigma^{\dagger})\Sigma V^T\\ &=UI_m\Sigma V^T\\ &=U\Sigma V^T \end{aligned}$
(b)
$\begin{aligned} M^{\dagger} &= V\Sigma^{\dagger} U^T(M 可逆，则m=n，且\Sigma^{\dagger}=\Sigma^{-1}) \\ &=V\Sigma^{-1} U^T \\ &=(U\Sigma V^T)^{-1}\\ &=M^{-1} \end{aligned}$

(a)
$\forall x，x^T ZZ^T x= (Z^Tx)^T(Z^Tx)\ge 0$ ，得证
(b)

必要性：由实对称矩阵可知，存在正交矩阵Q和对角矩阵 $\Lambda$ 使面等式成立
$Q^TAQ=\Lambda,\Lambda =\text{diag} \{\lambda_1,...,\lambda_n\}$

由已知易知， $\Lambda$ >0，则必要性证明成功

充分性：取 $Qe_i\neq 0,i=1,...,n$ ，且 $e_i \in \mathbb R^n ,(e_i)_j = 1\{i=j\}$

则有 $\begin{aligned} x^T \Lambda x &= e_i^T Q^TA Qe_i \\ &=e_i^T \Lambda e_i \\ &=\lambda_i\\ &>0 \end{aligned}$ 充分性证明成功
在这里插入图片描述

$max u^tx=1$ ，u和x同向
$min u^tx=-1$ ，u和x反向
$min |u^tx|=0$ ，u和x正交

3.Calculus

$\begin{aligned} \frac {d f(x)}{dx} &= \frac {-2e^{-2x}} {1+e^{-2x}}=-\frac {2}{1+e^{2x}} \\ \frac {\partial g(x,y)}{\partial y} &= 2e^{2y}+e^{3xy^2} 6xy \end{aligned}$

$\begin{aligned} \frac {\partial f}{\partial v} &=\frac {\partial f}{\partial x} \frac {\partial x}{\partial v}+ \frac {\partial f}{\partial y} \frac {\partial y}{\partial v}\\ &=-y\sin(u+v)-x\cos (u-v) \end{aligned}$

一阶偏导：
$\begin{aligned} \frac{\partial E(u,v)}{\partial u} &=2(ue^v -2ve^{-u} )(e^v +2ve^{-u})\\ \frac{\partial E(u,v)}{\partial v} &=2(ue^v -2ve^{-u} )(ue^v -2e^{-u})\\ \end{aligned}$
二阶偏导：
$\begin{aligned} \frac{\partial^2 E(u,v)}{\partial u^2} &=2\frac{\partial}{\partial u}(ue^{2v} -2v e^{v-u}+2uve^{v-u}-4v^2 e^{-2u})\\ &=2(e^{2v} +2v e^{v-u}+2ve^{v-u}-2uve^{v-u}+8v^2 e^{-2u}) \\ &=2(e^{2v} +4v e^{v-u}-2uve^{v-u}+8v^2 e^{-2u})\\ \frac{\partial^2 E(u,v)}{\partial v^2} &=2\frac{\partial}{\partial v}(u^2e^{2v} -2uv e^{v-u}-2ue^{v-u}+4ve^{-2u})\\ &=2(2u^2e^{2v} -2u e^{v-u}-2uv e^{v-u}-2ue^{v-u}+4e^{-2u})\\ &=2(2u^2e^{2v} -4u e^{v-u}-2uv e^{v-u}+4e^{-2u}) \\ \frac{\partial^2 E(u,v)}{\partial u \partial v} &=2\frac{\partial}{\partial v}(ue^{2v} -2v e^{v-u}+2uve^{v-u}-4v^2 e^{-2u})\\ &=2(2ue^{2v} -2 e^{v-u}-2ve^{v-u}+2ue^{v-u}+2uve^{v-u} -8v e^{-2u}) \\ \end{aligned}$
代入题目条件u、v=1有
$\begin{aligned} \frac{\partial E(u,v)}{\partial u} \Big |_{u=1,v=1} &=2(e^2 -4e^{-2}) \\ \frac{\partial E(u,v)}{\partial v} \Big |_{u=1,v=1} &=2(e^2 +4e^{-2}-4) \\ \frac{\partial^2 E(u,v)}{\partial u^2}\Big |_{u=1,v=1}& =2(e^2+8e^{-2}+2) \\ \frac{\partial^2 E(u,v)}{\partial v^2}\Big |_{u=1,v=1}& =2(2e^2+4e^{-2}-6) \\ \frac{\partial^2 E(u,v)}{\partial u\partial v}\Big |_{u=1,v=1}& =2(2e^2 -8e^{-2}) \\ \end{aligned}$
最后结果为：
$\begin{aligned} \nabla E= \left( \begin{matrix} 2(e^2 -4e^{-2}) \\ 2(e^2 +4e^{-2}-4) \end{matrix} \right) , \nabla^2 E = \left( \begin{matrix} 2(e^2+8e^{-2}+2) & 2(2e^2 -8e^{-2}) \\ 2(2e^2 -8e^{-2}) & 2(2e^2+4e^{-2}-6) \end{matrix} \right) \end{aligned}$
在这里插入图片描述
$\begin{aligned} E(u,v) &\approx E(1, 1)+ \nabla E^T \left( \begin{matrix} u- 1 \\ v- 1 \end{matrix} \right) + \frac 1 {2!} \left( \begin{matrix} u- 1 & v- 1 \end{matrix} \right) \nabla^2 E \left( \begin{matrix} u- 1 \\ v- 1 \end{matrix} \right) \\ &=(e-2e^{-1})^2 + 2(e^2 -4e^{-2}) (u-1)+ 2(e^2 +4e^{-2}-4)(v-1)\\ &+(e^2+8e^{-2}+2) (u-1)^2 + (2e^2+4e^{-2}-6)(v-1)^2+2(2e^2 -8e^{-2})(u-1)(v-1) \end{aligned}$

$\begin{aligned} &Ae^{\alpha} +Be^{-2\alpha}\\ &=\frac A 2 e^{\alpha}+\frac A 2 e^{\alpha} +Be^{-2\alpha}\\ &\ge 3 \sqrt[3] {\frac A 2 e^{\alpha}\times\frac A 2 e^{\alpha} \times Be^{-2\alpha}} \\ &=3\sqrt[3]{\frac {A^2B} 4} \end{aligned}$ 当且仅当
$\begin{aligned}&\frac A 2 e^{\alpha}=\frac A 2 e^{\alpha} =Be^{-2\alpha}\\ &\alpha =\frac 1 3 \ln \frac {2B} A\end{aligned}$

原式展开有：
$\begin{aligned} E(w) & =\frac 1 2 \sum_{i=1}^d \sum_{j=1}^d w_{i}A_{ij}w_j + \sum_{i=1}^d w_i b_i \end{aligned}$
由于A是对称矩阵：
$\begin{aligned} \frac{\partial E(w)}{\partial w_k} &=\frac 1 2 \sum_{j=1}^d A_{kj}w_j+\frac 1 2 \sum_{i=1}^d w_{i}A_{ik} + b_k \\ &=\sum_{j=1}^d A_{kj}w_j +b_k \\ \frac{\partial^2 E(w)}{\partial w_l\partial w_k} &= \frac{\partial}{\partial w_l} \Big(\sum_{j=1}^d A_{kj}w_j +b_k \Big)\\ &= A_{kl} \\ \end{aligned}$
合成为矩阵形式有：
$\begin{aligned} \nabla E(w)& =Aw+b\\ \nabla E^2(w)& =A \end{aligned}$