Skip to content

Commit

Permalink
Merge pull request chenzomi12#289 from rebornwwp/main
Browse files Browse the repository at this point in the history
Updatee 03GradMode.md 更正反向传播参数错误
  • Loading branch information
chenzomi12 authored Oct 26, 2024
2 parents 004b81f + fc98eec commit 4f922a7
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 2 deletions.
2 changes: 1 addition & 1 deletion 05Framework/01Foundation/02Fundamentals.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ $$ loss(w)=f(w)-g $$

按照高中数学的基本概念,假设神经网络是一个复合函数(高维函数),那么对这个复合函数求导,用的是链式法则。举个简单的例子,考虑函数 $z=f(x,y)$,其中 $x=g(t),t=h(t)$ ,其中 $g(t), h(t)$ 是可微函数,那么对函数 $z$ 关于 $t$ 求导,函数会顺着链式向外逐层进行求导。

$$ \frac{\mathrm{d} x}{\mathrm{d} t} = \frac{\partial z}{\partial x} \frac{\mathrm{d} x}{\mathrm{d} t} + \frac{\partial z}{\partial y} \frac{\mathrm{d} y}{\mathrm{d} t} $$
$$ \frac{\mathrm{d} z}{\mathrm{d} t} = \frac{\partial z}{\partial x} \frac{\mathrm{d} x}{\mathrm{d} t} + \frac{\partial z}{\partial y} \frac{\mathrm{d} y}{\mathrm{d} t} $$

既然有了链式求导法则,而神经网络其实就是个庞大的复合函数,直接求导不就解决问题了吗?反向到底起了什么作用?下面来看几组公式。

Expand Down
2 changes: 1 addition & 1 deletion 05Framework/02AutoDiff/03GradMode.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ $$
转化成如上 DAG(有向无环图)结构之后,我们可以很容易分步计算函数的值,并求取它每一步的导数值,然后,我们把 $df/dx_1$ 求导过程利用链式法则表示成如下的形式:

$$
\dfrac{df}{dx_1}= \dfrac{dv_{-1}}{dx_1} \cdot (\dfrac{dv_{1}}{dv_{-1}} \cdot \dfrac{dv_{4}}{dv_{1}} + \dfrac{dv_{2}}{dv_{-1}} \cdot \dfrac{dv_{4}}{dx_{2}}) \cdot \dfrac{dv_{5}}{dv_{4}} \cdot \dfrac{df}{dv_{5}}
\dfrac{df}{dx_1}= \dfrac{dv_{-1}}{dx_1} \cdot (\dfrac{dv_{1}}{dv_{-1}} \cdot \dfrac{dv_{4}}{dv_{1}} + \dfrac{dv_{2}}{dv_{-1}} \cdot \dfrac{dv_{4}}{dv_{2}}) \cdot \dfrac{dv_{5}}{dv_{4}} \cdot \dfrac{df}{dv_{5}}
$$

> 整个求导可以被拆成一系列微分算子的组合。
Expand Down

0 comments on commit 4f922a7

Please sign in to comment.