GitHub

RNN Network

Structure

Dimensions

$X\in \mathbb{R}^{n} $ \;,\; $Y\in \mathbb{R}^{n} $ \;,\; $S\in \mathbb{R}^{d} $ \;,\; $Z\in \mathbb{R}^{n} $ \;,\; $U\in \mathbb{R}^{d . n} $ \;,\; $V\in \mathbb{R}^{n . d} $ \;,\; $W\in \mathbb{R}^{d . d}$

Relations

$Y^t=V.S^t$ \;\; et \;\; $S^t=f(Z^t)$ \;\; avec \;\ $Z^t=U.X^t + W.S^{t-1}$ \;\; et \;\; $f = tanh$

Gradients

U

$\frac{\partial E^T}{\partial U} = \left ( \sum_{t=1}^{T} \frac{\partial E^t}{\partial u_{ij}}\right)_{ij}$

$\frac{\partial E^t}{\partial u_{ij}} = \sum_{k=1}^{n}\sum_{l=1}^{d}-2g(\hat{y}^t_k-y^t_k) . v_{kl} . \frac{\partial s^t_l}{\partial u_{ij}}$

$\frac{\partial s_l^0}{\partial u_{ij}} = 0 \;\;\;\; et \;\;\;\; \forall t \in [0,T], \;\frac{\partial s_l^t}{\partial u_{ij}} = f'(z^t_l).\left(x^t_j.\delta_{li} + \sum_{m=1}^{d}w_{lm}\frac{\partial s^t_m}{\partial u_{ij}} \right)$

V

$\frac{\partial E^T}{\partial V} = \left ( \sum_{t=1}^{T} \frac{\partial E^t}{\partial v_{ij}}\right)_{ij} = \left( \sum_{t=1}^{T}\frac{\partial E^t}{\partial y_i} . \frac{\partial y_i}{\partial v_{ij}} \right)_{ij}$

Results

Several Experts Only one expert

Several layers

Low level getes High level gates

Name		Name	Last commit message	Last commit date
Latest commit History 29 Commits
data		data
img		img
launch		launch
scripts		scripts
CMakeLists.txt		CMakeLists.txt
README.md		README.md
package.xml		package.xml

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

RNN Network

Structure

Dimensions

Relations

Gradients

U

$\frac{\partial s_l^0}{\partial u_{ij}} = 0 \;\;\;\; et \;\;\;\; \forall t \in [0,T], \;\frac{\partial s_l^t}{\partial u_{ij}} = f'(z^t_l).\left(x^t_j.\delta_{li} + \sum_{m=1}^{d}w_{lm}\frac{\partial s^t_m}{\partial u_{ij}} \right)$

V

Results

Several layers

About

Releases

Packages

Contributors 2

Languages

Aightech/rnn_python

Folders and files

Latest commit

History

Repository files navigation

RNN Network

Structure

Dimensions

Relations

Gradients

U

V

Results

Several layers

About

Resources

Stars

Watchers

Forks

Releases

Packages 0

Contributors 2

Languages

Packages