2D RNN

2D Recurrent Layer implementation.

Forward

  • Netzparameter: $\bar S$ Init Parameter Stati; $W$, $\bar W$ weights for recursive connections; $b$ constant.
  • $S^{(0,u)}=S^{(t,0)}=\bar S$
  • Für $1\le t\le T$ $1\le u\le U$
  • $z^{(t,u)}=W^IX^{(t,u)}+WS^{(t-1,u)}+ \bar WS^{(t,u-1)}+b$; with $X$ Inputs
  • $S^{t,u}=g(z^{(t,u)})$

Backward

  • ${\delta E\over\delta z_i^{(t,u)}}={\delta S_i^{(t,u)}\over\delta z_i^{(t,u)}}{\delta E\over\delta S_i^{(t,u)}}=g'(z_i^{(t,u)}){\delta E\over\delta S_i^{(t,u)}}$
  • ${\delta E\over\delta S_i^{(t,u)}}=\sum_j{\delta z_j^{(t+1,u)}\over\delta S_i^{(t,u)}}{\delta E\over\delta z_j^{(t+1,u)}}+
    \sum_j{\delta z_j^{(t,u+1)}\over\delta S_i^{(t,u)}}{\delta E\over\delta z_j^{(t,u+1)}}
    =\sum_j w_{ji}{\delta E\over\delta z_j^{(t+1,u)}}+\sum_j \bar w_{ji}{\delta E\over\delta z_j^{(t,u+1)}}
    $ + influence $S_i^{(t,u)}$ on child layer.
  • ${\delta E\over\delta X_i^{(t,u)}}$
  • ${\delta E\over\delta W_{i,j}^{(t,u)}}$
  • ${\delta E\over\delta \bar W_{i,j}^{(t,u)}}$
  • ${\delta E\over\delta b_{i}^{(t,u)}}$
  • ${\delta E\over\delta\bar S_i^{(t,u)}}$