Merge

Forward

  • $z^t=\sum_i h^{t-1,i}$ with $h^{t-1,i}\in R^{n\times m}$ Inputs from previous layers.
  • $h^t=g(z^t)$ with $g$ activation.

Back

  • ${\delta E\over\delta z^t}={\delta h^t\over\delta z^t}{\delta E\over\delta h^t}=g'(z^t){\delta E\over\delta h^t}$
  • ${\delta E\over\delta h^{t-1,i}}={\delta z^t\over\delta h^{t-1,i}}{\delta E\over\delta z^t}={\delta E\over\delta z^t}$