modeci_mdf.functions.onnx.rnn
- modeci_mdf.functions.onnx.rnn(*args, **kwargs)
Computes an one-layer simple RNN. This operator is usually supported via some custom implementation such as CuDNN.
Notations:
X - input tensor
i - input gate
t - time step (t-1 means previous time step)
Wi - W parameter weight matrix for input gate
Ri - R recurrence weight matrix for input gate
Wbi - W parameter bias vector for input gate
Rbi - R parameter bias vector for input gate
WBi - W parameter weight matrix for backward input gate
RBi - R recurrence weight matrix for backward input gate
WBbi - WR bias vectors for backward input gate
RBbi - RR bias vectors for backward input gate
H - Hidden state
num_directions - 2 if direction == bidirectional else 1
Activation functions:
Relu(x) - max(0, x)
Tanh(x) - (1 - e^{-2x})/(1 + e^{-2x})
Sigmoid(x) - 1/(1 + e^{-x})
(NOTE: Below are optional)
Affine(x) - alpha*x + beta
LeakyRelu(x) - x if x >= 0 else alpha * x
ThresholdedRelu(x) - x if x >= alpha else 0
ScaledTanh(x) - alpha*Tanh(beta*x)
HardSigmoid(x) - min(max(alpha*x + beta, 0), 1)
Elu(x) - x if x >= 0 else alpha*(e^x - 1)
Softsign(x) - x/(1 + |x|)
Softplus(x) - log(1 + e^x)
Equations (Default: f=Tanh):
Ht = f(Xt*(Wi^T) + Ht-1*(Ri^T) + Wbi + Rbi)
This operator has optional inputs/outputs. See [the doc](IR.md) for more details about the representation of optional arguments. An empty string may be used in the place of an actual argument’s name to indicate a missing argument. Trailing optional arguments (those not followed by an argument that is present) may also be simply omitted.