modeci_mdf.functions.onnx.lstm

modeci_mdf.functions.onnx.lstm(*args, **kwargs)

Computes an one-layer LSTM. This operator is usually supported via some custom implementation such as CuDNN.

Notations:

• X - input tensor

• i - input gate

• o - output gate

• f - forget gate

• c - cell gate

• t - time step (t-1 means previous time step)

• W[iofc] - W parameter weight matrix for input, output, forget, and cell gates

• R[iofc] - R recurrence weight matrix for input, output, forget, and cell gates

• Wb[iofc] - W bias vectors for input, output, forget, and cell gates

• Rb[iofc] - R bias vectors for input, output, forget, and cell gates

• P[iof] - P peephole weight vector for input, output, and forget gates

• WB[iofc] - W parameter weight matrix for backward input, output, forget, and cell gates

• RB[iofc] - R recurrence weight matrix for backward input, output, forget, and cell gates

• WBb[iofc] - W bias vectors for backward input, output, forget, and cell gates

• RBb[iofc] - R bias vectors for backward input, output, forget, and cell gates

• PB[iof] - P peephole weight vector for backward input, output, and forget gates

• H - Hidden state

• num_directions - 2 if direction == bidirectional else 1

Activation functions:

• Relu(x) - max(0, x)

• Tanh(x) - (1 - e^{-2x})/(1 + e^{-2x})

• Sigmoid(x) - 1/(1 + e^{-x})

NOTE: Below are optional

• Affine(x) - alpha*x + beta

• LeakyRelu(x) - x if x >= 0 else alpha * x

• ThresholdedRelu(x) - x if x >= alpha else 0

• ScaledTanh(x) - alpha*Tanh(beta*x)

• HardSigmoid(x) - min(max(alpha*x + beta, 0), 1)

• Elu(x) - x if x >= 0 else alpha*(e^x - 1)

• Softsign(x) - x/(1 + |x|)

• Softplus(x) - log(1 + e^x)

Equations (Default: f=Sigmoid, g=Tanh, h=Tanh):

• it = f(Xt*(Wi^T) + Ht-1*(Ri^T) + Pi (.) Ct-1 + Wbi + Rbi)

• ft = f(Xt*(Wf^T) + Ht-1*(Rf^T) + Pf (.) Ct-1 + Wbf + Rbf)

• ct = g(Xt*(Wc^T) + Ht-1*(Rc^T) + Wbc + Rbc)

• Ct = ft (.) Ct-1 + it (.) ct

• ot = f(Xt*(Wo^T) + Ht-1*(Ro^T) + Po (.) Ct + Wbo + Rbo)

• Ht = ot (.) h(Ct)

This operator has optional inputs/outputs. See [the doc](IR.md) for more details about the representation of optional arguments. An empty string may be used in the place of an actual argument’s name to indicate a missing argument. Trailing optional arguments (those not followed by an argument that is present) may also be simply omitted.