ModECI Model Description Format (MDF)

A simple example with 2 Nodes connected by an Edge:

With more detail on Nodes (showing Input Ports (green), Parameters (blue) and Output Ports) (red) and Edges:

ABCD

Another simple example with more Nodes.

Arrays

An example using arrays for Parameters and weights on Edges.

States

An example with Nodes containing persistent States.

Conditions

A simple 3 Nodes graph with scheduling Conditions. For more examples of conditions see here.

Parameters and Functions

A simple Node with a number of different types of Parameters (in blue; fixed and stateful) and Functions (in purple; can be built in or ONNX based).

More examples

There are further examples under development, including of a Recurrent Neural Network (RNN), and an Integrate and Fire (IaF) neuron model in this directory.

Interactions between MDF and ACT-R

This directory contains examples of ACT-R models converted to MDF. The ACT-R models count.lisp and addition.lisp are based on the ACT-R tutorial.

The scripts count.py and addition.py can be run to create the MDF .json and .yaml files for the given example and execute it using the MDF scheduler.

The below graph represents the basic structure of all ACT-R models in MDF:

There are also more detailed graphs count.png and addition.png for each example.

Count Model

ACT-R | JSON | YAML | Python Script | Graph

The count model counts from a start value to an end value. The script count.py first reads the original ACT-R model file count.lisp, generates an MDF representation using the MDF ACT-R interface, and outputs the JSON and YAML files. It then executes the MDF model using the MDF scheduler and outputs the final goal set by the model once execution is finished. The final goal has the form:

{'name': 'first-goal', 'ISA': 'count-from', 'start': 'two', 'end': 'four', 'count': 'four'}

In this example, the start value is two, the end value is four, and the final value of count is four, indicating that the model counted from two to four. The start and end values can be modified in line 17 of count.lisp, which sets the initial goal of the model:

(first-goal ISA count-from start two end four)

The script can use any values specified in count.lisp, so the model can be modified and run multiple times in order to test different values. The count graph represents this example.

Addition Model

ACT-R | JSON | YAML | Python Script | Graph

The addition model computes the sum of two numbers. The script addition.py functions identically to the previous example, but uses the addition model instead. The final goal has the form:

{'name': 'second-goal', 'ISA': 'add', 'arg1': 'five', 'arg2': 'two', 'sum': 'seven', 'count': 'nil'}

In this case, the first argument is five, the second argument is two, and the model has calculated the sum, seven. Like the previous example, the arguments can be modified in line 23 of addition.lisp in order to test different values:

(second-goal ISA add arg1 five arg2 two)

The addition graph represents this example.

Interactions between NeuroML and MDF

1) Converting NeuroML to MDF

1.1) Simple ABCD model

Summary: A model is created in NeuroML (using cell dynamics specified in LEMS and a network in NeuroMLlite) and converted to the equivalent model in MDF, which runs with the reference MDF execution engine.

1.1.1) ABCD - NeuroMLlite version

ABCD.py is a script using the NeuroMLlite package to create a simple network with 4 connected elements. The network built can be seen below (this can be generated with python ABCD.py -graph2):

ABCD.nmllite.png

1.1.2) ABCD - NeuroML2 version

A version of the network in NeuroML 2 can be generated with python ABCD.py -nml, or generated and executed with jNeuroML with python ABCD.py -nml. This will produce the NeuroML file: ABCD.net.nml (note though this is not valid, as not all the elements included are pure NeuroML). A LEMS Simulation file is generated for running the model in jNeuroML or pyNeuroML: LEMS_SimABCD.xml

The definitions of the components used for A, B, etc. can be found in PNL.xml. This is a set of definitions of component types based on those present in PsyNeuLink. A graph depicting the definitions of the network elements can be generated with pynml LEMS_SimABCD.xml -lems-graph:

1.1.3) ABCD - MDF version

A version of the network in MDF can be generated from NeuroMLlite definition with: python ABCD.py -mdf producing ABCD.mdf.yaml and ABCD.mdf.json.

A graph of the structure of the MDF model can be generated with: python -m modeci_mdf.interfaces.graphviz.exporter ABCD.mdf.yaml 1 (left below), or with more detail: python -m modeci_mdf.interfaces.graphviz.exporter ABCD.mdf.yaml 3 (right below.)

ABCD.1.mdf.png ABCD.mdf.png

1.2) FitzHugh Nagumo cell models

1.2.1) FN - NeuroML version

A version of the FitzHugh Nagumo neuron model has been created using NeuroMLlite (FN.py) which generated LEMS (LEMS_SimFN.xml) which can simulate the NeuroML model (FN.net.nml).

A graphical representation of the LEMS is below:

It can be run with:

python FN.py -jnml      # Generate and run the LEMS file from the NeuroMLlite description
pynml LEMS_SimFN.xml    # Run the LEMS file using pyNeuroML

1.2.2) FN - MDF version

The NeuroMLlite version can also be used to generate MDF for the model:

python FN.py -mdf      # Generate the MDF serializations (JSON and YAML) from the NeuroMLlite description

These can be seen here: FN.mdf.json, FN.mdf.yaml, and a graphical version generated with:

python -m modeci_mdf.interfaces.graphviz.importer FN.mdf.yaml 3    #  Generate graph from MDF version

FN.gv.png

1.2.3) FN - Execute model using MDF

A script has been created (FNrun.py) where the model is loaded, run using the standard MDF execution engine, and plotted:

python FNrun.py    # Load FN model and run with MDF scheduler

Adding the option -multi to the Python script for running the FN example, modifies the graph to add an input node with an array of values, meaning multiple instances of the FN neuron will be simulated:

python FN.py -multi

MDFFNrun.multi.png

1.3) Izhikevich cell models

A version of the Izhikevich spiking neuron model has been created in NeuroML and can be exported to MDF and executed with the standard execution engine.

1.3.1) Izhikevich - NeuroML version

The single cell model has been created using NeuroMLlite (Izhikevich.py) which generated LEMS (LEMS_SimIzhikevichTest.xml) which can simulate the NeuroML model (IzhikevichTest.net.nml).

A graphical representation of the LEMS is below:

It can be run with:

python Izhikevich.py -jnml      # Generate and run the LEMS file from the NeuroMLlite description
pynml LEMS_SimIzhikevichTest.xml    # Run the LEMS file using pyNeuroML

1.3.2) Izhikevich - MDF version

The NeuroMLlite version can also be used to generate MDF for the model:

python Izhikevich.py -mdf      # Generate the MDF serializations (JSON and YAML) from the NeuroMLlite description

These can be seen here: IzhikevichTest.mdf.json, IzhikevichTest.mdf.yaml, and a graphical version generated with:

python -m modeci_mdf.interfaces.graphviz.importer IzhikevichTest.mdf.yaml 3    #  Generate graph from MDF version

IzhikevichTest.gv.png

1.3.3) Izhikevich - Execute model using MDF

A script has been created (Izh_run.py) where the model is loaded, run using the standard MDF execution engine, and plotted:

python Izh_run.py    # Load Izh model and run with MDF scheduler

2) Converting MDF to NeuroML/LEMS

It is also possible to convert MDF models into equivalents in NeuroML/LEMS:

cd ../MDF  # convert some of the examples in the examples/MDF directory

python -m modeci_mdf.interfaces.neuroml.exporter Simple.json -run
python -m modeci_mdf.interfaces.neuroml.exporter ABCD.json -run
python -m modeci_mdf.interfaces.neuroml.exporter States.json -run

ONNX MDF Converter

ONNX (Open Neural Network Exchange) is an important format for exchanging models between machine learning environments. It is used in the MDF function ontology, and models in ONNX format can be exported to MDF. Converting MDF->ONNX is best enabled currently by converting the model to PyTorch and from there to ONNX.

ONNX to MDF

AB Sequential Model - 2 nodes

Python source | JSON | YAML

This is an example of a PyTorch model with 2 nodes. First, the script saves the PyTorch model as ONNX and then converts this to MDF. The graphical view of the generated MDF is shown below.

ABC Sequential Model with Loop

Python source | JSON | YAML

Note: Example still in development!

This is an example of a PyTorch model that is implemented in onnx_mdf/examples/simple_abc.py. The model code is very simple:

import torch

class A(torch.nn.Module):
    def forward(self, x):
        return x + 1

@torch.jit.script
def loop_b(x, y):
    for i in range(int(y)):
        x = x / 10
    return x

class B(torch.nn.Module):
    def forward(self, x, y):
        return loop_b(x, y)

class C(torch.nn.Module):
    def forward(self, x):
        return x * 100

class ABC(torch.nn.Module):
    def __init__(self):
        super(ABC, self).__init__()
        self.A = A()
        self.B = B()
        self.C = C()

    def forward(self, x, B_loop_count):
        return self.C(self.B(self.A(x), B_loop_count))

This implements a PyTorch model with three modules. The modules process the input sequentially, and the inner B module has a loop construct.

ABC

It is exported to ONNX via a combination of tracing and scripting.

ABCD Branching Conditional Model

Python source | JSON | YAML

Note: Example still in development!

This is an example of a PyTorch model that have four components (A, B, C, D). We loop over the whole model 10 iterations. A is executed only on the first iteration, B is executed every iteration, C is executed every 5 times B is executed, and D is executed every 10 times B is executed. A, B, C, and D are each simple stateless linear functions. This type of conditional execution specification is common in PsyNeuLink. The PyTorch code for the model is fairly straightforward:

class Linear(torch.nn.Module):
    def __init__(self, slope=1.0, intercept=0.0):
        super(Linear, self).__init__()
        self.slope = slope
        self.intercept = intercept

    def forward(self, x):
        return self.slope*x + self.intercept


class ABCD(torch.nn.Module):
    def __init__(self, A, B, C, D):
        super(ABCD, self).__init__()
        self.A = A
        self.B = B
        self.C = C
        self.D = D

    def forward(self, x):

        # Since we are implementing conditions that reference the number of calls
        # to A and B, we need to keep track of this.
        num_A_calls = 0
        num_B_calls = 0

        # We need to initialize outputs, torchscript jit complains if c and d
        # are not defined in the FALSE branches of our conditionals.
        a = torch.zeros_like(x)
        b = torch.zeros_like(x)
        c = torch.zeros_like(x)
        d = torch.zeros_like(x)

        for i in range(10):

            # A: pnl.AtNCalls(A, 0),
            if num_A_calls == 0:
                a = self.A(x)
                num_A_calls = num_A_calls + 1

            # B: pnl.Always()
            b = self.B(a)
            num_B_calls = num_B_calls + 1

            # C: pnl.EveryNCalls(B, 5),
            if num_B_calls % 5 == 0:
                c = self.C(b)

            # D: pnl.EveryNCalls(B, 10)
            if num_B_calls % 10 == 0:
                d = self.D(b)

        return c, d

The ONNX IR representation of this model is shown below. The small computation sub-graphs contained in the if and else body attributes are not shown. These are either a simple multiplication and addition or an identity.

Interactions between PsyNeuLink and MDF

Simple

ABCD

An example with four Nodes, as in other environments.

SimpleLinear

SimpleLinear-conditional

A three-Node example with Conditions.

SimpleLinear-timing

The same model as in SimpleLinear-conditional with Conditions for timeline scheduling. Note: these conditions are still not fully implemented by the scheduler.

Nested

Nested without scheduling

A model with several Nodes in two Graphs, one of which contains the other.

Nested with scheduling

A similar model as in Nested without scheduling with Conditions.

SimpleFN

An example with a single Node using the PsyNeuLink implementation of the FitzHugh–Nagumo model.

SimpleFN-timing

The same model as in SimpleFN with Conditions for timeline scheduling. Note: these conditions are still not fully implemented by the scheduler.

SimpleFN-conditional

The same model in SimpleFN with scheduling Conditions that mimic the behavior in SimpleFN-timing.

Stroop

A model representing the Stroop effect with conflict monitoring that uses Conditions.

PyTorch and MDF

Models can be created in PyTorch and exported into MDF format, or MDF models can be converted to code which executes natively in PyTorch.

MDF to PyTorch

To export an MDF model to PyTorch, provide an MDF model as an input to the mdf_to_pytorch() function.

The output of mdf_to_pytorch is a PyTorch model.

mdf_to_pytorch(
      mdf_model: model in MDF format
      eval_models: Set Evaluation of model to True or False
      version: MDF version
      model_input: input file name
)

It returns a dictionary where key = model name and value = PyTorch model object.

A test script demonstrating conversion of MDF model to PyTorch is at MDF_to_PyTorch.py. This converts multiple MDF models to their respective PyTorch models. The converted models are available in folder: MDF_PyTorch.

Examples

Below are some working examples of this functionality.

1) Simple ABCD example

We convert one of the sample MDF examples ABCD.json:

ABCD

This is converted to PyTorch and can be seen here: ABCD_pytorch.py.

The PyTorch model is further converted to ONNX ABCD.onnx. An image of the contents of the ONNX model (visualized using NETRON) is below.

2) Multi-Layer Perceptron MDF to PyTorch Conversion:

To run an example where a simple Multi-Layer Perceptron (MLP) created using the MDF specification and executed using sample digit-recognition data, run:

python mlp_pure_mdf.py

A graph of the network can be created with python mlp_pure_mdf.py -graph:

The network can be run against images from the MNIST database with: python mlp_pure_mdf.py -run, and produce 98% accuracy. The image below shows the results of 300 images:

mlp_pure_mdf.results.png

PyTorch to MDF

The current implementation of our PyTorch to MDF conversion functionality is built on top of the TorchScript infrastructure provided by PyTorch. PyTorch models that can be translated to TorchScript (via torch.jit.script or torch.jit.trace) should then be able to be converted to their MDF representation automatically. Below are several working examples of this functionality.

To perform an PyTorch to MDF conversion, provide a PyTorch model as an input to the pytorch_to_mdf() function which is available in importer.py. The output of pytorch_to_mdf() is an MDF model.

pytorch_to_mdf(
      model: The model to translate into MDF.
      args: The input arguments for this model. If a nn.Module is passed then the model will be traced with these
          inputs. If a ScriptModule is passed, they are still needed to deterimine input shapes.
      trace: Force the use of tracing to compile the model. The default is to use torch.jit.script
      use_onnx_ops: Use ONNX ops when possible, fallback to ATEN ops when not available. Default is True. If False,
          use only ATEN ops.
)

Returns a translated MDF model.

Examples of usage

1) Simple PyTorch To MDF

This is a simple fully-connected neural network model example consisting of input image of 224 * 224 * 3 and resulting in two classes as the output To run an example of converting a PyTorch model written in PyTorch to its MDF representation simply run:

python simple_pytorch_to_mdf.py

Code is present in simple_pytorch_to_mdf.py The graph representation of the ONNX model can be generated with:

python simple_pytorch_to_mdf.py -graph-onnx

NOTE: This command will run the NETRON python server on the local host where we can export the graph as svg/png

The graph representation of the MDF model can be generated with:

python simple_pytorch_to_mdf.py -graph

Graphical export from MDF level 1:

simple_pytorch_to_mdf.1.png

Graphical export from MDF level 3:

To visualize the PyTorch model:

python simple_pytorch_to_mdf.py -graph-torch

The MDF for this model is the written to simple_pytorch_to_mdf.json. The model is then executed via the MDF scheduler and the results are compared to the native execution in PyTorch.

2) Inception Blocks Model

To run an example of converting a PyTorch InceptionV3 like model written in PyTorch to its MDF representation simply run:

python inception.py

Code is present in inception.py This will define the model in PyTorch, invoke the TorchScript tracing compiler, convert the underlying IR representation of the model to MDF. The MDF for this model is the written to inception.json. The model is then executed via the MDF scheduler and the results are compared to the native execution in PyTorch.

The graph representation of the MDF model can be generated with:

python inception.py -graph

Interactions between MDF and Quantum computing technologies

Starting summer 2021, we will develop tools for interfacing between MDF and quantum computers. This interface is motivated by expectations that quantum hardware will provide speedups for solving Ising-type MDF problems. We will address both gate- and annealing- based quantum computers:

for gate-based quantum computers, we will bridge from MDF to OpenQASM, the leading quantum Intermediate Representation.
for annealing-based quantum computers, we will target platforms such as D-Wave Ocean.

Our work will be agnostic to the exact quantum algorithm/solver used, though we will provide sample implementations using Variational Quantum Eigensolver (VQE) and Quantum Approximate Optimization Algorithm.

As a first step, we have begun developing implementations targeting quantum hardware for the key computations in several cognitive models as listed below. Next, we will extend MDF so that quantum implementations such as the ones we develop, can be expressed in it.

Tasks	Models	Key computations	Quantum algorithms
Two alternative forced choice	Quantum walk	Evolution, Projection	Unitary evolution, Hamiltonian simulation
Multiple alternative models	Potential wells	Eigenstates and values	Variational methods (e.g., subspace and deflation)
Bistable perception	Quantum walk	Evolution, projection	Unitary evolution, Hamiltonian simulation
Control	Leaky Competing Integrator	Optimization	Quantum annealing
Parameter estimation	Data fitting	Optimization	Quantum annealing

MDF in WebGME

This contains a tool for converting the MDF specification into JSON compatible with JSON importer. This allows us to programmatically create a metamodel and, as a result, use WebGME as a design environment for MDF.

Quick Start

Starting WebGME app

First, install the mdf_gme following:

NodeJS (LTS recommended)
MongoDB

Second, start mongodb locally by running the mongod executable in your mongodb installation (you may need to create a data directory or set --dbpath).

Then, run webgme start from the project root to start . Finally, navigate to http://localhost:8888 to start using mdf_gme!

Loading the spec into WebGME

First, install dependencies with npm install. Then convert the MDF specification using

node spec_to_gme.js path/to/MDF/spec.json

Finally, import the JSON into WebGME just like the examples (suffixed with “_meta”)!

Loading instances to and from WebGME importable JSON and MDF

node bin/instance_converter path/to/MDForGME/instance.json

Specification of standard functions in ModECI v0.4

Note: the ModECI MDF specification is still in development! See here for ongoing discussions. These functions are defined in Python API module modeci_mdf.functions.

ONNX Functions

Abs
Acos
Acosh
Add
And
ArgMax
ArgMin
Asin
Asinh
Atan
Atanh
AveragePool
BatchNormalization
Bernoulli
BitShift
Cast
CastLike
Ceil
Celu
Clip
Compress
Concat
ConcatFromSequence
Constant
ConstantOfShape
Conv
ConvInteger
ConvTranspose
Cos
Cosh
CumSum
DepthToSpace
DequantizeLinear
Det
Div
Dropout
DynamicQuantizeLinear
Einsum
Elu
Equal
Erf
Exp
Expand
EyeLike
Flatten
Floor
GRU
Gather
GatherElements
GatherND
Gemm
GlobalAveragePool
GlobalLpPool
GlobalMaxPool
Greater
GreaterOrEqual
HardSigmoid
HardSwish
Hardmax
Identity
InstanceNormalization
IsInf
IsNaN
LRN
LSTM
LeakyRelu
Less
LessOrEqual
Log
LogSoftmax
LpNormalization
LpPool
MatMul
MatMulInteger
Max
MaxPool
MaxRoiPool
MaxUnpool
Mean
MeanVarianceNormalization
Min
Mod
Mul
Multinomial
Neg
NegativeLogLikelihoodLoss
NonMaxSuppression
NonZero
Not
OneHot
Optional
OptionalGetElement
OptionalHasElement
Or
PRelu
Pad
Pow
QLinearConv
QLinearMatMul
QuantizeLinear
RNN
RandomNormal
RandomNormalLike
RandomUniform
RandomUniformLike
Range
Reciprocal
ReduceL1
ReduceL2
ReduceLogSum
ReduceLogSumExp
ReduceMax
ReduceMean
ReduceMin
ReduceProd
ReduceSum
ReduceSumSquare
Relu
Reshape
Resize
ReverseSequence
RoiAlign
Round
Scatter
ScatterElements
ScatterND
Selu
SequenceAt
SequenceConstruct
SequenceEmpty
SequenceErase
SequenceInsert
SequenceLength
Shape
Shrink
Sigmoid
Sign
Sin
Sinh
Size
Slice
Softmax
SoftmaxCrossEntropyLoss
Softplus
Softsign
SpaceToDepth
Split
SplitToSequence
Sqrt
Squeeze
StringNormalizer
Sub
Sum
Tan
Tanh
TfIdfVectorizer
ThresholdedRelu
Tile
TopK
Transpose
Trilu
Unique
Unsqueeze
Upsample
Where
Xor

MatMul

Matrix multiplication (work in progress...)

MatMul(A, B) = A @ B

Python version: A @ B

Relu

Rectified linear function (work in progress...)

Relu(A) = A * (A > 0)

Python version: A * (A > 0)

change_goal

Modifies the current goal buffer using the given pattern.

change_goal(pattern, curr_goal) = actr.change_goal(pattern,curr_goal)

Python version: actr.change_goal(pattern,curr_goal)

check_termination

Function used to check if no production was selected.

check_termination(production) = actr.check_termination(production)

Python version: actr.check_termination(production)

chunk_to_string

Converts a chunk dictionary to a string format.

chunk_to_string(chunk) = actr.chunk_to_string(chunk)

Python version: actr.chunk_to_string(chunk)

conflict_resolution_function

ACT-R conflict resolution function. Currently selects a production at random from the already matched productions, since utility values and learning are not implemented yet.

conflict_resolution_function(productions) = actr.conflict_resolution_function(productions)

Python version: actr.conflict_resolution_function(productions)

cos

Cosine function

cos(variable0, scale) = scale * cos(variable0)

Python version: scale * numpy.cos(variable0)

cosh

Hyperbolic cosine function

cosh(variable0, scale) = scale * cosh(variable0)

Python version: scale * numpy.cosh(variable0)

drift_diffusion_integrator

Integrates the drift diffusion model for a single trial using and implementation of the using the Euler-Maruyama method. This is a proof of concept implementation and is not optimized for speed.

drift_diffusion_integrator(starting_point, non_decision_time, drift_rate, threshold, noise, dt) = ddm.drift_diffusion_integrator(starting_point,non_decision_time,drift_rate,threshold,noise,dt)

Python version: ddm.drift_diffusion_integrator(starting_point,non_decision_time,drift_rate,threshold,noise,dt)

exponential

Exponential function

exponential(variable0, scale, rate, bias, offset) = scale * exp((rate * variable0) + bias) + offset

Python version: scale * numpy.exp((rate * variable0) + bias) + offset

linear

A linear function, calculated from a slope and an intercept

linear(variable0, slope, intercept) = (variable0 * slope + intercept)

Python version: (variable0 * slope + intercept)

logistic

Logistic function

logistic(variable0, gain, bias, offset) = 1/(1 + exp(-1*gain*(variable0 + bias) + offset))

Python version: 1/(1 + numpy.exp(-1*gain*(variable0 + bias) + offset))

match_production

Returns True if the production's left hand side matches the given context and adds the matching bindings to the production.

match_production(production, context) = actr.match_production(production,context)

Python version: actr.match_production(production,context)

Abs

Absolute takes one input data (Tensor) and produces one output data (Tensor) where the absolute is, y = abs(x), is applied to the tensor elementwise.

Python version: onnx_ops.abs(X)

Acos

Calculates the arccosine (inverse of cosine) of the given input tensor, element-wise.

Python version: onnx_ops.anumpy.cos(input)

Acosh

Calculates the hyperbolic arccosine of the given input tensor element-wise.

Python version: onnx_ops.anumpy.cosh(input)

Add

Performs element-wise binary addition (with Numpy-style broadcasting support).

This operator supports multidirectional (i.e., Numpy-style) broadcasting; for more details please check the doc.

(Opset 14 change): Extend supported types to include uint8, int8, uint16, and int16.

Python version: onnx_ops.add(A, B)

And

Returns the tensor resulted from performing the `and` logical operation elementwise on the input tensors `A` and `B` (with Numpy-style broadcasting support).

This operator supports multidirectional (i.e., Numpy-style) broadcasting; for more details please check the doc.

Python version: onnx_ops.and(A, B)

ArgMax

Computes the indices of the max elements of the input tensor's element along the provided axis. The resulting tensor has the same rank as the input if keepdims equals 1. If keepdims equals 0, then the resulting tensor has the reduced dimension pruned. If select_last_index is True (default False), the index of the last occurrence of the max is selected if the max appears more than once in the input. Otherwise the index of the first occurrence is selected. The type of the output tensor is integer.

Python version: onnx_ops.argmax(data, axis, keepdims, select_last_index)

ArgMin

Computes the indices of the min elements of the input tensor's element along the provided axis. The resulting tensor has the same rank as the input if keepdims equals 1. If keepdims equals 0, then the resulting tensor has the reduced dimension pruned. If select_last_index is True (default False), the index of the last occurrence of the min is selected if the min appears more than once in the input. Otherwise the index of the first occurrence is selected. The type of the output tensor is integer.

Python version: onnx_ops.argmin(data, axis, keepdims, select_last_index)

Asin

Calculates the arcsine (inverse of sine) of the given input tensor, element-wise.

Python version: onnx_ops.anumpy.sin(input)

Asinh

Calculates the hyperbolic arcsine of the given input tensor element-wise.

Python version: onnx_ops.anumpy.sinh(input)

Atan

Calculates the arctangent (inverse of tangent) of the given input tensor, element-wise.

Python version: onnx_ops.anumpy.tan(input)

Atanh

Calculates the hyperbolic arctangent of the given input tensor element-wise.

Python version: onnx_ops.anumpy.tanh(input)

AveragePool

AveragePool consumes an input tensor X and applies average pooling across the tensor according to kernel sizes, stride sizes, and pad lengths. average pooling consisting of computing the average on all values of a subset of the input tensor according to the kernel size and downsampling the data into the output tensor Y for further processing. The output spatial shape will be following: ``` output_spatial_shape[i] = floor((input_spatial_shape[i] + pad_shape[i] - ((kernel_spatial_shape[i] - 1) * dilations[i] + 1)) / strides_spatial_shape[i] + 1) ``` or ``` output_spatial_shape[i] = ceil((input_spatial_shape[i] + pad_shape[i] - ((kernel_spatial_shape[i] - 1) * dilations[i] + 1)) / strides_spatial_shape[i] + 1) ``` if ceil_mode is enabled

* pad_shape[i] is sum of pads along axis i

auto_pad is a DEPRECATED attribute. If you are using them currently, the output spatial shape will be following:

VALID: output_spatial_shape[i] = ceil((input_spatial_shape[i] - ((kernel_spatial_shape[i] - 1) * dilations[i] + 1) + 1) / strides_spatial_shape[i]) SAME_UPPER or SAME_LOWER: output_spatial_shape[i] = ceil(input_spatial_shape[i] / strides_spatial_shape[i])

And pad shape will be following if SAME_UPPER or SAME_LOWER:

pad_shape[i] = (output_spatial_shape[i] - 1) * strides_spatial_shape[i] + ((kernel_spatial_shape[i] - 1) * dilations[i] + 1) - input_spatial_shape[i]

The output of each pooling window is divided by the number of elements (exclude pad when attribute count_include_pad is zero).

Python version: onnx_ops.averagepool(X, auto_pad, ceil_mode, count_include_pad, kernel_shape, pads, strides)

BatchNormalization

Carries out batch normalization as described in the paper https://arxiv.org/abs/1502.03167. Depending on the mode it is being run, There are five required inputs 'X', 'scale', 'B', 'input_mean' and 'input_var'. Note that 'input_mean' and 'input_var' are expected to be the estimated statistics in inference mode (training_mode=False, default), and the running statistics in training mode (training_mode=True). There are multiple cases for the number of outputs, which we list below:

Output case #1: Y, running_mean, running_var (training_mode=True)

Output case #2: Y (training_mode=False)

When training_mode=False, extra outputs are invalid. The outputs are updated as follows when training_mode=True:

running_mean = input_mean * momentum + current_mean * (1 - momentum) running_var = input_var * momentum + current_var * (1 - momentum) Y = (X - current_mean) / sqrt(current_var + epsilon) * scale + B

where:

current_mean = ReduceMean(X, axis=all_except_channel_index) current_var = ReduceVar(X, axis=all_except_channel_index)

Notice that ReduceVar refers to the population variance, and it equals to sum(sqrd(x_i - x_avg)) / N where N is the population size (this formula does not use sample size N - 1).

The computation of ReduceMean and ReduceVar uses float to avoid overflow for float16 inputs.

When training_mode=False:

Y = (X - input_mean) / sqrt(input_var + epsilon) * scale + B

For previous (depreciated) non-spatial cases, implementors are suggested to flatten the input shape to (N x C * D1 * D2 * … * Dn) before a BatchNormalization Op. This operator has optional inputs/outputs. See the doc for more details about the representation of optional arguments. An empty string may be used in the place of an actual argument’s name to indicate a missing argument. Trailing optional arguments (those not followed by an argument that is present) may also be simply omitted.

Python version: onnx_ops.batchnormalization(X, scale, B, input_mean, input_var, epsilon, momentum, training_mode)

Bernoulli

Draws binary random numbers (0 or 1) from a Bernoulli distribution. The input tensor should be a tensor containing probabilities p (a value in the range [0,1]) to be used for drawing the binary random number, where an output of 1 is produced with probability p and an output of 0 is produced with probability (1-p).

This operator is non-deterministic and may not produce the same values in different implementations (even if a seed is specified).

Python version: onnx_ops.bernoulli(input, dtype, seed)

BitShift

Bitwise shift operator performs element-wise operation. For each input element, if the attribute "direction" is "RIGHT", this operator moves its binary representation toward the right side so that the input value is effectively decreased. If the attribute "direction" is "LEFT", bits of binary representation moves toward the left side, which results the increase of its actual value. The input X is the tensor to be shifted and another input Y specifies the amounts of shifting. For example, if "direction" is "Right", X is [1, 4], and S is [1, 1], the corresponding output Z would be [0, 2]. If "direction" is "LEFT" with X=[1, 2] and S=[1, 2], the corresponding output Y would be [2, 8].

Because this operator supports Numpy-style broadcasting, X’s and Y’s shapes are not necessarily identical. This operator supports multidirectional (i.e., Numpy-style) broadcasting; for more details please check the doc.

Python version: onnx_ops.bitshift(X, Y, direction)

Cast

The operator casts the elements of a given input tensor to a data type specified by the 'to' argument and returns an output tensor of the same size in the converted type. The 'to' argument must be one of the data types specified in the 'DataType' enum field in the TensorProto message.

Casting from string tensor in plain (e.g., “3.14” and “1000”) and scientific numeric representations (e.g., “1e-5” and “1E8”) to float types is supported. For example, converting string “100.5” to an integer may yield result 100. There are some string literals reserved for special floating-point values; “+INF” (and “INF”), “-INF”, and “NaN” are positive infinity, negative infinity, and not-a-number, respectively. Any string which can exactly match “+INF” in a case-insensitive way would be mapped to positive infinite. Similarly, this case-insensitive rule is applied to “INF” and “NaN”. When casting from numeric tensors to string tensors, plain floating-point representation (such as “314.15926”) would be used. Converting non-numerical-literal string such as “Hello World!” is an undefined behavior. Cases of converting string representing floating-point arithmetic value, such as “2.718”, to INT is an undefined behavior.

Conversion from a numerical type to any numerical type is always allowed. User must be aware of precision loss and value change caused by range difference between two types. For example, a 64-bit float 3.1415926459 may be round to a 32-bit float 3.141592. Similarly, converting an integer 36 to Boolean may produce 1 because we truncate bits which can’t be stored in the targeted type.

In more detail, the conversion among numerical types should follow these rules:

Casting from floating point to:

floating point: +/- infinity if OOR (out of range).

fixed point: undefined if OOR.

bool: +/- 0.0 to False; all else to True.

Casting from fixed point to:

floating point: +/- infinity if OOR. (+ infinity in the case of uint)

fixed point: when OOR, discard higher bits and reinterpret (with respect to two’s complement representation for signed types). For example, 200 (int16) -> -56 (int8).

bool: zero to False; nonzero to True.

Casting from bool to:

floating point: {1.0, 0.0}.

fixed point: {1, 0}.

bool: no change.

Python version: onnx_ops.cast(input, to)

CastLike

The operator casts the elements of a given input tensor (the first input) to the same data type as the elements of the second input tensor. See documentation of the Cast operator for further details.

Python version: onnx_ops.castlike(input, target_type)

Ceil

Ceil takes one input data (Tensor) and produces one output data (Tensor) where the ceil is, y = ceil(x), is applied to the tensor elementwise. If x is integral, +0, -0, NaN, or infinite, x itself is returned.

Python version: onnx_ops.ceil(X)

Celu

Continuously Differentiable Exponential Linear Units: Perform the linear unit element-wise on the input tensor X using formula:

max(0,x) + min(0,alpha*(exp(x/alpha)-1))

Python version: onnx_ops.celu(X, alpha)

Clip

Clip operator limits the given input within an interval. The interval is specified by the inputs 'min' and 'max'. They default to numeric_limits::lowest() and numeric_limits::max(), respectively.

Python version: onnx_ops.clip(input, min, max)

Compress

Selects slices from an input tensor along a given axis where condition evaluates to True for each axis index. In case axis is not provided, input is flattened before elements are selected. Compress behaves like numpy.compress: https://docs.scipy.org/doc/numpy/reference/generated/numpy.compress.html

Python version: onnx_ops.compress(input, condition, axis)

Concat

Concatenate a list of tensors into a single tensor. All input tensors must have the same shape, except for the dimension size of the axis to concatenate on.

Python version: onnx_ops.concat(inputs, axis)

ConcatFromSequence

Concatenate a sequence of tensors into a single tensor. All input tensors must have the same shape, except for the dimension size of the axis to concatenate on. By default 'new_axis' is 0, the behavior is similar to numpy.concatenate. When 'new_axis' is 1, the behavior is similar to numpy.stack.

Python version: onnx_ops.concatfromsequence(input_sequence, axis, new_axis)

Constant

This operator produces a constant tensor. Exactly one of the provided attributes, either value, sparse_value, or value_* must be specified.

Python version: onnx_ops.constant(sparse_value, value, value_float, value_floats, value_int, value_ints, value_string, value_strings)

ConstantOfShape

Generate a tensor with given value and shape.

Python version: onnx_ops.constantofshape(input, value)

Conv

The convolution operator consumes an input tensor and a filter, and computes the output.

Python version: onnx_ops.conv(X, W, B, auto_pad, dilations, group, kernel_shape, pads, strides)

ConvInteger

The integer convolution operator consumes an input tensor, its zero-point, a filter, and its zero-point, and computes the output. The production MUST never overflow. The accumulation may overflow if and only if in 32 bits.

Python version: onnx_ops.convinteger(x, w, x_zero_point, w_zero_point, auto_pad, dilations, group, kernel_shape, pads, strides)