Skip to content

Commit

Permalink
Use resources
Browse files Browse the repository at this point in the history 
Now that component model `resource`s are specified and implemented, it
should be possible to use the `resource` type for specifying tensors,
graphs and execution contexts.

Closes WebAssembly#47.
  • Loading branch information
@abrown
abrown committed Nov 9, 2023
1 parent 8bd40e2 commit 5435296
Show file tree
Hide file tree
Showing 2 changed files with 135 additions and 106 deletions.
188 changes: 108 additions & 80 deletions ml.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ Then, the user passes <em>tensor</em> inputs to the <em>graph</em>, computes the
<ul>
<li>interface <a href="#wasi:nn_tensor"><code>wasi:nn/tensor</code></a></li>
<li>interface <a href="#wasi:nn_errors"><code>wasi:nn/errors</code></a></li>
<li>interface <a href="#wasi:nn_graph"><code>wasi:nn/graph</code></a></li>
<li>interface <a href="#wasi:nn_inference"><code>wasi:nn/inference</code></a></li>
<li>interface <a href="#wasi:nn_graph"><code>wasi:nn/graph</code></a></li>
</ul>
</li>
</ul>
Expand Down Expand Up @@ -45,12 +45,46 @@ and the array length must match the product of all of the dimensions and the num
in the type (e.g., a 2x2 tensor with 4-byte f32 elements would have a data array of length
16). Naturally, this representation requires some knowledge of how to lay out data in
memory--e.g., using row-major ordering--and could perhaps be improved.</p>
<h4><a name="tensor"><code>record tensor</code></a></h4>
<h5>Record Fields</h5>
<h4><a name="tensor"><code>resource tensor</code></a></h4>
<hr />
<h3>Functions</h3>
<h4><a name="constructor_tensor"><code>[constructor]tensor: func</code></a></h4>
<h5>Params</h5>
<ul>
<li><a name="constructor_tensor.dimensions"><code>dimensions</code></a>: <a href="#tensor_dimensions"><a href="#tensor_dimensions"><code>tensor-dimensions</code></a></a></li>
<li><a name="constructor_tensor.ty"><code>ty</code></a>: <a href="#tensor_type"><a href="#tensor_type"><code>tensor-type</code></a></a></li>
<li><a name="constructor_tensor.data"><code>data</code></a>: <a href="#tensor_data"><a href="#tensor_data"><code>tensor-data</code></a></a></li>
</ul>
<h5>Return values</h5>
<ul>
<li><a name="tensor.dimensions"><code>dimensions</code></a>: <a href="#tensor_dimensions"><a href="#tensor_dimensions"><code>tensor-dimensions</code></a></a></li>
<li><a name="tensor.tensor_type"><a href="#tensor_type"><code>tensor-type</code></a></a>: <a href="#tensor_type"><a href="#tensor_type"><code>tensor-type</code></a></a></li>
<li><a name="tensor.data"><code>data</code></a>: <a href="#tensor_data"><a href="#tensor_data"><code>tensor-data</code></a></a></li>
<li><a name="constructor_tensor.0"></a> own&lt;<a href="#tensor"><a href="#tensor"><code>tensor</code></a></a>&gt;</li>
</ul>
<h4><a name="method_tensor.dimensions"><code>[method]tensor.dimensions: func</code></a></h4>
<h5>Params</h5>
<ul>
<li><a name="method_tensor.dimensions.self"><code>self</code></a>: borrow&lt;<a href="#tensor"><a href="#tensor"><code>tensor</code></a></a>&gt;</li>
</ul>
<h5>Return values</h5>
<ul>
<li><a name="method_tensor.dimensions.0"></a> <a href="#tensor_dimensions"><a href="#tensor_dimensions"><code>tensor-dimensions</code></a></a></li>
</ul>
<h4><a name="method_tensor.ty"><code>[method]tensor.ty: func</code></a></h4>
<h5>Params</h5>
<ul>
<li><a name="method_tensor.ty.self"><code>self</code></a>: borrow&lt;<a href="#tensor"><a href="#tensor"><code>tensor</code></a></a>&gt;</li>
</ul>
<h5>Return values</h5>
<ul>
<li><a name="method_tensor.ty.0"></a> <a href="#tensor_type"><a href="#tensor_type"><code>tensor-type</code></a></a></li>
</ul>
<h4><a name="method_tensor.data"><code>[method]tensor.data: func</code></a></h4>
<h5>Params</h5>
<ul>
<li><a name="method_tensor.data.self"><code>self</code></a>: borrow&lt;<a href="#tensor"><a href="#tensor"><code>tensor</code></a></a>&gt;</li>
</ul>
<h5>Return values</h5>
<ul>
<li><a name="method_tensor.data.0"></a> <a href="#tensor_data"><a href="#tensor_data"><code>tensor-data</code></a></a></li>
</ul>
<h2><a name="wasi:nn_errors">Import interface wasi:nn/errors</a></h2>
<p>TODO: create function-specific errors (https://github.com/WebAssembly/wasi-nn/issues/42)</p>
Expand All @@ -67,6 +101,59 @@ memory--e.g., using row-major ordering--and could perhaps be improved.</p>
<li><a name="error.too_large"><code>too-large</code></a></li>
<li><a name="error.not_found"><code>not-found</code></a></li>
</ul>
<h2><a name="wasi:nn_inference">Import interface wasi:nn/inference</a></h2>
<p>An inference &quot;session&quot; is encapsulated by a <a href="#graph_execution_context"><code>graph-execution-context</code></a>. This structure binds a
<a href="#graph"><code>graph</code></a> to input tensors before <code>compute</code>-ing an inference:</p>
<hr />
<h3>Types</h3>
<h4><a name="error"><code>type error</code></a></h4>
<p><a href="#error"><a href="#error"><code>error</code></a></a></p>
<p>
#### <a name="tensor">`type tensor`</a>
[`tensor`](#tensor)
<p>
#### <a name="tensor_data">`type tensor-data`</a>
[`tensor-data`](#tensor_data)
<p>
#### <a name="graph_execution_context">`resource graph-execution-context`</a>
<hr />
<h3>Functions</h3>
<h4><a name="method_graph_execution_context.set_input"><code>[method]graph-execution-context.set-input: func</code></a></h4>
<p>Define the inputs to use for inference.</p>
<h5>Params</h5>
<ul>
<li><a name="method_graph_execution_context.set_input.self"><code>self</code></a>: borrow&lt;<a href="#graph_execution_context"><a href="#graph_execution_context"><code>graph-execution-context</code></a></a>&gt;</li>
<li><a name="method_graph_execution_context.set_input.name"><code>name</code></a>: <code>string</code></li>
<li><a name="method_graph_execution_context.set_input.tensor"><a href="#tensor"><code>tensor</code></a></a>: own&lt;<a href="#tensor"><a href="#tensor"><code>tensor</code></a></a>&gt;</li>
</ul>
<h5>Return values</h5>
<ul>
<li><a name="method_graph_execution_context.set_input.0"></a> result&lt;_, <a href="#error"><a href="#error"><code>error</code></a></a>&gt;</li>
</ul>
<h4><a name="method_graph_execution_context.compute"><code>[method]graph-execution-context.compute: func</code></a></h4>
<p>Compute the inference on the given inputs.</p>
<p>Note the expected sequence of calls: <code>set-input</code>, <code>compute</code>, <code>get-output</code>. TODO: this
expectation could be removed as a part of
https://github.com/WebAssembly/wasi-nn/issues/43.</p>
<h5>Params</h5>
<ul>
<li><a name="method_graph_execution_context.compute.self"><code>self</code></a>: borrow&lt;<a href="#graph_execution_context"><a href="#graph_execution_context"><code>graph-execution-context</code></a></a>&gt;</li>
</ul>
<h5>Return values</h5>
<ul>
<li><a name="method_graph_execution_context.compute.0"></a> result&lt;_, <a href="#error"><a href="#error"><code>error</code></a></a>&gt;</li>
</ul>
<h4><a name="method_graph_execution_context.get_output"><code>[method]graph-execution-context.get-output: func</code></a></h4>
<p>Extract the outputs after inference.</p>
<h5>Params</h5>
<ul>
<li><a name="method_graph_execution_context.get_output.self"><code>self</code></a>: borrow&lt;<a href="#graph_execution_context"><a href="#graph_execution_context"><code>graph-execution-context</code></a></a>&gt;</li>
<li><a name="method_graph_execution_context.get_output.name"><code>name</code></a>: <code>string</code></li>
</ul>
<h5>Return values</h5>
<ul>
<li><a name="method_graph_execution_context.get_output.0"></a> result&lt;own&lt;<a href="#tensor"><a href="#tensor"><code>tensor</code></a></a>&gt;, <a href="#error"><a href="#error"><code>error</code></a></a>&gt;</li>
</ul>
<h2><a name="wasi:nn_graph">Import interface wasi:nn/graph</a></h2>
<p>A <a href="#graph"><code>graph</code></a> is a loaded instance of a specific ML model (e.g., MobileNet) for a specific ML
framework (e.g., TensorFlow):</p>
Expand All @@ -78,10 +165,10 @@ framework (e.g., TensorFlow):</p>
#### <a name="tensor">`type tensor`</a>
[`tensor`](#tensor)
<p>
#### <a name="graph">`type graph`</a>
`u32`
<p>An execution graph for performing inference (i.e., a model).
<p>TODO: replace with <code>resource</code> (https://github.com/WebAssembly/wasi-nn/issues/47).</p>
#### <a name="graph_execution_context">`type graph-execution-context`</a>
[`graph-execution-context`](#graph_execution_context)
<p>
#### <a name="graph">`resource graph`</a>
<h4><a name="graph_encoding"><code>enum graph-encoding</code></a></h4>
<p>Describes the encoding of the graph. This allows the API to be implemented by various
backends that encode (i.e., serialize) their graph IR with different formats.</p>
Expand Down Expand Up @@ -109,6 +196,15 @@ backends that encode (i.e., serialize) their graph IR with different formats.</p
graph IR in parts (e.g., OpenVINO stores its IR and weights separately).</p>
<hr />
<h3>Functions</h3>
<h4><a name="method_graph.init_execution_context"><code>[method]graph.init-execution-context: func</code></a></h4>
<h5>Params</h5>
<ul>
<li><a name="method_graph.init_execution_context.self"><code>self</code></a>: borrow&lt;<a href="#graph"><a href="#graph"><code>graph</code></a></a>&gt;</li>
</ul>
<h5>Return values</h5>
<ul>
<li><a name="method_graph.init_execution_context.0"></a> result&lt;own&lt;<a href="#graph_execution_context"><a href="#graph_execution_context"><code>graph-execution-context</code></a></a>&gt;, <a href="#error"><a href="#error"><code>error</code></a></a>&gt;</li>
</ul>
<h4><a name="load"><code>load: func</code></a></h4>
<p>Load a <a href="#graph"><code>graph</code></a> from an opaque sequence of bytes to use for inference.</p>
<h5>Params</h5>
Expand All @@ -119,7 +215,7 @@ graph IR in parts (e.g., OpenVINO stores its IR and weights separately).</p>
</ul>
<h5>Return values</h5>
<ul>
<li><a name="load.0"></a> result&lt;<a href="#graph"><a href="#graph"><code>graph</code></a></a>, <a href="#error"><a href="#error"><code>error</code></a></a>&gt;</li>
<li><a name="load.0"></a> result&lt;own&lt;<a href="#graph"><a href="#graph"><code>graph</code></a></a>&gt;, <a href="#error"><a href="#error"><code>error</code></a></a>&gt;</li>
</ul>
<h4><a name="load_by_name"><code>load-by-name: func</code></a></h4>
<p>Load a <a href="#graph"><code>graph</code></a> by name.</p>
Expand All @@ -132,73 +228,5 @@ range from simple to complex (e.g., URLs?) and caching mechanisms of various kin
</ul>
<h5>Return values</h5>
<ul>
<li><a name="load_by_name.0"></a> result&lt;<a href="#graph"><a href="#graph"><code>graph</code></a></a>, <a href="#error"><a href="#error"><code>error</code></a></a>&gt;</li>
</ul>
<h2><a name="wasi:nn_inference">Import interface wasi:nn/inference</a></h2>
<p>An inference &quot;session&quot; is encapsulated by a <a href="#graph_execution_context"><code>graph-execution-context</code></a>. This structure binds a
<a href="#graph"><code>graph</code></a> to input tensors before <a href="#compute"><code>compute</code></a>-ing an inference:</p>
<hr />
<h3>Types</h3>
<h4><a name="error"><code>type error</code></a></h4>
<p><a href="#error"><a href="#error"><code>error</code></a></a></p>
<p>
#### <a name="tensor">`type tensor`</a>
[`tensor`](#tensor)
<p>
#### <a name="tensor_data">`type tensor-data`</a>
[`tensor-data`](#tensor_data)
<p>
#### <a name="graph">`type graph`</a>
[`graph`](#graph)
<p>
#### <a name="graph_execution_context">`type graph-execution-context`</a>
`u32`
<p>Bind a `graph` to the input and output tensors for an inference.
<p>TODO: this is no longer necessary in WIT (https://github.com/WebAssembly/wasi-nn/issues/43)</p>
<hr />
<h3>Functions</h3>
<h4><a name="init_execution_context"><code>init-execution-context: func</code></a></h4>
<p>Create an execution instance of a loaded graph.</p>
<h5>Params</h5>
<ul>
<li><a name="init_execution_context.graph"><a href="#graph"><code>graph</code></a></a>: <a href="#graph"><a href="#graph"><code>graph</code></a></a></li>
</ul>
<h5>Return values</h5>
<ul>
<li><a name="init_execution_context.0"></a> result&lt;<a href="#graph_execution_context"><a href="#graph_execution_context"><code>graph-execution-context</code></a></a>, <a href="#error"><a href="#error"><code>error</code></a></a>&gt;</li>
</ul>
<h4><a name="set_input"><code>set-input: func</code></a></h4>
<p>Define the inputs to use for inference.</p>
<h5>Params</h5>
<ul>
<li><a name="set_input.ctx"><code>ctx</code></a>: <a href="#graph_execution_context"><a href="#graph_execution_context"><code>graph-execution-context</code></a></a></li>
<li><a name="set_input.name"><code>name</code></a>: <code>string</code></li>
<li><a name="set_input.tensor"><a href="#tensor"><code>tensor</code></a></a>: <a href="#tensor"><a href="#tensor"><code>tensor</code></a></a></li>
</ul>
<h5>Return values</h5>
<ul>
<li><a name="set_input.0"></a> result&lt;_, <a href="#error"><a href="#error"><code>error</code></a></a>&gt;</li>
</ul>
<h4><a name="compute"><code>compute: func</code></a></h4>
<p>Compute the inference on the given inputs.</p>
<p>Note the expected sequence of calls: <a href="#set_input"><code>set-input</code></a>, <a href="#compute"><code>compute</code></a>, <a href="#get_output"><code>get-output</code></a>. TODO: this
expectation could be removed as a part of https://github.com/WebAssembly/wasi-nn/issues/43.</p>
<h5>Params</h5>
<ul>
<li><a name="compute.ctx"><code>ctx</code></a>: <a href="#graph_execution_context"><a href="#graph_execution_context"><code>graph-execution-context</code></a></a></li>
</ul>
<h5>Return values</h5>
<ul>
<li><a name="compute.0"></a> result&lt;_, <a href="#error"><a href="#error"><code>error</code></a></a>&gt;</li>
</ul>
<h4><a name="get_output"><code>get-output: func</code></a></h4>
<p>Extract the outputs after inference.</p>
<h5>Params</h5>
<ul>
<li><a name="get_output.ctx"><code>ctx</code></a>: <a href="#graph_execution_context"><a href="#graph_execution_context"><code>graph-execution-context</code></a></a></li>
<li><a name="get_output.name"><code>name</code></a>: <code>string</code></li>
</ul>
<h5>Return values</h5>
<ul>
<li><a name="get_output.0"></a> result&lt;<a href="#tensor_data"><a href="#tensor_data"><code>tensor-data</code></a></a>, <a href="#error"><a href="#error"><code>error</code></a></a>&gt;</li>
<li><a name="load_by_name.0"></a> result&lt;own&lt;<a href="#graph"><a href="#graph"><code>graph</code></a></a>&gt;, <a href="#error"><a href="#error"><code>error</code></a></a>&gt;</li>
</ul>
53 changes: 27 additions & 26 deletions wit/wasi-nn.wit
View file
Original file line number Diff line number Diff line change
Expand Up @@ -43,16 +43,18 @@ interface tensor {
/// memory--e.g., using row-major ordering--and could perhaps be improved.
type tensor-data = list<u8>;

record tensor {
resource tensor {
constructor(dimensions: tensor-dimensions, ty: tensor-type, data: tensor-data);

// Describe the size of the tensor (e.g., 2x2x2x2 -> [2, 2, 2, 2]). To represent a tensor
// containing a single value, use `[1]` for the tensor dimensions.
dimensions: tensor-dimensions,
dimensions: func() -> tensor-dimensions;

// Describe the type of element in the tensor (e.g., `f32`).
tensor-type: tensor-type,
ty: func() -> tensor-type;

// Contains the tensor data.
data: tensor-data,
// Return the tensor data.
data: func() -> tensor-data;
}
}

Expand All @@ -61,11 +63,12 @@ interface tensor {
interface graph {
use errors.{error};
use tensor.{tensor};
use inference.{graph-execution-context};

/// An execution graph for performing inference (i.e., a model).
///
/// TODO: replace with `resource` (https://github.com/WebAssembly/wasi-nn/issues/47).
type graph = u32;
resource graph {
init-execution-context: func() -> result<graph-execution-context, error>;
}

/// Describes the encoding of the graph. This allows the API to be implemented by various
/// backends that encode (i.e., serialize) their graph IR with different formats.
Expand Down Expand Up @@ -107,27 +110,25 @@ interface graph {
interface inference {
use errors.{error};
use tensor.{tensor, tensor-data};
use graph.{graph};

/// Bind a `graph` to the input and output tensors for an inference.
///
/// TODO: this is no longer necessary in WIT (https://github.com/WebAssembly/wasi-nn/issues/43)
type graph-execution-context = u32;

/// Create an execution instance of a loaded graph.
init-execution-context: func(graph: graph) -> result<graph-execution-context, error>;

/// Define the inputs to use for inference.
set-input: func(ctx: graph-execution-context, name: string, tensor: tensor) -> result<_, error>;

/// Compute the inference on the given inputs.
///
/// Note the expected sequence of calls: `set-input`, `compute`, `get-output`. TODO: this
/// expectation could be removed as a part of https://github.com/WebAssembly/wasi-nn/issues/43.
compute: func(ctx: graph-execution-context) -> result<_, error>;

/// Extract the outputs after inference.
get-output: func(ctx: graph-execution-context, name: string) -> result<tensor-data, error>;
/// TODO: this may no longer be necessary in WIT
/// (https://github.com/WebAssembly/wasi-nn/issues/43)
resource graph-execution-context {
/// Define the inputs to use for inference.
set-input: func(name: string, tensor: tensor) -> result<_, error>;

/// Compute the inference on the given inputs.
///
/// Note the expected sequence of calls: `set-input`, `compute`, `get-output`. TODO: this
/// expectation could be removed as a part of
/// https://github.com/WebAssembly/wasi-nn/issues/43.
compute: func() -> result<_, error>;

/// Extract the outputs after inference.
get-output: func(name: string) -> result<tensor, error>;
}
}

/// TODO: create function-specific errors (https://github.com/WebAssembly/wasi-nn/issues/42)
Expand Down

0 comments on commit 5435296

Please sign in to comment.