Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use resources #59

Merged
merged 1 commit into from
Nov 9, 2023
Merged

Use resources #59

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
188 changes: 108 additions & 80 deletions ml.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ Then, the user passes <em>tensor</em> inputs to the <em>graph</em>, computes the
<ul>
<li>interface <a href="#wasi:nn_tensor"><code>wasi:nn/tensor</code></a></li>
<li>interface <a href="#wasi:nn_errors"><code>wasi:nn/errors</code></a></li>
<li>interface <a href="#wasi:nn_graph"><code>wasi:nn/graph</code></a></li>
<li>interface <a href="#wasi:nn_inference"><code>wasi:nn/inference</code></a></li>
<li>interface <a href="#wasi:nn_graph"><code>wasi:nn/graph</code></a></li>
</ul>
</li>
</ul>
Expand Down Expand Up @@ -45,12 +45,46 @@ and the array length must match the product of all of the dimensions and the num
in the type (e.g., a 2x2 tensor with 4-byte f32 elements would have a data array of length
16). Naturally, this representation requires some knowledge of how to lay out data in
memory--e.g., using row-major ordering--and could perhaps be improved.</p>
<h4><a name="tensor"><code>record tensor</code></a></h4>
<h5>Record Fields</h5>
<h4><a name="tensor"><code>resource tensor</code></a></h4>
<hr />
<h3>Functions</h3>
<h4><a name="constructor_tensor"><code>[constructor]tensor: func</code></a></h4>
<h5>Params</h5>
<ul>
<li><a name="constructor_tensor.dimensions"><code>dimensions</code></a>: <a href="#tensor_dimensions"><a href="#tensor_dimensions"><code>tensor-dimensions</code></a></a></li>
<li><a name="constructor_tensor.ty"><code>ty</code></a>: <a href="#tensor_type"><a href="#tensor_type"><code>tensor-type</code></a></a></li>
<li><a name="constructor_tensor.data"><code>data</code></a>: <a href="#tensor_data"><a href="#tensor_data"><code>tensor-data</code></a></a></li>
</ul>
<h5>Return values</h5>
<ul>
<li><a name="tensor.dimensions"><code>dimensions</code></a>: <a href="#tensor_dimensions"><a href="#tensor_dimensions"><code>tensor-dimensions</code></a></a></li>
<li><a name="tensor.tensor_type"><a href="#tensor_type"><code>tensor-type</code></a></a>: <a href="#tensor_type"><a href="#tensor_type"><code>tensor-type</code></a></a></li>
<li><a name="tensor.data"><code>data</code></a>: <a href="#tensor_data"><a href="#tensor_data"><code>tensor-data</code></a></a></li>
<li><a name="constructor_tensor.0"></a> own&lt;<a href="#tensor"><a href="#tensor"><code>tensor</code></a></a>&gt;</li>
</ul>
<h4><a name="method_tensor.dimensions"><code>[method]tensor.dimensions: func</code></a></h4>
<h5>Params</h5>
<ul>
<li><a name="method_tensor.dimensions.self"><code>self</code></a>: borrow&lt;<a href="#tensor"><a href="#tensor"><code>tensor</code></a></a>&gt;</li>
</ul>
<h5>Return values</h5>
<ul>
<li><a name="method_tensor.dimensions.0"></a> <a href="#tensor_dimensions"><a href="#tensor_dimensions"><code>tensor-dimensions</code></a></a></li>
</ul>
<h4><a name="method_tensor.ty"><code>[method]tensor.ty: func</code></a></h4>
<h5>Params</h5>
<ul>
<li><a name="method_tensor.ty.self"><code>self</code></a>: borrow&lt;<a href="#tensor"><a href="#tensor"><code>tensor</code></a></a>&gt;</li>
</ul>
<h5>Return values</h5>
<ul>
<li><a name="method_tensor.ty.0"></a> <a href="#tensor_type"><a href="#tensor_type"><code>tensor-type</code></a></a></li>
</ul>
<h4><a name="method_tensor.data"><code>[method]tensor.data: func</code></a></h4>
<h5>Params</h5>
<ul>
<li><a name="method_tensor.data.self"><code>self</code></a>: borrow&lt;<a href="#tensor"><a href="#tensor"><code>tensor</code></a></a>&gt;</li>
</ul>
<h5>Return values</h5>
<ul>
<li><a name="method_tensor.data.0"></a> <a href="#tensor_data"><a href="#tensor_data"><code>tensor-data</code></a></a></li>
</ul>
<h2><a name="wasi:nn_errors">Import interface wasi:nn/errors</a></h2>
<p>TODO: create function-specific errors (https://github.com/WebAssembly/wasi-nn/issues/42)</p>
Expand All @@ -67,6 +101,59 @@ memory--e.g., using row-major ordering--and could perhaps be improved.</p>
<li><a name="error.too_large"><code>too-large</code></a></li>
<li><a name="error.not_found"><code>not-found</code></a></li>
</ul>
<h2><a name="wasi:nn_inference">Import interface wasi:nn/inference</a></h2>
<p>An inference &quot;session&quot; is encapsulated by a <a href="#graph_execution_context"><code>graph-execution-context</code></a>. This structure binds a
<a href="#graph"><code>graph</code></a> to input tensors before <code>compute</code>-ing an inference:</p>
<hr />
<h3>Types</h3>
<h4><a name="error"><code>type error</code></a></h4>
<p><a href="#error"><a href="#error"><code>error</code></a></a></p>
<p>
#### <a name="tensor">`type tensor`</a>
[`tensor`](#tensor)
<p>
#### <a name="tensor_data">`type tensor-data`</a>
[`tensor-data`](#tensor_data)
<p>
#### <a name="graph_execution_context">`resource graph-execution-context`</a>
<hr />
<h3>Functions</h3>
<h4><a name="method_graph_execution_context.set_input"><code>[method]graph-execution-context.set-input: func</code></a></h4>
<p>Define the inputs to use for inference.</p>
<h5>Params</h5>
<ul>
<li><a name="method_graph_execution_context.set_input.self"><code>self</code></a>: borrow&lt;<a href="#graph_execution_context"><a href="#graph_execution_context"><code>graph-execution-context</code></a></a>&gt;</li>
<li><a name="method_graph_execution_context.set_input.name"><code>name</code></a>: <code>string</code></li>
<li><a name="method_graph_execution_context.set_input.tensor"><a href="#tensor"><code>tensor</code></a></a>: own&lt;<a href="#tensor"><a href="#tensor"><code>tensor</code></a></a>&gt;</li>
</ul>
<h5>Return values</h5>
<ul>
<li><a name="method_graph_execution_context.set_input.0"></a> result&lt;_, <a href="#error"><a href="#error"><code>error</code></a></a>&gt;</li>
</ul>
<h4><a name="method_graph_execution_context.compute"><code>[method]graph-execution-context.compute: func</code></a></h4>
<p>Compute the inference on the given inputs.</p>
<p>Note the expected sequence of calls: <code>set-input</code>, <code>compute</code>, <code>get-output</code>. TODO: this
expectation could be removed as a part of
https://github.com/WebAssembly/wasi-nn/issues/43.</p>
<h5>Params</h5>
<ul>
<li><a name="method_graph_execution_context.compute.self"><code>self</code></a>: borrow&lt;<a href="#graph_execution_context"><a href="#graph_execution_context"><code>graph-execution-context</code></a></a>&gt;</li>
</ul>
<h5>Return values</h5>
<ul>
<li><a name="method_graph_execution_context.compute.0"></a> result&lt;_, <a href="#error"><a href="#error"><code>error</code></a></a>&gt;</li>
</ul>
<h4><a name="method_graph_execution_context.get_output"><code>[method]graph-execution-context.get-output: func</code></a></h4>
<p>Extract the outputs after inference.</p>
<h5>Params</h5>
<ul>
<li><a name="method_graph_execution_context.get_output.self"><code>self</code></a>: borrow&lt;<a href="#graph_execution_context"><a href="#graph_execution_context"><code>graph-execution-context</code></a></a>&gt;</li>
<li><a name="method_graph_execution_context.get_output.name"><code>name</code></a>: <code>string</code></li>
</ul>
<h5>Return values</h5>
<ul>
<li><a name="method_graph_execution_context.get_output.0"></a> result&lt;own&lt;<a href="#tensor"><a href="#tensor"><code>tensor</code></a></a>&gt;, <a href="#error"><a href="#error"><code>error</code></a></a>&gt;</li>
</ul>
<h2><a name="wasi:nn_graph">Import interface wasi:nn/graph</a></h2>
<p>A <a href="#graph"><code>graph</code></a> is a loaded instance of a specific ML model (e.g., MobileNet) for a specific ML
framework (e.g., TensorFlow):</p>
Expand All @@ -78,10 +165,10 @@ framework (e.g., TensorFlow):</p>
#### <a name="tensor">`type tensor`</a>
[`tensor`](#tensor)
<p>
#### <a name="graph">`type graph`</a>
`u32`
<p>An execution graph for performing inference (i.e., a model).
<p>TODO: replace with <code>resource</code> (https://github.com/WebAssembly/wasi-nn/issues/47).</p>
#### <a name="graph_execution_context">`type graph-execution-context`</a>
[`graph-execution-context`](#graph_execution_context)
<p>
#### <a name="graph">`resource graph`</a>
<h4><a name="graph_encoding"><code>enum graph-encoding</code></a></h4>
<p>Describes the encoding of the graph. This allows the API to be implemented by various
backends that encode (i.e., serialize) their graph IR with different formats.</p>
Expand Down Expand Up @@ -109,6 +196,15 @@ backends that encode (i.e., serialize) their graph IR with different formats.</p
graph IR in parts (e.g., OpenVINO stores its IR and weights separately).</p>
<hr />
<h3>Functions</h3>
<h4><a name="method_graph.init_execution_context"><code>[method]graph.init-execution-context: func</code></a></h4>
<h5>Params</h5>
<ul>
<li><a name="method_graph.init_execution_context.self"><code>self</code></a>: borrow&lt;<a href="#graph"><a href="#graph"><code>graph</code></a></a>&gt;</li>
</ul>
<h5>Return values</h5>
<ul>
<li><a name="method_graph.init_execution_context.0"></a> result&lt;own&lt;<a href="#graph_execution_context"><a href="#graph_execution_context"><code>graph-execution-context</code></a></a>&gt;, <a href="#error"><a href="#error"><code>error</code></a></a>&gt;</li>
</ul>
<h4><a name="load"><code>load: func</code></a></h4>
<p>Load a <a href="#graph"><code>graph</code></a> from an opaque sequence of bytes to use for inference.</p>
<h5>Params</h5>
Expand All @@ -119,7 +215,7 @@ graph IR in parts (e.g., OpenVINO stores its IR and weights separately).</p>
</ul>
<h5>Return values</h5>
<ul>
<li><a name="load.0"></a> result&lt;<a href="#graph"><a href="#graph"><code>graph</code></a></a>, <a href="#error"><a href="#error"><code>error</code></a></a>&gt;</li>
<li><a name="load.0"></a> result&lt;own&lt;<a href="#graph"><a href="#graph"><code>graph</code></a></a>&gt;, <a href="#error"><a href="#error"><code>error</code></a></a>&gt;</li>
</ul>
<h4><a name="load_by_name"><code>load-by-name: func</code></a></h4>
<p>Load a <a href="#graph"><code>graph</code></a> by name.</p>
Expand All @@ -132,73 +228,5 @@ range from simple to complex (e.g., URLs?) and caching mechanisms of various kin
</ul>
<h5>Return values</h5>
<ul>
<li><a name="load_by_name.0"></a> result&lt;<a href="#graph"><a href="#graph"><code>graph</code></a></a>, <a href="#error"><a href="#error"><code>error</code></a></a>&gt;</li>
</ul>
<h2><a name="wasi:nn_inference">Import interface wasi:nn/inference</a></h2>
<p>An inference &quot;session&quot; is encapsulated by a <a href="#graph_execution_context"><code>graph-execution-context</code></a>. This structure binds a
<a href="#graph"><code>graph</code></a> to input tensors before <a href="#compute"><code>compute</code></a>-ing an inference:</p>
<hr />
<h3>Types</h3>
<h4><a name="error"><code>type error</code></a></h4>
<p><a href="#error"><a href="#error"><code>error</code></a></a></p>
<p>
#### <a name="tensor">`type tensor`</a>
[`tensor`](#tensor)
<p>
#### <a name="tensor_data">`type tensor-data`</a>
[`tensor-data`](#tensor_data)
<p>
#### <a name="graph">`type graph`</a>
[`graph`](#graph)
<p>
#### <a name="graph_execution_context">`type graph-execution-context`</a>
`u32`
<p>Bind a `graph` to the input and output tensors for an inference.
<p>TODO: this is no longer necessary in WIT (https://github.com/WebAssembly/wasi-nn/issues/43)</p>
<hr />
<h3>Functions</h3>
<h4><a name="init_execution_context"><code>init-execution-context: func</code></a></h4>
<p>Create an execution instance of a loaded graph.</p>
<h5>Params</h5>
<ul>
<li><a name="init_execution_context.graph"><a href="#graph"><code>graph</code></a></a>: <a href="#graph"><a href="#graph"><code>graph</code></a></a></li>
</ul>
<h5>Return values</h5>
<ul>
<li><a name="init_execution_context.0"></a> result&lt;<a href="#graph_execution_context"><a href="#graph_execution_context"><code>graph-execution-context</code></a></a>, <a href="#error"><a href="#error"><code>error</code></a></a>&gt;</li>
</ul>
<h4><a name="set_input"><code>set-input: func</code></a></h4>
<p>Define the inputs to use for inference.</p>
<h5>Params</h5>
<ul>
<li><a name="set_input.ctx"><code>ctx</code></a>: <a href="#graph_execution_context"><a href="#graph_execution_context"><code>graph-execution-context</code></a></a></li>
<li><a name="set_input.name"><code>name</code></a>: <code>string</code></li>
<li><a name="set_input.tensor"><a href="#tensor"><code>tensor</code></a></a>: <a href="#tensor"><a href="#tensor"><code>tensor</code></a></a></li>
</ul>
<h5>Return values</h5>
<ul>
<li><a name="set_input.0"></a> result&lt;_, <a href="#error"><a href="#error"><code>error</code></a></a>&gt;</li>
</ul>
<h4><a name="compute"><code>compute: func</code></a></h4>
<p>Compute the inference on the given inputs.</p>
<p>Note the expected sequence of calls: <a href="#set_input"><code>set-input</code></a>, <a href="#compute"><code>compute</code></a>, <a href="#get_output"><code>get-output</code></a>. TODO: this
expectation could be removed as a part of https://github.com/WebAssembly/wasi-nn/issues/43.</p>
<h5>Params</h5>
<ul>
<li><a name="compute.ctx"><code>ctx</code></a>: <a href="#graph_execution_context"><a href="#graph_execution_context"><code>graph-execution-context</code></a></a></li>
</ul>
<h5>Return values</h5>
<ul>
<li><a name="compute.0"></a> result&lt;_, <a href="#error"><a href="#error"><code>error</code></a></a>&gt;</li>
</ul>
<h4><a name="get_output"><code>get-output: func</code></a></h4>
<p>Extract the outputs after inference.</p>
<h5>Params</h5>
<ul>
<li><a name="get_output.ctx"><code>ctx</code></a>: <a href="#graph_execution_context"><a href="#graph_execution_context"><code>graph-execution-context</code></a></a></li>
<li><a name="get_output.name"><code>name</code></a>: <code>string</code></li>
</ul>
<h5>Return values</h5>
<ul>
<li><a name="get_output.0"></a> result&lt;<a href="#tensor_data"><a href="#tensor_data"><code>tensor-data</code></a></a>, <a href="#error"><a href="#error"><code>error</code></a></a>&gt;</li>
<li><a name="load_by_name.0"></a> result&lt;own&lt;<a href="#graph"><a href="#graph"><code>graph</code></a></a>&gt;, <a href="#error"><a href="#error"><code>error</code></a></a>&gt;</li>
</ul>
53 changes: 27 additions & 26 deletions wit/wasi-nn.wit
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,16 +43,18 @@ interface tensor {
/// memory--e.g., using row-major ordering--and could perhaps be improved.
type tensor-data = list<u8>;

record tensor {
resource tensor {
constructor(dimensions: tensor-dimensions, ty: tensor-type, data: tensor-data);

// Describe the size of the tensor (e.g., 2x2x2x2 -> [2, 2, 2, 2]). To represent a tensor
// containing a single value, use `[1]` for the tensor dimensions.
dimensions: tensor-dimensions,
dimensions: func() -> tensor-dimensions;

// Describe the type of element in the tensor (e.g., `f32`).
tensor-type: tensor-type,
ty: func() -> tensor-type;

// Contains the tensor data.
data: tensor-data,
// Return the tensor data.
data: func() -> tensor-data;
}
}

Expand All @@ -61,11 +63,12 @@ interface tensor {
interface graph {
use errors.{error};
use tensor.{tensor};
use inference.{graph-execution-context};

/// An execution graph for performing inference (i.e., a model).
///
/// TODO: replace with `resource` (https://github.com/WebAssembly/wasi-nn/issues/47).
type graph = u32;
resource graph {
init-execution-context: func() -> result<graph-execution-context, error>;
}

/// Describes the encoding of the graph. This allows the API to be implemented by various
/// backends that encode (i.e., serialize) their graph IR with different formats.
Expand Down Expand Up @@ -107,27 +110,25 @@ interface graph {
interface inference {
use errors.{error};
use tensor.{tensor, tensor-data};
use graph.{graph};

/// Bind a `graph` to the input and output tensors for an inference.
///
/// TODO: this is no longer necessary in WIT (https://github.com/WebAssembly/wasi-nn/issues/43)
type graph-execution-context = u32;

/// Create an execution instance of a loaded graph.
init-execution-context: func(graph: graph) -> result<graph-execution-context, error>;

/// Define the inputs to use for inference.
set-input: func(ctx: graph-execution-context, name: string, tensor: tensor) -> result<_, error>;

/// Compute the inference on the given inputs.
///
/// Note the expected sequence of calls: `set-input`, `compute`, `get-output`. TODO: this
/// expectation could be removed as a part of https://github.com/WebAssembly/wasi-nn/issues/43.
compute: func(ctx: graph-execution-context) -> result<_, error>;

/// Extract the outputs after inference.
get-output: func(ctx: graph-execution-context, name: string) -> result<tensor-data, error>;
/// TODO: this may no longer be necessary in WIT
/// (https://github.com/WebAssembly/wasi-nn/issues/43)
resource graph-execution-context {
/// Define the inputs to use for inference.
set-input: func(name: string, tensor: tensor) -> result<_, error>;

/// Compute the inference on the given inputs.
///
/// Note the expected sequence of calls: `set-input`, `compute`, `get-output`. TODO: this
/// expectation could be removed as a part of
/// https://github.com/WebAssembly/wasi-nn/issues/43.
compute: func() -> result<_, error>;

/// Extract the outputs after inference.
get-output: func(name: string) -> result<tensor, error>;
}
}

/// TODO: create function-specific errors (https://github.com/WebAssembly/wasi-nn/issues/42)
Expand Down
Loading