Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

let Encoder and Decoder accept named params as encode() and decode() do #224

Merged
merged 5 commits into from
Mar 10, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,9 @@ https://github.com/msgpack/msgpack-javascript/compare/v2.8.0...v3.0.0-beta1
* Add an option `useBigInt64` to map JavaScript's BigInt to MessagePack's int64 and uint64 ([#223](https://github.com/msgpack/msgpack-javascript/pull/223))
* Drop IE11 support ([#221](https://github.com/msgpack/msgpack-javascript/pull/221))
* It also fixes [feature request: option to disable TEXT_ENCODING env check #219](https://github.com/msgpack/msgpack-javascript/issues/219)
*
* Change the interfaces of `Encoder` and `Decoder`, and describe the interfaces in README.md ([#224](https://github.com/msgpack/msgpack-javascript/pull/224)):
* `new Encoder(options: EncoderOptions)`: it takes the same named-options as `encode()`
* `new Decoder(options: DecoderOptions)`: it takes the same named-options as `decode()`

## 2.8.0 2022-09-02

Expand Down
40 changes: 20 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,14 +38,14 @@ deepStrictEqual(decode(encoded), object);
- [Table of Contents](#table-of-contents)
- [Install](#install)
- [API](#api)
- [`encode(data: unknown, options?: EncodeOptions): Uint8Array`](#encodedata-unknown-options-encodeoptions-uint8array)
- [`EncodeOptions`](#encodeoptions)
- [`decode(buffer: ArrayLike<number> | BufferSource, options?: DecodeOptions): unknown`](#decodebuffer-arraylikenumber--buffersource-options-decodeoptions-unknown)
- [`DecodeOptions`](#decodeoptions)
- [`decodeMulti(buffer: ArrayLike<number> | BufferSource, options?: DecodeOptions): Generator<unknown, void, unknown>`](#decodemultibuffer-arraylikenumber--buffersource-options-decodeoptions-generatorunknown-void-unknown)
- [`decodeAsync(stream: ReadableStreamLike<ArrayLike<number> | BufferSource>, options?: DecodeAsyncOptions): Promise<unknown>`](#decodeasyncstream-readablestreamlikearraylikenumber--buffersource-options-decodeasyncoptions-promiseunknown)
- [`decodeArrayStream(stream: ReadableStreamLike<ArrayLike<number> | BufferSource>, options?: DecodeAsyncOptions): AsyncIterable<unknown>`](#decodearraystreamstream-readablestreamlikearraylikenumber--buffersource-options-decodeasyncoptions-asynciterableunknown)
- [`decodeMultiStream(stream: ReadableStreamLike<ArrayLike<number> | BufferSource>, options?: DecodeAsyncOptions): AsyncIterable<unknown>`](#decodemultistreamstream-readablestreamlikearraylikenumber--buffersource-options-decodeasyncoptions-asynciterableunknown)
- [`encode(data: unknown, options?: EncoderOptions): Uint8Array`](#encodedata-unknown-options-encoderoptions-uint8array)
- [`EncoderOptions`](#encoderoptions)
- [`decode(buffer: ArrayLike<number> | BufferSource, options?: DecoderOptions): unknown`](#decodebuffer-arraylikenumber--buffersource-options-decoderoptions-unknown)
- [`DecoderOptions`](#decoderoptions)
- [`decodeMulti(buffer: ArrayLike<number> | BufferSource, options?: DecoderOptions): Generator<unknown, void, unknown>`](#decodemultibuffer-arraylikenumber--buffersource-options-decoderoptions-generatorunknown-void-unknown)
- [`decodeAsync(stream: ReadableStreamLike<ArrayLike<number> | BufferSource>, options?: DecoderOptions): Promise<unknown>`](#decodeasyncstream-readablestreamlikearraylikenumber--buffersource-options-decoderoptions-promiseunknown)
- [`decodeArrayStream(stream: ReadableStreamLike<ArrayLike<number> | BufferSource>, options?: DecoderOptions): AsyncIterable<unknown>`](#decodearraystreamstream-readablestreamlikearraylikenumber--buffersource-options-decoderoptions-asynciterableunknown)
- [`decodeMultiStream(stream: ReadableStreamLike<ArrayLike<number> | BufferSource>, options?: DecoderOptions): AsyncIterable<unknown>`](#decodemultistreamstream-readablestreamlikearraylikenumber--buffersource-options-decoderoptions-asynciterableunknown)
- [Reusing Encoder and Decoder instances](#reusing-encoder-and-decoder-instances)
- [Extension Types](#extension-types)
- [ExtensionCodec context](#extensioncodec-context)
Expand Down Expand Up @@ -80,7 +80,7 @@ npm install @msgpack/msgpack

## API

### `encode(data: unknown, options?: EncodeOptions): Uint8Array`
### `encode(data: unknown, options?: EncoderOptions): Uint8Array`

It encodes `data` into a single MessagePack-encoded object, and returns a byte array as `Uint8Array`. It throws errors if `data` is, or includes, a non-serializable object such as a `function` or a `symbol`.

Expand All @@ -105,7 +105,7 @@ const buffer: Buffer = Buffer.from(encoded.buffer, encoded.byteOffset, encoded.b
console.log(buffer);
```

#### `EncodeOptions`
#### `EncoderOptions`

Name|Type|Default
----|----|----
Expand All @@ -118,7 +118,7 @@ forceIntegerToFloat | boolean | false
ignoreUndefined | boolean | false
context | user-defined | -

### `decode(buffer: ArrayLike<number> | BufferSource, options?: DecodeOptions): unknown`
### `decode(buffer: ArrayLike<number> | BufferSource, options?: DecoderOptions): unknown`

It decodes `buffer` that includes a MessagePack-encoded object, and returns the decoded object typed `unknown`.

Expand All @@ -138,7 +138,7 @@ console.log(object);

NodeJS `Buffer` is also acceptable because it is a subclass of `Uint8Array`.

#### `DecodeOptions`
#### `DecoderOptions`

Name|Type|Default
----|----|----
Expand All @@ -152,7 +152,7 @@ context | user-defined | -

You can use `max${Type}Length` to limit the length of each type decoded.

### `decodeMulti(buffer: ArrayLike<number> | BufferSource, options?: DecodeOptions): Generator<unknown, void, unknown>`
### `decodeMulti(buffer: ArrayLike<number> | BufferSource, options?: DecoderOptions): Generator<unknown, void, unknown>`

It decodes `buffer` that includes multiple MessagePack-encoded objects, and returns decoded objects as a generator. See also `decodeMultiStream()`, which is an asynchronous variant of this function.

Expand All @@ -170,14 +170,12 @@ for (const object of decodeMulti(encoded)) {
}
```

### `decodeAsync(stream: ReadableStreamLike<ArrayLike<number> | BufferSource>, options?: DecodeAsyncOptions): Promise<unknown>`
### `decodeAsync(stream: ReadableStreamLike<ArrayLike<number> | BufferSource>, options?: DecoderOptions): Promise<unknown>`

It decodes `stream`, where `ReadableStreamLike<T>` is defined as `ReadableStream<T> | AsyncIterable<T>`, in an async iterable of byte arrays, and returns decoded object as `unknown` type, wrapped in `Promise`.

This function works asynchronously, and might CPU resources more efficiently compared with synchronous `decode()`, because it doesn't wait for the completion of downloading.

`DecodeAsyncOptions` is the same as `DecodeOptions` for `decode()`.

This function is designed to work with whatwg `fetch()` like this:

```typescript
Expand All @@ -193,7 +191,7 @@ if (contentType && contentType.startsWith(MSGPACK_TYPE) && response.body != null
} else { /* handle errors */ }
```

### `decodeArrayStream(stream: ReadableStreamLike<ArrayLike<number> | BufferSource>, options?: DecodeAsyncOptions): AsyncIterable<unknown>`
### `decodeArrayStream(stream: ReadableStreamLike<ArrayLike<number> | BufferSource>, options?: DecoderOptions): AsyncIterable<unknown>`

It is alike to `decodeAsync()`, but only accepts a `stream` that includes an array of items, and emits a decoded item one by one.

Expand All @@ -210,7 +208,7 @@ for await (const item of decodeArrayStream(stream)) {
}
```

### `decodeMultiStream(stream: ReadableStreamLike<ArrayLike<number> | BufferSource>, options?: DecodeAsyncOptions): AsyncIterable<unknown>`
### `decodeMultiStream(stream: ReadableStreamLike<ArrayLike<number> | BufferSource>, options?: DecoderOptions): AsyncIterable<unknown>`

It is alike to `decodeAsync()` and `decodeArrayStream()`, but the input `stream` must consist of multiple MessagePack-encoded items. This is an asynchronous variant for `decodeMulti()`.

Expand All @@ -233,7 +231,7 @@ This function is available since v2.4.0; previously it was called as `decodeStre

### Reusing Encoder and Decoder instances

`Encoder` and `Decoder` classes is provided to have better performance by reusing instances:
`Encoder` and `Decoder` classes are provided to have better performance by reusing instances:

```typescript
import { deepStrictEqual } from "assert";
Expand All @@ -251,6 +249,8 @@ than `encode()` function, and reusing `Decoder` instance is about 2% faster
than `decode()` function. Note that the result should vary in environments
and data structure.

`Encoder` and `Decoder` take the same options as `encode()` and `decode()` respectively.

## Extension Types

To handle [MessagePack Extension Types](https://github.com/msgpack/msgpack/blob/master/spec.md#extension-types), this library provides `ExtensionCodec` class.
Expand Down Expand Up @@ -304,7 +304,7 @@ Not that extension types for custom objects must be `[0, 127]`, while `[-1, -128

#### ExtensionCodec context

When you use an extension codec, it might be necessary to have encoding/decoding state to keep track of which objects got encoded/re-created. To do this, pass a `context` to the `EncodeOptions` and `DecodeOptions`:
When you use an extension codec, it might be necessary to have encoding/decoding state to keep track of which objects got encoded/re-created. To do this, pass a `context` to the `EncoderOptions` and `DecoderOptions`:

```typescript
import { encode, decode, ExtensionCodec } from "@msgpack/msgpack";
Expand Down
88 changes: 77 additions & 11 deletions src/Decoder.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,61 @@ import { utf8Decode } from "./utils/utf8";
import { createDataView, ensureUint8Array } from "./utils/typedArrays";
import { CachedKeyDecoder, KeyDecoder } from "./CachedKeyDecoder";
import { DecodeError } from "./DecodeError";
import type { ContextOf } from "./context";

export type DecoderOptions<ContextType = undefined> = Readonly<
Partial<{
extensionCodec: ExtensionCodecType<ContextType>;

/**
* Decodes Int64 and Uint64 as bigint if it's set to true.
* Depends on ES2020's {@link DataView#getBigInt64} and
* {@link DataView#getBigUint64}.
*
* Defaults to false.
*/
useBigInt64: boolean;

/**
* Maximum string length.
*
* Defaults to 4_294_967_295 (UINT32_MAX).
*/
maxStrLength: number;
/**
* Maximum binary length.
*
* Defaults to 4_294_967_295 (UINT32_MAX).
*/
maxBinLength: number;
/**
* Maximum array length.
*
* Defaults to 4_294_967_295 (UINT32_MAX).
*/
maxArrayLength: number;
/**
* Maximum map length.
*
* Defaults to 4_294_967_295 (UINT32_MAX).
*/
maxMapLength: number;
/**
* Maximum extension length.
*
* Defaults to 4_294_967_295 (UINT32_MAX).
*/
maxExtLength: number;

/**
* An object key decoder. Defaults to the shared instance of {@link CachedKeyDecoder}.
* `null` is a special value to disable the use of the key decoder at all.
*/
keyDecoder: KeyDecoder | null;
}>
> &
ContextOf<ContextType>;


const STATE_ARRAY = "array";
const STATE_MAP_KEY = "map_key";
Expand Down Expand Up @@ -54,6 +109,16 @@ const MORE_DATA = new DataViewIndexOutOfBoundsError("Insufficient data");
const sharedCachedKeyDecoder = new CachedKeyDecoder();

export class Decoder<ContextType = undefined> {
private readonly extensionCodec: ExtensionCodecType<ContextType>;
private readonly context: ContextType;
private readonly useBigInt64: boolean;
private readonly maxStrLength: number;
private readonly maxBinLength: number;
private readonly maxArrayLength: number;
private readonly maxMapLength: number;
private readonly maxExtLength: number;
private readonly keyDecoder: KeyDecoder | null;

private totalPos = 0;
private pos = 0;

Expand All @@ -62,17 +127,18 @@ export class Decoder<ContextType = undefined> {
private headByte = HEAD_BYTE_REQUIRED;
private readonly stack: Array<StackState> = [];

public constructor(
private readonly extensionCodec: ExtensionCodecType<ContextType> = ExtensionCodec.defaultCodec as any,
private readonly context: ContextType = undefined as any,
private readonly useBigInt64 = false,
private readonly maxStrLength = UINT32_MAX,
private readonly maxBinLength = UINT32_MAX,
private readonly maxArrayLength = UINT32_MAX,
private readonly maxMapLength = UINT32_MAX,
private readonly maxExtLength = UINT32_MAX,
private readonly keyDecoder: KeyDecoder | null = sharedCachedKeyDecoder,
) {}
public constructor(options?: DecoderOptions<ContextType>) {
this.extensionCodec = options?.extensionCodec ?? (ExtensionCodec.defaultCodec as ExtensionCodecType<ContextType>);
this.context = (options as { context: ContextType } | undefined)?.context as ContextType; // needs a type assertion because EncoderOptions has no context property when ContextType is undefined

this.useBigInt64 = options?.useBigInt64 ?? false;
this.maxStrLength = options?.maxStrLength ?? UINT32_MAX;
this.maxBinLength = options?.maxBinLength ?? UINT32_MAX;
this.maxArrayLength = options?.maxArrayLength ?? UINT32_MAX;
this.maxMapLength = options?.maxMapLength ?? UINT32_MAX;
this.maxExtLength = options?.maxExtLength ?? UINT32_MAX;
this.keyDecoder = (options?.keyDecoder !== undefined) ? options.keyDecoder : sharedCachedKeyDecoder;
}

private reinitializeState() {
this.totalPos = 0;
Expand Down
109 changes: 94 additions & 15 deletions src/Encoder.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,26 +3,105 @@ import { ExtensionCodec, ExtensionCodecType } from "./ExtensionCodec";
import { setInt64, setUint64 } from "./utils/int";
import { ensureUint8Array } from "./utils/typedArrays";
import type { ExtData } from "./ExtData";
import type { ContextOf, SplitUndefined } from "./context";


export const DEFAULT_MAX_DEPTH = 100;
export const DEFAULT_INITIAL_BUFFER_SIZE = 2048;

export type EncoderOptions<ContextType = undefined> = Partial<
Readonly<{
extensionCodec: ExtensionCodecType<ContextType>;

/**
* Encodes bigint as Int64 or Uint64 if it's set to true.
* {@link forceIntegerToFloat} does not affect bigint.
* Depends on ES2020's {@link DataView#setBigInt64} and
* {@link DataView#setBigUint64}.
*
* Defaults to false.
*/
useBigInt64: boolean;

/**
* The maximum depth in nested objects and arrays.
*
* Defaults to 100.
*/
maxDepth: number;

/**
* The initial size of the internal buffer.
*
* Defaults to 2048.
*/
initialBufferSize: number;

/**
* If `true`, the keys of an object is sorted. In other words, the encoded
* binary is canonical and thus comparable to another encoded binary.
*
* Defaults to `false`. If enabled, it spends more time in encoding objects.
*/
sortKeys: boolean;
/**
* If `true`, non-integer numbers are encoded in float32, not in float64 (the default).
*
* Only use it if precisions don't matter.
*
* Defaults to `false`.
*/
forceFloat32: boolean;

/**
* If `true`, an object property with `undefined` value are ignored.
* e.g. `{ foo: undefined }` will be encoded as `{}`, as `JSON.stringify()` does.
*
* Defaults to `false`. If enabled, it spends more time in encoding objects.
*/
ignoreUndefined: boolean;

/**
* If `true`, integer numbers are encoded as floating point numbers,
* with the `forceFloat32` option taken into account.
*
* Defaults to `false`.
*/
forceIntegerToFloat: boolean;
}>
> & ContextOf<ContextType>;

export class Encoder<ContextType = undefined> {
private pos = 0;
private view = new DataView(new ArrayBuffer(this.initialBufferSize));
private bytes = new Uint8Array(this.view.buffer);

public constructor(
private readonly extensionCodec: ExtensionCodecType<ContextType> = ExtensionCodec.defaultCodec as any,
private readonly context: ContextType = undefined as any,
private readonly useBigInt64 = false,
private readonly maxDepth = DEFAULT_MAX_DEPTH,
private readonly initialBufferSize = DEFAULT_INITIAL_BUFFER_SIZE,
private readonly sortKeys = false,
private readonly forceFloat32 = false,
private readonly ignoreUndefined = false,
private readonly forceIntegerToFloat = false,
) {}
private readonly extensionCodec: ExtensionCodecType<ContextType>;
private readonly context: ContextType;
private readonly useBigInt64: boolean;
private readonly maxDepth: number;
private readonly initialBufferSize: number;
private readonly sortKeys: boolean;
private readonly forceFloat32: boolean;
private readonly ignoreUndefined: boolean;
private readonly forceIntegerToFloat: boolean;

private pos: number;
private view: DataView;
private bytes: Uint8Array;

public constructor(options?: EncoderOptions<ContextType>) {
this.extensionCodec = options?.extensionCodec ?? (ExtensionCodec.defaultCodec as ExtensionCodecType<ContextType>);
this.context = (options as { context: ContextType } | undefined)?.context as ContextType; // needs a type assertion because EncoderOptions has no context property when ContextType is undefined

this.useBigInt64 = options?.useBigInt64 ?? false;
this.maxDepth = options?.maxDepth ?? DEFAULT_MAX_DEPTH;
this.initialBufferSize = options?.initialBufferSize ?? DEFAULT_INITIAL_BUFFER_SIZE;
this.sortKeys = options?.sortKeys ?? false;
this.forceFloat32 = options?.forceFloat32 ?? false;
this.ignoreUndefined = options?.ignoreUndefined ?? false;
this.forceIntegerToFloat = options?.forceIntegerToFloat ?? false;

this.pos = 0;
this.view = new DataView(new ArrayBuffer(this.initialBufferSize));
this.bytes = new Uint8Array(this.view.buffer);
}

private reinitializeState() {
this.pos = 0;
Expand Down
Loading