Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configurable Transports and Extraction of the Node HTTP Transport #265

Merged
merged 23 commits into from
Oct 27, 2018
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
fbebb1f
WIP to allow configuration of Transports.
jonny-improbable Oct 20, 2018
dcda21d
Fix linting
jonny-improbable Oct 20, 2018
0774e58
Allow independent configuration of XHR and Fetch Transports.
jonny-improbable Oct 21, 2018
17066a9
Fix lint
jonny-improbable Oct 21, 2018
dbf695b
Allow the user to specify the default transport.
jonny-improbable Oct 21, 2018
f3d41e7
Refactor transport package
jonny-improbable Oct 21, 2018
b08e5a8
Drop support for NodeJS transports.
jonny-improbable Oct 21, 2018
f35d8de
Allow extended configuration of FetchInit
jonny-improbable Oct 21, 2018
84e10ba
Add support for referrerPolicy, keepalive and integrity FetchInit opt…
jonny-improbable Oct 21, 2018
e671411
Remove NodeJS typings as they are no longer required.
jonny-improbable Oct 22, 2018
a0cab8f
Rename `HttpTransport` -> `CrossBrowserHttpTransport` to make it clea…
jonny-improbable Oct 22, 2018
3f668d7
Merge branch 'master' into feature/configurable-transports
jonny-improbable Oct 22, 2018
dbe3fbd
Move Node HTTP transport into the repo and include it in the tests.
jonny-improbable Oct 24, 2018
afc8caf
Fix linking script
jonny-improbable Oct 24, 2018
b5c274b
Fix linking
jonny-improbable Oct 24, 2018
ee85f3c
Trying to get linking working on travis...
jonny-improbable Oct 25, 2018
86b3a6f
Give up on using npm link :D
jonny-improbable Oct 25, 2018
ebcb001
Revert "Give up on using npm link :D"
jonny-improbable Oct 25, 2018
bf08173
Make NPM Linking work after praying to the correct gods.
jonny-improbable Oct 25, 2018
26b0c07
Remove duplicate call to `link-npm-modules.sh`
jonny-improbable Oct 25, 2018
43ab1e9
- Reinstate note on NodeJS usage in README
jonny-improbable Oct 25, 2018
ca16ed5
Rewrite transport docs.
jonny-improbable Oct 27, 2018
d66c8af
Fix style issues.
jonny-improbable Oct 27, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions test/ts/src/client.websocket.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ if (process.env.DISABLE_WEBSOCKET_TESTS) {
const client = grpc.client(TestService.PingStream, {
debug: DEBUG,
host: testHostUrl,
transport: grpc.WebsocketTransportFactory,
transport: grpc.WebsocketTransport(),
});
client.onHeaders((headers: grpc.Metadata) => {
DEBUG && debug("headers", headers);
Expand Down Expand Up @@ -86,7 +86,7 @@ if (process.env.DISABLE_WEBSOCKET_TESTS) {
const client = grpc.client(TestService.PingPongBidi, {
debug: DEBUG,
host: testHostUrl,
transport: grpc.WebsocketTransportFactory,
transport: grpc.WebsocketTransport(),
});
client.onHeaders((headers: grpc.Metadata) => {
DEBUG && debug("headers", headers);
Expand Down Expand Up @@ -145,7 +145,7 @@ if (process.env.DISABLE_WEBSOCKET_TESTS) {
const client = grpc.client(TestService.PingPongBidi, {
debug: DEBUG,
host: testHostUrl,
transport: grpc.WebsocketTransportFactory,
transport: grpc.WebsocketTransport(),
});
client.onHeaders((headers: grpc.Metadata) => {
DEBUG && debug("headers", headers);
Expand Down
8 changes: 4 additions & 4 deletions test/ts/src/testRpcCombinations.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,13 +52,13 @@ export function runWithHttp1AndHttp2(cb: (config: TestConfig) => void) {
});
}

export function runWithSupportedTransports(cb: (transport: grpc.TransportConstructor | undefined) => void) {
const transports: {[key: string]: grpc.TransportConstructor | undefined} = {
"defaultTransport": undefined
export function runWithSupportedTransports(cb: (transport: grpc.TransportFactory | undefined) => void) {
const transports: {[key: string]: grpc.TransportFactory | undefined} = {
"httpTransport": undefined
};

if (!process.env.DISABLE_WEBSOCKET_TESTS) {
transports["websocketTransport"] = grpc.WebsocketTransportFactory
transports["websocketTransport"] = grpc.WebsocketTransport()
}

for (let transportName in transports) {
Expand Down
4 changes: 2 additions & 2 deletions ts/docs/client.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ grpc.client(methodDescriptor: MethodDescriptor, props: ClientRpcOptions): Client

* `host: string`
* The server address (`"https://example.com:9100"`)
* `transport?: TransportConstructor`
* `transport?: TransportFactory`
* (optional) A function to build a `Transport` that will be used for the request. If no transport is specified then a browser-compatible transport will be used. See [transport](transport.md).
* `debug?: boolean`
* (optional) if `true`, debug information will be printed to the console
Expand Down Expand Up @@ -60,7 +60,7 @@ Sending multiple messages and indicating that the client has finished sending ar

Most browser networking methods do not allow control over the sending of the body of the request, meaning that sending a single request message forces the finishing of sending, limiting these transports to unary or server-streaming methods only.

For transports that do allow control over the sending of the body (e.g. websockets - coming soon), the client can optionally indicate that it has finished sending. This is useful for client-streaming or bi-directional methods in which the server will send responses after receiving all client messages. Usage with unary methods is likely not necessary as server handlers will assume the client has finished sending after receiving the single expected message.
For transports that do allow control over the sending of the body (e.g. websockets), the client can optionally indicate that it has finished sending. This is useful for client-streaming or bi-directional methods in which the server will send responses after receiving all client messages. Usage with unary methods is likely not necessary as server handlers will assume the client has finished sending after receiving the single expected message.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated change, but whilst I was here I wanted to fix this.


## Example:
```javascript
Expand Down
2 changes: 1 addition & 1 deletion ts/docs/invoke.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ grpc.invoke(methodDescriptor: MethodDescriptor, props: InvokeRpcOptions): Reques
* A callback for messages being received
* `onEnd: (code: grpc.Code, message: string, trailers: grpc.Metadata) => void)`
* A callback for the end of the request and trailers being received
* `transport?: TransportConstructor`
* `transport?: TransportFactory`
* (optional) A function to build a `Transport` that will be used for the request. If no transport is specified then a browser-compatible transport will be used. See [transport](transport.md).
* `debug?: boolean`
* (optional) if `true`, debug information will be printed to the console
Expand Down
10 changes: 4 additions & 6 deletions ts/docs/transport.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,13 @@ A “transport” in this context is a wrapper of one of these methods of creati

## How does grpc-web-client pick a transport?

You can specify the transport that you want to use for a specific invocation through the `library` property in the [`client`](client.md), [`invoke`](invoke.md) and [`unary`](unary.md) function options.
You can specify the transport that you want to use for a specific invocation through the `transport` property in the [`client`](client.md), [`invoke`](invoke.md) and [`unary`](unary.md) function options.

If a transport is not specified then a transport factory is used to determine the browser’s compatible transports. See [Available Transports](#available-transports)
If a transport is not specified then a grpc-web-client will fall back to a DefaultHttpTransport which works across the vast majority of browsers with some limitations. See [Available Transports](#available-transports)

If none are found then an exception is thrown.
### Automatic HTTP Transport Detection

### Available transports

In order of attempted usage.
grpc-web-client will, by default, attempt to detect the most suitable method for making HTTP based requests. In order of attempted usage:

#### Fetch
Uses [`fetch`](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch). Requires that the browser supports [Fetch with `body.getReader`](https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream).
Expand Down
2 changes: 1 addition & 1 deletion ts/docs/unary.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ grpc.unary(methodDescriptor: MethodDescriptor, props: UnaryRpcOptions): Request;
* The metadata to send to the server
* `onEnd: (output: UnaryOutput<TResponse>) => void)`
* A callback for the end of the request and trailers being received
* `transport?: TransportConstructor`
* `transport?: TransportFactory`
* (optional) A function to build a `Transport` that will be used for the request. If no transport is specified then a browser-compatible transport will be used. See [transport](transport.md).
* `debug?: boolean`
* (optional) if `true`, debug information will be printed to the console
Expand Down
21 changes: 7 additions & 14 deletions ts/src/client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,14 @@ import {ChunkParser, Chunk, ChunkType} from "./ChunkParser";
import {Code, httpStatusToCode} from "./Code";
import {debug} from "./debug";
import detach from "./detach";
import {Transport, TransportConstructor, DefaultTransportFactory} from "./transports/Transport";
import {Transport, DefaultHttpTransport, TransportFactory} from "./transports/Transport";
import {MethodDefinition} from "./service";
import {frameRequest} from "./util";
import {ProtobufMessage} from "./message";

export interface ClientRpcOptions {
host: string;
transport?: TransportConstructor;
transport?: TransportFactory;
debug?: boolean;
}

Expand Down Expand Up @@ -67,19 +67,12 @@ class GrpcClient<TRequest extends ProtobufMessage, TResponse extends ProtobufMes
onEnd: this.onTransportEnd.bind(this),
};

let transportConstructor = this.props.transport;
if (transportConstructor) {
const constructedTransport = transportConstructor(transportOptions);
if (constructedTransport instanceof Error) {
throw constructedTransport;
}
this.transport = constructedTransport;


if (this.props.transport) {
this.transport = this.props.transport(transportOptions);
} else {
const factoryTransport = DefaultTransportFactory(transportOptions);
if (factoryTransport instanceof Error) {
throw factoryTransport;
}
this.transport = factoryTransport;
this.transport = DefaultHttpTransport(transportOptions);
}
}

Expand Down
17 changes: 14 additions & 3 deletions ts/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
import {BrowserHeaders} from "browser-headers";
import * as impTransport from "./transports/Transport";
import * as impFetch from "./transports/fetch";
import * as impXhr from "./transports/xhr";
import * as impCode from "./Code";
import * as impInvoke from "./invoke";
import * as impUnary from "./unary";
Expand All @@ -13,9 +15,18 @@ export namespace grpc {

export interface Transport extends impTransport.Transport {}
export interface TransportOptions extends impTransport.TransportOptions {}
export interface TransportConstructor extends impTransport.TransportConstructor {}
export const DefaultTransportFactory = impTransport.DefaultTransportFactory;
export const WebsocketTransportFactory = impTransport.WebsocketTransportFactory;
export interface TransportFactory extends impTransport.TransportFactory {}

export const HttpTransport = impTransport.HttpTransport;
export interface HttpTransportInit extends impTransport.HttpTransportInit {}

export const FetchReadableStreamTransport = impTransport.FetchReadableStreamTransport;
export interface FetchReadableStreamInit extends impFetch.FetchTransportInit {}

export const XhrTransport = impTransport.XhrTransport;
export interface XhrTransportInit extends impXhr.XhrTransportInit {}

export const WebsocketTransport = impTransport.WebsocketTransport;

export interface UnaryMethodDefinition<TRequest extends ProtobufMessage, TResponse extends ProtobufMessage> extends impService.UnaryMethodDefinition<TRequest, TResponse> {}
export interface MethodDefinition<TRequest extends ProtobufMessage, TResponse extends ProtobufMessage> extends impService.MethodDefinition<TRequest, TResponse> {}
Expand Down
4 changes: 2 additions & 2 deletions ts/src/invoke.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import {Code} from "./Code";
import {TransportConstructor} from "./transports/Transport";
import {TransportFactory} from "./transports/Transport";
import {MethodDefinition} from "./service";
import {Metadata} from "./metadata";
import {client} from "./client";
Expand All @@ -16,7 +16,7 @@ export interface InvokeRpcOptions<TRequest extends ProtobufMessage, TResponse ex
onHeaders?: (headers: Metadata) => void;
onMessage?: (res: TResponse) => void;
onEnd: (code: Code, message: string, trailers: Metadata) => void;
transport?: TransportConstructor;
transport?: TransportFactory;
debug?: boolean;
}

Expand Down
59 changes: 30 additions & 29 deletions ts/src/transports/Transport.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
import {Metadata} from "../metadata";
import fetchRequest, {detectFetchSupport} from "./fetch";
import xhrRequest, {detectXHRSupport} from "./xhr";
import fetchRequest, {detectFetchSupport, FetchTransportInit} from "./fetch";
import xhrRequest, {XhrTransportInit} from "./xhr";
import mozXhrRequest, {detectMozXHRSupport} from "./mozXhr";
import httpNodeRequest, {detectNodeHTTPSupport} from "./nodeHttp";
import {MethodDefinition} from "../service";
import {ProtobufMessage} from "../message";
import websocketRequest from "./websocket";
Expand All @@ -14,8 +13,8 @@ export interface Transport {
start(metadata: Metadata): void
}

export interface TransportConstructor {
(options: TransportOptions): Transport | Error;
export interface HttpTransportConstructor {
(options: TransportOptions, config: HttpTransportInit): Transport;
}

export interface TransportOptions {
Expand All @@ -27,40 +26,42 @@ export interface TransportOptions {
onEnd: (err?: Error) => void;
}

let selectedTransport: TransportConstructor;
export function DefaultTransportFactory(transportOptions: TransportOptions): Transport | Error {
// The transports provided by DefaultTransportFactory do not support client-streaming
if (transportOptions.methodDefinition.requestStream) {
return new Error("No transport available for client-streaming (requestStream) method");
}
export function DefaultHttpTransport(transportOptions: TransportOptions): Transport {
return HttpTransport({ withCredentials: false })(transportOptions);
}

if (!selectedTransport) {
selectedTransport = detectTransport();
}
export interface HttpTransportInit {
withCredentials?: boolean
}

return selectedTransport(transportOptions);
export interface TransportFactory {
(options: TransportOptions): Transport;
}

function detectTransport(): TransportConstructor {
export function HttpTransport(init: HttpTransportInit): TransportFactory {
if (detectFetchSupport()) {
return fetchRequest;
}

if (detectMozXHRSupport()) {
return mozXhrRequest;
return FetchReadableStreamTransport({ credentials: init.withCredentials ? 'include' : 'same-origin' })
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do you configure this to omit now?

Copy link

@MOZGIII MOZGIII Oct 21, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This implementation allows explicitly using transport: FetchReadableStreamTransport({ credentials: "omit" }) in the invocation/client instead of transport: HttpTransport({ /* whatever */ }).
HttpTransport can't support omit because it's not in the "greatest common interface", as XMLHttpRequest can only allow include or same-origin - https://xhr.spec.whatwg.org/#the-send()-method see credentials mode at (6) there.

}
return XhrTransport({ withCredentials: init.withCredentials });
}

if (detectXHRSupport()) {
return xhrRequest;
export function XhrTransport(init: XhrTransportInit): TransportFactory {
return (opts: TransportOptions) => {
if (detectMozXHRSupport()) {
return mozXhrRequest(opts, init);
}
return xhrRequest(opts, init);
}
}

if (detectNodeHTTPSupport()) {
return httpNodeRequest;
export function FetchReadableStreamTransport(init: FetchTransportInit): TransportFactory {
return (opts: TransportOptions) => {
return fetchRequest(opts, init);
}

throw new Error("No suitable transport found for gRPC-Web");
}

Copy link

@MOZGIII MOZGIII Oct 26, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you view this file it's easy to notice a lot of empty lines - we should probably clean them up.
Have you considered using prettier?

export function WebsocketTransportFactory(transportOptions: TransportOptions): Transport | Error {
return websocketRequest(transportOptions);
export function WebsocketTransport(): TransportFactory {
return (opts: TransportOptions) => {
return websocketRequest(opts);
}
}
15 changes: 10 additions & 5 deletions ts/src/transports/fetch.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,13 @@ import {Transport, TransportOptions} from "./Transport";
import {debug} from "../debug";
import detach from "../detach";

/* fetchRequest uses Fetch (with ReadableStream) to read response chunks without buffering the entire response. */
export default function fetchRequest(options: TransportOptions): Transport {
export interface FetchTransportInit {
credentials: "omit" | "same-origin" | "include"
}

export default function fetchRequest(options: TransportOptions, init: FetchTransportInit): Transport {
options.debug && debug("fetchRequest", options);
return new Fetch(options);
return new Fetch(options, init);
}

declare const Response: any;
Expand All @@ -15,12 +18,14 @@ declare const Headers: any;
class Fetch implements Transport {
cancelled: boolean = false;
options: TransportOptions;
init: FetchTransportInit;
reader: ReadableStreamReader;
metadata: Metadata;
controller: AbortController | undefined = (window as any).AbortController && new AbortController();

constructor(transportOptions: TransportOptions) {
constructor(transportOptions: TransportOptions, init: FetchTransportInit) {
this.options = transportOptions;
this.init = init;
}

pump(readerArg: ReadableStreamReader, res: Response) {
Expand Down Expand Up @@ -63,7 +68,7 @@ class Fetch implements Transport {
headers: this.metadata.toHeaders(),
method: "POST",
body: msgBytes,
credentials: "same-origin",
credentials: this.init.credentials,
signal: this.controller && this.controller.signal
}).then((res: Response) => {
this.options.debug && debug("Fetch.response", res);
Expand Down
12 changes: 9 additions & 3 deletions ts/src/transports/mozXhr.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,23 +3,26 @@ import {Transport, TransportOptions} from "./Transport";
import {debug} from "../debug";
import detach from "../detach";
import {xhrSupportsResponseType} from "./xhrUtil";
import {XhrTransportInit} from "./xhr";

/* mozXhrRequest uses XmlHttpRequest with responseType "moz-chunked-arraybuffer" to support binary streaming in Firefox.
* Firefox's Fetch as of version 52 does not implement a ReadableStream interface. moz-chunked-arraybuffer enables
* receiving byte chunks without buffering the entire response as the xhrRequest transport does. */
export default function mozXhrRequest(options: TransportOptions): Transport {
export default function mozXhrRequest(options: TransportOptions, init: XhrTransportInit): Transport {
options.debug && debug("mozXhrRequest", options);
return new MozXHR(options);
return new MozXHR(options, init);
}

class MozXHR implements Transport {
options: TransportOptions;
init: XhrTransportInit;
xhr: XMLHttpRequest;
metadata: Metadata;
index: 0;

constructor(transportOptions: TransportOptions) {
constructor(transportOptions: TransportOptions, init: XhrTransportInit) {
this.options = transportOptions;
this.init = init;
}

onProgressEvent() {
Expand Down Expand Up @@ -68,6 +71,9 @@ class MozXHR implements Transport {
this.metadata.forEach((key, values) => {
xhr.setRequestHeader(key, values.join(", "));
});

xhr.withCredentials = Boolean(this.init.withCredentials);

xhr.addEventListener("readystatechange", this.onStateChange.bind(this));
xhr.addEventListener("progress", this.onProgressEvent.bind(this));
xhr.addEventListener("loadend", this.onLoadEvent.bind(this));
Expand Down
Loading