Skip to content
This repository has been archived by the owner on Oct 11, 2024. It is now read-only.

Updated prettier to include all markdown files #830

Merged
merged 3 commits into from
Jun 10, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 0 additions & 6 deletions .prettierignore
Original file line number Diff line number Diff line change
@@ -1,10 +1,4 @@
lib
docs
*README.md
CHANGELOG.md
CONTRIBUTING.md
ISSUE_TEMPLATE.md
pull_request_template.md
ethereum/blockwatch/testdata
packages/**/generated/**
*.d.ts
296 changes: 142 additions & 154 deletions CHANGELOG.md

Large diffs are not rendered by default.

5 changes: 3 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ _outside_ of your `GOPATH`.
0x Mesh uses two main branches:

1. The `development` branch contains the latest (possibly unreleased) changes
and is not guaranteed to be stable.
and is not guaranteed to be stable.
2. The `master` branch contains the latest stable release.

If you intend to fork 0x Mesh and open a PR, you should work off of the
Expand Down Expand Up @@ -46,7 +46,7 @@ make deps

## Building TypeScript packages

Mesh contains some TypeScript packages, all of which are contained in a small monorepo in the __packages/__ directory. Some
Mesh contains some TypeScript packages, all of which are contained in a small monorepo in the **packages/** directory. Some
packages are published, and some are only used internally for development and testing.

To build all the TypeScript packages:
Expand Down Expand Up @@ -91,6 +91,7 @@ make test-all
```

### Potential Issues

The default maximum number of open files is too low in some operating systems
for the tests to be run successfully. If an error that reads like "Too many open files,"
it may be necessary to increase this limit. On Unix-like operating systems, the `ulimit`
Expand Down
25 changes: 12 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,24 +12,23 @@

We have reached the point where Mesh is being used by some teams in production. We feel that for many use cases, Mesh is stable enough for production. However, we caution that there are some issues and shortcomings in its current state, which generally fall into two categories:

- Order sharing: We have recently made significant improvements to our order sharing algorithm, including reducing bandwidth usage and CPU usage by at least an order of magnitude on average. See https://github.com/0xProject/0x-mesh/pull/692 and https://github.com/0xProject/0x-mesh/pull/732. However, we are still working on accurately testing and measuring the speed at which orders propagate through the network with different network sizes and topologies. In some circumstances, it may take longer than we would like for orders to reach the majority of nodes in the network. This is an area we will continue to focus on and improve.
- Browser usage: Mesh can run directly in the browser via the [@0x/mesh-browser](https://www.npmjs.com/package/@0x/mesh-browser) package. We have supported this for a while and have examples and integration tests in this repository. While we have made recent improvements to stability and performance (see https://github.com/0xProject/0x-mesh/pull/703, https://github.com/0xProject/0x-mesh/pull/697, and https://github.com/0xProject/0x-mesh/pull/694), there are still some [important missing features and issues to address](https://github.com/0xProject/0x-mesh/issues?q=is%3Aopen+is%3Aissue+label%3Abrowser) before `@0x/mesh-browser` is feasible for most production use cases.

- Order sharing: We have recently made significant improvements to our order sharing algorithm, including reducing bandwidth usage and CPU usage by at least an order of magnitude on average. See https://github.com/0xProject/0x-mesh/pull/692 and https://github.com/0xProject/0x-mesh/pull/732. However, we are still working on accurately testing and measuring the speed at which orders propagate through the network with different network sizes and topologies. In some circumstances, it may take longer than we would like for orders to reach the majority of nodes in the network. This is an area we will continue to focus on and improve.
- Browser usage: Mesh can run directly in the browser via the [@0x/mesh-browser](https://www.npmjs.com/package/@0x/mesh-browser) package. We have supported this for a while and have examples and integration tests in this repository. While we have made recent improvements to stability and performance (see https://github.com/0xProject/0x-mesh/pull/703, https://github.com/0xProject/0x-mesh/pull/697, and https://github.com/0xProject/0x-mesh/pull/694), there are still some [important missing features and issues to address](https://github.com/0xProject/0x-mesh/issues?q=is%3Aopen+is%3Aissue+label%3Abrowser) before `@0x/mesh-browser` is feasible for most production use cases.

## Overview

0x Mesh has a lot of different use cases for different categories of users:

- Relayers can use Mesh to share orders with one another and to receive orders
from market makers. This allows them to increase the depth of their order
books and provide a better user experience.
- Market makers can use Mesh to reach a broader audience. Their orders will be
sent throughout the network and picked up by many trading venues and are therefore more likely to be filled.
- Mesh allows for a new type of relayer called a "serverless relayer". In the
serverless relayer model, each user runs Mesh in their browser and there is
no backend server or database. Instead, peers share orders directly with one
another. (There are pros and cons to this approach and it is probably not
suitable for all markets).
- Relayers can use Mesh to share orders with one another and to receive orders
from market makers. This allows them to increase the depth of their order
books and provide a better user experience.
- Market makers can use Mesh to reach a broader audience. Their orders will be
sent throughout the network and picked up by many trading venues and are therefore more likely to be filled.
- Mesh allows for a new type of relayer called a "serverless relayer". In the
serverless relayer model, each user runs Mesh in their browser and there is
no backend server or database. Instead, peers share orders directly with one
another. (There are pros and cons to this approach and it is probably not
suitable for all markets).

Both Relayers and Market makers can use Mesh to watch a set of 0x orders for changes in fillability (e.g., cancellations, fills, expirations, etc...).

Expand Down
22 changes: 0 additions & 22 deletions cmd/cut-release/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -38,28 +38,6 @@ func main() {
}

generateTypescriptDocs()
createReleaseChangelog(env.Version)
}

func createReleaseChangelog(version string) {
regex := fmt.Sprintf(`(?ms)(## v%s\n)(.*?)(## v)`, version)
changelog, err := getFileContentsWithRegex("CHANGELOG.md", regex)
if err != nil {
log.Println("No CHANGELOG entries found for version", version)
return // Noop
}

releaseChangelog := fmt.Sprintf(`- [Docker image](https://hub.docker.com/r/0xorg/mesh/tags)
- [README](https://github.com/0xProject/0x-mesh/blob/v%s/README.md)

## Summary
%s
`, version, changelog)

err = ioutil.WriteFile("RELEASE_CHANGELOG.md", []byte(releaseChangelog), 0644)
if err != nil {
log.Fatal(err)
}
}

func generateTypescriptDocs() {
Expand Down
20 changes: 10 additions & 10 deletions docs/browser.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,15 @@ approach comes with a lot of advantages, but also has some drawbacks:

### Advantages

- Increased decentralization
- Little to no hosting costs
- Ability to trade experimental/niche assets
- Increased decentralization
- Little to no hosting costs
- Ability to trade experimental/niche assets

### Drawbacks

- Longer warm-up time
- Increased risk of trade collisions
- Consumes more end-user resources
- Longer warm-up time
- Increased risk of trade collisions
- Consumes more end-user resources

## @0x/mesh-browser

Expand Down Expand Up @@ -77,10 +77,10 @@ yarn add @0x/mesh-browser-lite

## Documentation

* Documentation for the `@0x/mesh-browser` package is available at
[0x-org.gitbook.io/mesh/browser-bindings/browser](https://0x-org.gitbook.io/mesh/browser-bindings/browser).
* Documentation for the `@0x/mesh-browser-lite` package is available at
[0x-org.gitbook.io/mesh/browser-bindings/browser-lite](https://0x-org.gitbook.io/mesh/browser-bindings/browser-lite).
- Documentation for the `@0x/mesh-browser` package is available at
[0x-org.gitbook.io/mesh/browser-bindings/browser](https://0x-org.gitbook.io/mesh/browser-bindings/browser).
- Documentation for the `@0x/mesh-browser-lite` package is available at
[0x-org.gitbook.io/mesh/browser-bindings/browser-lite](https://0x-org.gitbook.io/mesh/browser-bindings/browser-lite).

### Example usage

Expand Down
39 changes: 21 additions & 18 deletions docs/custom_order_filters.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@

Mesh supports the creation of separate sub-networks where 0x orders that adhere to a specific schema are shared. Each sub-network is built around a custom order filter. The custom filter defines which orders are allowed to be shared within a sub-network. For example:

- All orders for a specific asset pair (e.g., WETH/DAI)
- All orders for non-fungibles (i.e., ERC721, ERC1155)
- All orders used by a specific DApp
- All orders for a specific asset pair (e.g., WETH/DAI)
- All orders for non-fungibles (i.e., ERC721, ERC1155)
- All orders used by a specific DApp

A custom filter may be passed into Mesh as a [JSON Schema](https://json-schema.org/) via the `CUSTOM_ORDER_FILTER` environment variable. Messages that contain orders that don't match this schema will be dropped. As a limitation, filtering is only possible by looking at the static fields of an order. So for example, it is not possible to filter orders by doing an on-chain check or sending an HTTP request to a third-party API. We don't expect that this limitation is going to be a problem in practice and it comes with the huge benefit of enabling cross-topic forwarding in the future (more on that later).

Expand All @@ -14,17 +14,20 @@ All orders must match the following JSON Schema:

```json
{
"id": "/rootOrder",
"allOf": [{
"$ref": "/customOrder"
}, {
"$ref": "/signedOrder"
}]
"id": "/rootOrder",
"allOf": [
{
"$ref": "/customOrder"
},
{
"$ref": "/signedOrder"
}
]
}
```

- `/signedOrder` is the JSON Schema that will match any valid 0x orders.
- `/customOrder` is the custom schema passed in through the `CUSTOM_ORDER_FILTER` environment variable.
- `/signedOrder` is the JSON Schema that will match any valid 0x orders.
- `/customOrder` is the custom schema passed in through the `CUSTOM_ORDER_FILTER` environment variable.

Organizing the JSON Schema for orders like this means that `CUSTOM_ORDER_FILTER` can be relatively small. It doesn't need to contain all the required fields for a signed 0x order. It just needs to contain any _additional_ requirements on top of the default ones.

Expand All @@ -42,16 +45,17 @@ The following `CUSTOM_ORDER_FILTER` doesn't add any additional requirements. All

```json
{
"properties": {
"senderAddress": {
"pattern": "0x00000000000000000000000000000000ba5eba11",
"type": "string"
}
}
"properties": {
"senderAddress": {
"pattern": "0x00000000000000000000000000000000ba5eba11",
"type": "string"
}
}
}
```

#### Mainnet WETH <-> DAI orders:

```json
{
"oneOf": [
Expand Down Expand Up @@ -122,7 +126,6 @@ The following `CUSTOM_ORDER_FILTER` doesn't add any additional requirements. All

Where `${AUGUR_ERC1155_CONTRACT_ADDRESS}` needs to be replaced with the Augur ERC1155 token used to represent the outcomes of their various prediction markets.


As you can see by the above examples, JSON-Schema has support for [regular expressions](https://json-schema.org/understanding-json-schema/reference/regular_expressions.html) allowing for partial matching of any 0x order field.

## Limitations
Expand Down
20 changes: 10 additions & 10 deletions docs/db_syncing.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,12 @@ When first connecting the DB and Mesh node, we first need to make sure both have

Subscribe to the Mesh node's `orders` subscription over a WS connection. This can be done using our [golang](https://godoc.org/github.com/0xProject/0x-mesh/rpc) or [Typescript/Javascript](json_rpc_clients/typescript/README.md) clients or any other JSON-RPC WebSocket client. Whenever you receive an order event from this subscription, make the appropriate updates to your DB. Each order event has an associated [OrderEventEndState](https://godoc.org/github.com/0xProject/0x-mesh/zeroex#pkg-constants).

| End state | DB operation |
|--------------------------------------------|---------------------------------|
| ADDED | Insert |
| FILLED | Update |
| FULLY_FILLED, EXPIRED, CANCELLED, UNFUNDED | Remove |
| FILLABILITY_INCREASED | Upsert |
| End state | DB operation |
| ------------------------------------------ | ------------ |
| ADDED | Insert |
| FILLED | Update |
| FULLY_FILLED, EXPIRED, CANCELLED, UNFUNDED | Remove |
| FILLABILITY_INCREASED | Upsert |

**Note:** Updates refer to updating the order's `fillableTakerAssetAmount` in the DB.

Expand All @@ -37,10 +37,10 @@ There might have been orders stored in Mesh that the DB doesn't know about at th

Since there might also be orders added to the database that Mesh doesn't know about, we must also add all DB orders to Mesh. We can do this using the [mesh_addOrders](rpc_api.md#mesh_addorders) JSON-RPC method. This method accepts an array of signed 0x orders and returns which have been accepted and rejected. The accepted orders are returned with their `fillableTakerAssetAmount` and so these amounts should be updated in the database. Rejected orders are rejected with a specific [RejectedOrderStatus](https://godoc.org/github.com/0xProject/0x-mesh/zeroex#pkg-variables), including an identifying `code`.

| Code | Reason | Should be retried? |
|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------|--------------------|
| EthRPCRequestFailed, CoordinatorRequestFailed, CoordinatorEndpointNotFound, InternalError | Failure to validate the order | Yes |
| MaxOrderSizeExceeded, OrderMaxExpirationExceeded, OrderForIncorrectChain, SenderAddressNotAllowed | Failed Mesh-specific criteria | No |
| Code | Reason | Should be retried? |
| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------- | ------------------ |
| EthRPCRequestFailed, CoordinatorRequestFailed, CoordinatorEndpointNotFound, InternalError | Failure to validate the order | Yes |
| MaxOrderSizeExceeded, OrderMaxExpirationExceeded, OrderForIncorrectChain, SenderAddressNotAllowed | Failed Mesh-specific criteria | No |
| OrderHasInvalidMakerAssetData, OrderHasInvalidTakerAssetData, OrderHasInvalidSignature, OrderUnfunded, OrderCancelled, OrderFullyFilled, OrderHasInvalidMakerAssetAmount, OrderHasInvalidTakerAssetAmount, OrderExpired | Invalid or unfillable order | No |

If an order was rejected with a code related to the "failure to validate the order" reason above, you can re-try adding the order to Mesh after a back-off period. For all other rejection reasons, the orders should be removed from the database.
Expand Down
2 changes: 1 addition & 1 deletion docs/deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ machine where all Mesh-related data will be stored.
- Ports 60557, 60558, and 60559 are the default ports used for the JSON RPC endpoint, communicating with peers over TCP, and communicating with peers over WebSockets, respectively.
- In order to disable P2P order discovery and sharing, set `USE_BOOTSTRAP_LIST` to `false`.
- Running a VPN may interfere with Mesh. If you are having difficulty connecting to peers, disable your VPN.
- If you are running against a POA testnet (e.g., Kovan), you might want to shorten the `BLOCK_POLLING_INTERVAL` since blocks are mined more frequently then on mainnet. If you do this, your node will use more Ethereum RPC calls, so you will also need to adjust the `ETHEREUM_RPC_MAX_REQUESTS_PER_24_HR_UTC` upwards (*warning:* changing this setting can exceed the limits of your Ethereum RPC provider).
- If you are running against a POA testnet (e.g., Kovan), you might want to shorten the `BLOCK_POLLING_INTERVAL` since blocks are mined more frequently then on mainnet. If you do this, your node will use more Ethereum RPC calls, so you will also need to adjust the `ETHEREUM_RPC_MAX_REQUESTS_PER_24_HR_UTC` upwards (_warning:_ changing this setting can exceed the limits of your Ethereum RPC provider).
- If you want to run the mesh in "detached" mode, add the `-d` switch to the docker run command so that your console doesn't get blocked.

## Persisting State
Expand Down
23 changes: 11 additions & 12 deletions docs/deployment_with_telemetry.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ easily modified to deploy on

### Setting up docker-compose.yml

Let's start by creating a new directory and adding a __docker-compose.yml__ file
Let's start by creating a new directory and adding a **docker-compose.yml** file
with the following contents:

```
Expand Down Expand Up @@ -82,12 +82,12 @@ services:
- PRIVATE_KEY_PATH=/app/keys/privkey
```

In most cases, the only change you need to make to the __docker-compose.yml__
file is to set `ETHEREUM_RPC_URL` to your own Ethereum JSON RPC endpoint. The
`WS_RPC_ADDR` and `HTTP_RPC_ADDR` above will allow any Docker containers running in the same Docker
Compose file to access the Mesh RPC API via
[links](https://docs.docker.com/compose/networking/#links). To use this feature,
be sure to add the following line to any containers you wish to access the Mesh
In most cases, the only change you need to make to the **docker-compose.yml**
file is to set `ETHEREUM_RPC_URL` to your own Ethereum JSON RPC endpoint. The
`WS_RPC_ADDR` and `HTTP_RPC_ADDR` above will allow any Docker containers running in the same Docker
Compose file to access the Mesh RPC API via
[links](https://docs.docker.com/compose/networking/#links). To use this feature,
be sure to add the following line to any containers you wish to access the Mesh
RPC API from:

```
Expand All @@ -97,12 +97,11 @@ links:

You can then use the URL `ws://mesh:60557` to access the RPC API.

Alternatively, if you want to open up your Mesh RPC API to the public internet,
you can set `WS_RPC_ADDR=0.0.0.0:60557` and `HTTP_RPC_ADDR=0.0.0.0:60556`. If
you choose to go this route, we strongly recommend using an external firewall
Alternatively, if you want to open up your Mesh RPC API to the public internet,
you can set `WS_RPC_ADDR=0.0.0.0:60557` and `HTTP_RPC_ADDR=0.0.0.0:60556`. If
you choose to go this route, we strongly recommend using an external firewall
to restrict who can access your RPC API.


### Deploying with Docker Machine

We are now ready to deploy our instance. Before we can continue, you will need
Expand Down Expand Up @@ -164,7 +163,7 @@ logs called `myPeerID`:

```json
{
"myPeerID": "QmbKkHnmkmFxKbPWbBNz3inKizDuqjTsWsVyutnshYULLp",
"myPeerID": "QmbKkHnmkmFxKbPWbBNz3inKizDuqjTsWsVyutnshYULLp"
}
```

Expand Down
Loading