pokt-network · Olshansk · Feb 3, 2023 · Dec 28, 2022 · Dec 28, 2022 · Dec 28, 2022
diff --git a/.github/ISSUE_TEMPLATE/issue.md b/.github/ISSUE_TEMPLATE/issue.md
@@ -31,7 +31,6 @@ assignees: ""
 
 ## Deliverable
 
-
 <!-- REMOVE this comment block after following the instructions
   Make a list of deliverables that are expected outputs of this task. These should be concrete and tangible.
 -->
@@ -76,3 +75,5 @@ assignees: ""
 
 **Creator**: [github username of the creator]
 **Co-Owners**: [optional - github usernames of the co-owner(s)]
+
+<!-- GITHUB_WIKI: repository/issue_template -->
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
@@ -54,7 +54,10 @@ Please mark the relevant option(s):
 - [ ] I have updated the corresponding CHANGELOG
 
 ### If Applicable Checklist
+
 - [ ] I have updated the corresponding README(s); local and/or global
 - [ ] I have added tests that prove my fix is effective or that my feature works
 - [ ] I have added, or updated, [mermaid.js](https://mermaid-js.github.io) diagrams in the corresponding README(s)
 - [ ] I have added, or updated, documentation and [mermaid.js](https://mermaid-js.github.io) diagrams in `shared/docs/*` if I updated `shared/*`README(s)
+
+<!-- GITHUB_WIKI: repository/pull_request_template -->
diff --git a/.github/workflows/wiki_sync_process.yml b/.github/workflows/wiki_sync_process.yml
@@ -0,0 +1,24 @@
+name: Publish docs to Wiki
+
+on:
+  push:
+    branches:
+      - main
+    paths:
+      - '**/*.md'
+
+env:
+  USER_TOKEN: ${{ secrets.WIKI_SYNC_SECRET }}
+  USER_NAME: profishional
+  USER_EMAIL: profishcrytpo@gmail.com
+  OWNER: ${{ github.event.repository.owner.name }}
+  REPOSITORY_NAME: ${{ github.event.repository.name }}
+
+jobs:
+  publish_docs_to_wiki:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v2
+    - name: Generate wiki
+      run: |
+        python tools/wiki_sync.py
diff --git a/README.md b/README.md
@@ -14,7 +14,8 @@ The official implementation of the [V1 Pocket Network Protocol Specification](ht
 
 ---
 
-All the links you'll need are listed below. If you'd like to contribute to the Pocket V1 Protocol, start by:
+Some relevant links are listed below, however for a the full ongoing documentation **please refer to the [wiki repository](https://github.com/pokt-network/pocket/wiki)**.
+If you'd like to contribute to the Pocket V1 Protocol, start by:
 
 1. Get up and running by reading the [Development Guide](docs/development/README.md)
 2. Find a task by reading the [Contribution Guide](docs/contributing/README.md)
@@ -72,3 +73,5 @@ All the links you'll need are listed below. If you'd like to contribute to the P
 ## License
 
 This project is licensed under the MIT License; see the [LICENSE](LICENSE) file for details.
+
+<!-- GITHUB_WIKI: home/readme -->
diff --git a/build/config/README.md b/build/config/README.md
@@ -47,3 +47,5 @@ The config and genesis files located in the `./build/config/` directory are need
 These builds currently expect four (validator) `config.json` file and a single `genesis.json` file.
 
 Until #186 is implemented, take caution when overwriting / deleting the files with different configurations.
+
+<!-- GITHUB_WIKI: build/config/readme -->
diff --git a/consensus/doc/CHANGELOG.md b/consensus/doc/CHANGELOG.md
@@ -124,3 +124,5 @@ General
 - Benchmarking via `make benchmark_sortition`
 - VRF Wrapper library in `consensus/leader_election/vrf/` of github.com/ProtonMail/go-ecvrf/ecvrf
 - Implementation of Algorand's Leader Election sortition algorithm in `consensus/leader_election/sortition/`
+
+<!-- GITHUB_WIKI: changelog/consensus_changelog -->
diff --git a/consensus/doc/PROTOCOL_STATE_SYNC.md b/consensus/doc/PROTOCOL_STATE_SYNC.md
@@ -45,25 +45,29 @@ type PeerSyncAggregate interface {
   ...
 }
 ```
+
 Using the `PeerSyncAggregate`, a Node is able to compare its local `SyncState` against that of the Global Network.
 
 ## State Sync Operation Modes
 
 State sync can be viewed as a state machine that transverses various modes the node can be in, including:
+
 * Synced Mode
 * Sync Mode
 * Server Mode
 
-The functionality of the node depends on the mode it is operating it. 
+The functionality of the node depends on the mode it is operating it.
 
 *NOTE: that the modes are not necessarily mutually exclusive (e.g. the node can be in `Server Mode` and `Synced Mode` at the same time).*
 
 ### Synced Mode
+
 If the Node is `Synced` or `localSyncState.Height == GlobalSyncMeta.Height` then the `StateSync` protocol is in `SyncedMode`.
 
 In `SyncedMode`, the Node is caught up to the latest block and relies on new blocks to be propagated via the P2P network after Validators reach consensus.
 
 ### Sync Mode
+
 If the Node is `Syncing` or `localSyncState.Height < GlobalSyncMeta.Height` then the `StateSync` protocol is in `SyncMode`.
 
 In `SyncMode`, the Node is catching up to the latest block by making `BlockRequests` to its fellow eligible peers. A peer is eligible for a `BlockRequest` if `PeerMeta.MinHeight` <= `self.MaxBlockHeight` <= `PeerMeta.MaxHeight`.
@@ -98,6 +102,7 @@ graph TD
 * `Block Stream Design` - Update the Block by Block design to stream blocks via a WebSocket from a single connectable peer.
 
 ## Research Items
+
 TODO(M5): Create issues to track and discuss these work items.
 
 How the persistence layer design of `pruning` the Merkle Tree affects `StateSync`.
@@ -117,3 +122,5 @@ How the Churn Management operations of RainTree might provide opportunities and
 `Network Peer`: Another node on the network that this node can directly communicate with, without going through a third-party server. Peers may start the connection through an `inbound` or `outbound` initialization to share and transmit data.
 
 `SyncState`: The local block state of the node vs the global network block state.
+
+<!-- GITHUB_WIKI: consensus/protocol_state_sync -->
diff --git a/docs/contributing/CONTRIBUTING.md b/docs/contributing/CONTRIBUTING.md
@@ -37,3 +37,5 @@ Congratulations, if you're reading this, you truly are committed to building one
 Well Done! You finished your milestone, make sure to follow this guide to integrate your awesome changes in the next release.
 
 [Integration Branch (Milestone) Merge Guide](contribution/merge.md)
+
+<!-- GITHUB_WIKI: guides/contributing/community_guide -->
diff --git a/docs/contributing/README.md b/docs/contributing/README.md
@@ -9,3 +9,5 @@ Once V1 is closer to mainnet launch, we will have an [extensive contribution gui
 ![](./badge.png)
 
 3. Collaborate with the team and other core contributors on Discord in the [#v1-dev channel](https://discord.com/channels/553741558869131266/986789914379186226) or [#v1-research channel](https://discord.com/channels/553741558869131266/936666517498650644).
+
+<!-- GITHUB_WIKI: guides/contributing/readme -->
diff --git a/docs/demos/iteration_3_end_to_end_tx.md b/docs/demos/iteration_3_end_to_end_tx.md
@@ -139,3 +139,5 @@ go run app/client/*.go --path_to_private_key_file=/Users/olshansky/workspace/poc
 ```bash
 make swagger-ui
 ```
+
+<!-- GITHUB_WIKI: guides/demos/iteration_three_end_to_end -->
diff --git a/docs/deps/README.md b/docs/deps/README.md
@@ -48,3 +48,5 @@ This is okay in the case of Pocket's Leader Election Algorithm because the seed
 2. The network leverages consensus messages to distribute the keys throughout the network in `O(N)`
 3. The VRF keys begin to be used for leader election at some `height (N+M)` where `M > 0`
 4. The input to the VRF for each `height (N+M')` where `M' ≥ M` will use publicly known information (e.g. appHash, byzValidators, etc..) known at `height (N+M'-1)` and therefore satisfy the security notice above.
+
+<!-- GITHUB_WIKI: guides/dependencies/readme -->
diff --git a/docs/development/FAQ.md b/docs/development/FAQ.md
@@ -4,7 +4,9 @@ A list of common issues & resolutions shared by the V1 contributors
 
 ## Avoid redundant files from iCloud backup
 
-* **Issue**: when working on MacOS with iCloud backup turned on, redundant files could be generated in GitHub projects. (e.g. `file.go` and `file 2.go`) Details can be found here in [this link](https://stackoverflow.com/a/62387243). 
+* **Issue**: when working on MacOS with iCloud backup turned on, redundant files could be generated in GitHub projects. (e.g. `file.go` and `file 2.go`) Details can be found here in [this link](https://stackoverflow.com/a/62387243).
 * **Solution**: adding `.nosync` as suffix to the workspace folder, e.g. `pocket.nosync`. Alternative, working in a folder that iCloud doesn't touch also works.
 
-_NOTE: Consider turning of the `gofmt` in your IDE to prevent unexpected formatting_
+_NOTE: Consider turning of the `gofmt` in your IDE to prevent unexpected formatting_
+
+<!-- GITHUB_WIKI: guides/development/FAQ -->
diff --git a/docs/development/README.md b/docs/development/README.md
@@ -61,11 +61,12 @@ Software:
 Generate local files
 
 ```bash
-$ git clone git@github.com:pokt-network/pocket.git && cd pocket
-$ make develop_start
+git clone git@github.com:pokt-network/pocket.git && cd pocket
+make develop_start
 ```
 
 Optionally activate changelog pre-commit hook
+
 ```bash
 cp .githooks/pre-commit .git/hooks/pre-commit
 chmod +x .git/hooks/pre-commit
@@ -77,13 +78,13 @@ unless the CHANGELOG is updated._
 ### View Available Commands
 
 ```bash
-$ make
+make
 ```
 
 ### Running Unit Tests
 
 ```bash
-$ make test_all
+make test_all
 ```
 
 Note that there are a few tests in the library that are prone to race conditions and we are working on improving them. This can be checked with `make test_race`.
@@ -95,19 +96,19 @@ Note that there are a few tests in the library that are prone to race conditions
 1. Delete any previous docker state
 
 ```bash
-$ make docker_wipe
+make docker_wipe
 ```
 
 2. In one shell, run the 4 nodes setup:
 
 ```bash
-$ make compose_and_watch
+make compose_and_watch
 ```
 
 4. In another shell, run the development client:
 
 ```bash
-$ make client_start && make client_connect
+make client_start && make client_connect
 ```
 
 4. Check the state of each node:
@@ -170,6 +171,10 @@ Pocket
 ├── Makefile         # [to-be-deleted] The source of targets used to develop, build and test
 ```
 
+## Maintaining Documentation
+
+Markdown files based on the following command `find . -name "*.md" | grep -v -e "vendor" -e "app"` are added to the [repository wiki](https://github.com/pokt-network/pocket/wiki). To organize the wiki a comment is added to the end of the files e.g. you can find one at the end of this file `<!-- GITHUB_WIKI: guides/development/readme -->`. The structure of the comment is to indicate the category (guides), subcategories (development) and file name (readme). Looking at the current wiki choose the right placement for the new markdown or simply create a new category.
+
 ### Linters
 
 We utilize `golangci-lint` to run the linters. It is a wrapper around a number of linters and is configured to run many at once. The linters are configured to run on every commit and pull request via CI, and all code issues are populated as GitHub annotations to let developers and reviewers easily locate an issue.
@@ -210,3 +215,5 @@ The official documentation includes a list of different linters and their config
 We can write custom linters using [`go-ruleguard`](https://go-ruleguard.github.io/). The rules are located in the [`build/linters`](../../build/linters) directory. The rules are written in the [Ruleguard DSL](https://github.com/quasilyte/go-ruleguard/blob/master/_docs/dsl.md), if you've never worked with ruleguard in the past, it makes sense to go through [introduction article](https://quasilyte.dev/blog/post/ruleguard/) and [Ruleguard by example tour](https://go-ruleguard.github.io/by-example/).
 
 Ruleguard is run via `gocritic` linter which is a part of `golangci-lint`, so if you wish to change configuration or debug a particular rule, you can modify the `.golangci.yml` file.
+
+<!-- GITHUB_WIKI: guides/development/readme -->
diff --git a/docs/learning/README.md b/docs/learning/README.md
@@ -195,3 +195,5 @@ We do not consider them bad, but time is limited so it is important to focus on
 
 - [Blockchains Meet Distributed Hash Tables: Decoupling Validation from State Storage](https://arxiv.org/abs/1904.01935)
   - An "extended abstract" of how Authenticated Data Structures (i.e. Merkle Trees) could be _"sharded"_ across nodes using Distributed Hash Tables (DHTs) to reduce the state required to be maintained and synched by each node.
+
+<!-- GITHUB_WIKI: guides/learning/readme -->
diff --git a/docs/releases/README.md b/docs/releases/README.md
@@ -43,3 +43,5 @@ Extended images with additional tooling are built to aid in troubleshoot and deb
 ## [TODO] Magefile build system
 
 Once the V1 implementation reaches the stage of testable binaries, we are looking to use [Mage](https://magefile.org/) which is being tracked in [pocket/issues/43](https://github.com/pokt-network/pocket/issues/43) that'll inject a version with the `-version` flag.
+
+<!-- GITHUB_WIKI: guides/releases/readme -->
diff --git a/docs/roadmap/README.md b/docs/roadmap/README.md
@@ -42,12 +42,12 @@ gantt
 
 ### [M1. Pocket PoS (Proof of Stake)](https://github.com/pokt-network/pocket/milestone/7)
 
-#### Goals:
+#### Goals
 
 - Basic LocalNet development environment
 - Tendermint Core equivalent for a proof-of-stake blockchain custom-built for Pocket
 
-#### Non-goals:
+#### Non-goals
 
 - Pocket specific utility
 - Devnet infrastructure
@@ -63,15 +63,15 @@ gantt
 
 ### [M3. Pocket RoS (Relay or Slash)](https://github.com/pokt-network/pocket/milestone/15)
 
-#### Goals:
+#### Goals
 
 - Pocket Network utility-specific business logic including:
   - Devnet supporting ETH relays
   - Initial implementation of reward distribution, report cards, etc...
 
 ### M4. [Pocket CoS (Cloud of Services)](https://github.com/pokt-network/pocket/milestone/20)
 
-#### Goals:
+#### Goals
 
 *Note: more details will be added to this milestone in the future*
 
@@ -81,7 +81,7 @@ gantt
 
 ### [M5. Pocket IoS (Innovate or Skip)](https://github.com/pokt-network/pocket/milestone/16)
 
-#### Goals:
+#### Goals
 
 *Note: more details will be added to this milestone in the future*
 
@@ -91,7 +91,7 @@ gantt
 
 ### [M6. Pocket FoS (Finish or Serve)](https://github.com/pokt-network/pocket/milestone/18)
 
-#### Goals:
+#### Goals
 
 *Note: more details will be added to this milestone in the future*
 
@@ -103,3 +103,5 @@ gantt
 ## M7. Pocket NoS (North Star)
 
 Shoot for the ✨ and we will land on the 🌕
+
+<!-- GITHUB_WIKI: guides/roadmap/readme -->
diff --git a/p2p/CHANGELOG.md b/p2p/CHANGELOG.md
@@ -40,7 +40,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [0.0.0.7] - 2022-10-24
 
 - Updated README to reference the python simulator as a learning references and unit test generation tool
-- Added a RainTree unit test for 12 nodes using the simulator in https://github.com/pokt-network/rain-tree-sim/blob/main/python
+- Added a RainTree unit test for 12 nodes using the simulator in <https://github.com/pokt-network/rain-tree-sim/blob/main/python>
 
 ## [0.0.0.6] - 2022-10-20
 
@@ -86,3 +86,5 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [0.0.0.0] - 2022-06-16
 
 - RainTree first iteration in Pre2P module (no cleanup or redundancy)
+
+<!-- GITHUB_WIKI: changelog/p2p_changelog -->
diff --git a/p2p/README.md b/p2p/README.md
@@ -124,3 +124,5 @@ You can read the documentation in the [python simulator](https://github.com/pokt
 
 - **Deterministic Private Key Generation**: Since RainTree is dependant on the lexicographic order of the addresses, the generation of the private keys (and in turn the public keys and addresses) is important and cannot be randomized for the time being.
 - **Variable Coupling**:There is an implicit coupling between `validatorId`, `serviceUrl` and `genericParam` that requires understanding of the codebase. Reach out to @olshansk or @andrewnguyen22 for clarity on this.
+
+<!-- GITHUB_WIKI: p2p/readme -->
diff --git a/persistence/CHANGELOG.md b/persistence/CHANGELOG.md
@@ -140,7 +140,7 @@ Deprecate PrePersistence
 
 ## [0.0.0.1] - 2022-07-05
 
-Pocket Persistence 1st Iteration (https://github.com/pokt-network/pocket/pull/73)
+Pocket Persistence 1st Iteration (<https://github.com/pokt-network/pocket/pull/73>)
 
 # Added
 
@@ -154,3 +154,5 @@ Pocket Persistence 1st Iteration (https://github.com/pokt-network/pocket/pull/73
 - Update to the Persistence module interface to enable historical height queries
 - Library / infrastructure for persistence unit fuzz testing
 - Tests triggered via `make test_persistence`
+
+<!-- GITHUB_WIKI: changelog/persistence_changelog_1 -->
diff --git a/persistence/docs/CHANGELOG.md b/persistence/docs/CHANGELOG.md
@@ -84,15 +84,15 @@ TODO: consolidate `persistence/docs/CHANGELOG` and `persistence/CHANGELOG.md`
 **Main persistence module changes:**
 
 - Split `ConnectAndInitializeDatabase` into `connectToDatabase` and `initializeDatabase`
-    - This enables creating multiple contexts in parallel without re-initializing the DB connection
+  - This enables creating multiple contexts in parallel without re-initializing the DB connection
 - Fix the SQL query used in `SelectActors`, `SelectAccounts` & `SelectPools`
-    - Add a generalized unit test for all actors
+  - Add a generalized unit test for all actors
 - Remove `NewPersistenceModule` and an injected `Config` + `Create`
-    - This improves isolation a a “injection-like” paradigm for unit testing
+  - This improves isolation a a “injection-like” paradigm for unit testing
 - Change `SetupPostgresDocker` to `SetupPostgresDockerPersistenceMod`
-    - This enables more “functional” like testing by returning a persistence module and avoiding global testing
+  - This enables more “functional” like testing by returning a persistence module and avoiding global testing
       variables
-    - Only return once a connection to the DB has been initialized reducing the likelihood of test race conditions
+  - Only return once a connection to the DB has been initialized reducing the likelihood of test race conditions
 - Implemented `NewReadContext` with a proper read-only context
 - Add `ResetContext` to the persistence module and `Close` to the read context
 
@@ -113,7 +113,7 @@ TODO: consolidate `persistence/docs/CHANGELOG` and `persistence/CHANGELOG.md`
 - Added `genesis_time` to `consensus_genesis_state`
 - Added `chainID` to `consensus_genesis_state`
 - Added `max_block_bytes` to `consensus_genesis_state`
-- Added `accounts` and` pools to utility_genesis_state`
+- Added `accounts` and`pools to utility_genesis_state`
 - Added `validators` to `utility_genesis_state`
 - Added `applications` to `utility_genesis_state`
 - Added `service_nodes` to `utility_genesis_state`
@@ -130,7 +130,7 @@ TODO: consolidate `persistence/docs/CHANGELOG` and `persistence/CHANGELOG.md`
 - Added config and genesis generator to build package
 - Deprecated old build files
 - Use new config and genesis files for make `compose_and_watch`
-- Use new config and genesis files for make `client_start && `make client_connect`
+- Use new config and genesis files for make `client_start &&`make client_connect`
 
 ## [0.0.0.2] - 2022-08-03
 
@@ -149,17 +149,19 @@ Deprecate PrePersistence
 
 ## [0.0.0.1] - 2022-07-05
 
-Pocket Persistence 1st Iteration (https://github.com/pokt-network/pocket/pull/73)
+Pocket Persistence 1st Iteration (<https://github.com/pokt-network/pocket/pull/73>)
 
 - Base persistence module implementation for the following actors: `Account`, `Pool`, `Validator`, `Fisherman`
   , `ServiceNode`, `Application`
 - Generalization of common protocol actor behvaiours via the `ProtocolActor` and `BaseActor` interface and
   implementation
 - A PostgreSQL based implementation of the persistence middleware including:
-    - SQL query implementation for each actor
-    - SQL schema definition for each actor
-    - SQL execution for common actor behaviours
-    - Golang interface implementation of the Persistence module
+  - SQL query implementation for each actor
+  - SQL schema definition for each actor
+  - SQL execution for common actor behaviours
+  - Golang interface implementation of the Persistence module
 - Update to the Persistence module interface to enable historical height queries
 - Library / infrastructure for persistence unit fuzz testing
 - Tests triggered via `make test_persistence`
+
+<!-- GITHUB_WIKI: changelog/persistence_changelog_2 -->