Merge branch 'development' into development_tfgrid_check_storage_state

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,32 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+## Describe the bug
+
+Thanks for taking the time to fill out this bug report! Please add a clear and concise description of what the bug is.
+
+## To Reproduce
+
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+## Expected 
+
+A clear and concise description of what you expected to happen.
+
+## Screenshots
+
+If applicable, add screenshots to help explain your problem.
+
+## Additional context
+
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,23 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+## Is your feature request related to a problem? Please describe
+
+Thanks for taking the time to fill out this feature request. Please add a clear and concise description of what the problem is. Ex. I'm always frustrated when \[...]
+
+## Describe the solution you'd like
+
+A clear and concise description of what you want to happen.
+
+## Describe alternatives you've considered
+A clear and concise description of any alternative solutions or features you've considered.
+
+## Additional context
+Add any other context or screenshots about the feature request here.
+

.github/ISSUE_TEMPLATE/question.md

@@ -0,0 +1,21 @@
+---
+name: Question
+about: Ask a question about this project
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+## What is your question?
+
+Ex. How I can \[...]
+
+## What have you tried so far?
+
+If applicable, add the approaches that you have already tried.
+
+## Screenshots
+
+If applicable, add screenshots to help explain your question.
+

.github/pull_request_template.md

@@ -0,0 +1,25 @@
+## Description
+
+Please include a summary of the changes and the related issue. Please also include relevant motivation and context, including:
+
+- What does this PR do?
+- Why are these changes needed?
+- How were these changes implemented and what do they affect?
+
+## Related Issues:
+
+Use [Github semantic linking](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword) to address any open issues this PR relates to or closes.
+
+- Fixes # (issue number, if applicable)
+
+- Closes # (issue number, if applicable)
+
+## Checklist:
+
+Please delete options that are not relevant.
+
+- [ ] My change requires a change to the documentation and I have updated it accordingly
+- [ ] My change requires storage migration and I have included and tested it following fork off and try_runtime instructions.
+- [ ] I have added tests to cover my changes.
+- [ ] I followed the **[Release](https://github.com/threefoldtech/tfchain/blob/development/docs/production/releases.md)** document.
+- [ ] My commits follow this [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) guide.

.github/workflows/030_create_release.yaml

@@ -91,7 +91,7 @@ jobs:
         id: create_release
         uses: softprops/action-gh-release@v1
         env:
-          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          GITHUB_TOKEN: ${{ secrets.RELEASE_KEY }}
         with:
           tag_name: ${{ github.ref }}
           name: Release ${{ github.ref_name }}

activation-service/.github/workflows/publish_container_image.yml → .github/workflows/040_publish_activation_service_image.yml

@@ -1,6 +1,7 @@
-name: Publish Docker image
+name: Publish activation service image
 
 on:
+  workflow_dispatch:
   release:
     types: [published]
 
@@ -26,13 +27,14 @@ jobs:
         id: meta
         uses: docker/metadata-action@v4
         with:
-          images: ghcr.io/${{ github.repository }}
+          images: ghcr.io/threefoldtech/tfchain_activation_service
           tags: |
             type=semver,pattern={{version}}
 
       - name: Build and push Docker image
-        uses: docker/build-push-action@v3
+        uses: docker/build-push-action@v4
         with:
+          context: ./activation-service
           push: true
           tags: ${{ steps.meta.outputs.tags }}
           labels: ${{ steps.meta.outputs.labels }}

.github/workflows/050_publish_tfchainstellar_bridge_image.yml

@@ -0,0 +1,41 @@
+name: Publish bridge image
+
+on:
+  workflow_dispatch:
+  release:
+    types: [published]
+
+jobs:
+  build-and-push:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
+
+    steps:
+      - name: Checkout the repo
+        uses: actions/checkout@v3
+
+      - name: Log in to the Container registry
+        uses: docker/login-action@v2.1.0
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Extract metadata for Docker
+        id: meta
+        uses: docker/metadata-action@v4
+        with:
+          images: ghcr.io/threefoldtech/tfchain_stellar_bridge
+          tags: |
+            type=semver,pattern={{version}}
+
+      - name: Build and push Docker image
+        uses: docker/build-push-action@v4
+        with:
+          file: ./bridge/tfchain_bridge/Dockerfile
+          context: .
+          push: true
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}

CODEOWNERS

@@ -1 +1,2 @@
-* @DylanVerstraete @brandonpille @renauter @robvanmieghem @leesmet
+* @renauter @leesmet @sameh-farouk
+

activation-service/helm/tfchainactivationservice/Chart.yaml

@@ -2,5 +2,5 @@ apiVersion: v2
 name: tfchainactivationservice
 description: TFchain account activation funding service
 type: application
-version: 0.1.3
-appVersion: "2.5.0-rc7"
+version: 2.6.0-rc1
+appVersion: '2.6.0-rc1'

activation-service/package.json

@@ -1,6 +1,6 @@
 {
   "name": "substrate-funding-service",
-  "version": "0.0.1",
+  "version": "2.6.0-rc1",
   "description": "Substrate funding service",
   "main": "index.js",
   "scripts": {
@@ -36,4 +36,4 @@
     "pino-pretty": "^5.0.2",
     "standard": "^16.0.3"
   }
-}
+}

bridge/README.md

@@ -23,6 +23,13 @@ Bridge account on testnet `GA2CWNBUHX7NZ3B5GR4I23FMU7VY5RPA77IUJTIXTTTGKYSKDSV6L
 
 Can be interacted with on: https://dashboard.test.grid.tf
 
+### QAnet
+
+Bridget account on qanet `GAQH7XXFBRWXT2SBK6AHPOLXDCLXVFAKFSOJIRMRNCDINWKHGI6UYVKM`
+
+Can be interacted with on: https://dashboard.qa.grid.tf
+
+
 ### Devnet
 
 Bridge account on devnet: `GDHJP6TF3UXYXTNEZ2P36J5FH7W4BJJQ4AYYAXC66I2Q2AH5B6O6BCFG`

bridge/docs/bridging.md

@@ -6,8 +6,8 @@ This document will explain how you can transfer TFT from TF Chain to Stellar and
 
 ## Prerequisites
 
-- Threefold Connect application or any other Stellar wallet
-- A running bridge and bridge wallet address
+*   Threefold Connect application or any other Stellar wallet
+*   A running bridge and bridge wallet address
 
 ## Stellar to TF Chain
 
@@ -17,20 +17,116 @@ Transfer the TFT from your Stellar wallet to bridge wallet address that you conf
 
 We also enabled deposits to TF Grid objects. Following objects can be deposited to:
 
-- Twin
-- Farm
-- Entity
-- Node
+*   Twin
+*   Farm
+*   Entity
+*   Node
 
-To deposit to any of these objects, a memo text in format `object_objectID` must be passed on the deposit to the bridge wallet. Example: `twin_1`. 
+To deposit to any of these objects, a memo text in format `object_objectID` must be passed on the deposit to the bridge wallet. Example: `twin_1`.
 
 To deposit to a TF Grid object, this object **must** exists. If the object is not found on chain, a refund is issued.
 
 ## TF Chain to Stellar
 
-Browse to https://polkadot.js.org/apps/?rpc=ws%3A%2F%2F127.0.0.1%3A9944#/extrinsics , select tftBridgeModule and extrinsic: `swap_to_stellar`. Provide your stellar target address and amount and sign it with your account holding the tft balance.
+Browse to https://polkadot.js.org/apps/?rpc=wss%3A%2F%2Ftfchain.grid.tf#/extrinsics (for mainnet), select tftBridgeModule and extrinsic: `swap_to_stellar()`. Provide your stellar target address and amount and sign it with your account holding the tft balance.
 Again, a withdrawfee of 1 TFT will be taken, so make sure you send a larger amount as 1 TFT.
 
 The amount withdrawn from TF Chain will be sent to your Stellar wallet.
 
-Example: ![swap_to_stellar](swap_to_stellar.png)
+Example: ![swap\_to\_stellar](swap_to_stellar.png)
+
+## Deeper look at how the TFChain Bridge works
+
+### STELLAR -> TFCHAIN (Lock-and-Mint flow)
+
+In this section, we look into the details of transferring TFT from a Stellar Account to a TFChain Account.
+
+1.  A transaction is received on the bridge Stellar account (aka. vault) and noticed by bridge validators (deamons).
+
+2.  Each time such transaction event is received by a bridge validator it undergoes some validation. If it fails, a refund is issued (sent back to the stellar source account). Here we assume that the validation has passed, but we will examine the refund flow in the next section.
+
+3.  The first bridge validator reporting the transaction will propose a mint by calling `propose_or_vote_mint_transaction()` extrinsic on the TFTBridgeModule in TFChain. This extrinsic inserts a new `MintTransaction` in `MintTransactions` storage that includes the `amount`, `target`, `block`, `votes`, and emits a `MintTransactionProposed` event. The mint is considered processed by the bridge side at this point.
+
+4.  Other bridge validators that report the transaction later will only add their votes for that proposal by calling same TFChain extrinsic. Since the `MintTransaction` already exists in `MintTransactions` storage, the extrinsic will increment the `votes` count for the specified `MintTransaction` and emit `MintTransactionVoted` event.
+
+5.  From the TFChain side, if the majority (more than the half) of bridge validators agree on the transaction, tokens are minted to the target address. This check happens every time the `propose_or_vote_mint_transaction()` extrinsic is executed by validator call. Then, the transaction is removed from bridge pallet `MintTransactions` storage and added to `ExecutedMintTransactions`. Finally, a `MintCompleted` event is emitted.
+
+#### Overview of the TFChain Minting events
+
+1.  `tftBridgeModule.MintTransactionProposed`: A bridge validator proposed a mint transaction after being the first to report a stellar deposit.
+2.  `tftBridgeModule.MintTransactionVoted`: Other bridge validators reported same stellar deposit and voted for the mint proposal.
+3.  `tftBridgeModule.MintCompleted`: Enough bridge validator votes was collected and the tokens was successfully minted to the target address.
+
+#### When a Refund-on-Stellar occurs?
+
+A refund on Stellar occurs when one of the following conditions is met:
+
+*   The deposited amount is lower than the deposit fee.
+*   The memo message is empty.
+*   The transaction contains more than one payment.
+*   The memo is not formatted correctly.
+*   The grid type is not supported (not one of grid, farm, node, or entity) or not found.
+
+### STELLAR -> TFCHAIN (Refund-on-Stellar flow)
+
+In this section, we look into the details of what happens when the a Stellar deposit can not be processed due to a validation problem.
+
+1.  A transaction is received on the bridge Stellar account (aka. vault) and noticed by bridge validators (deamons).
+
+2.  Each time such transaction event is received by a bridge validator it undergoes some validation. Here, we assume that the validation has failed because of one of the violations mentioned in the previous section so a refund flow is initiated.
+
+3.  The first bridge validator reporting a violation will initiate the refund by calling TFTBridgeModule `create_refund_transaction_or_add_sig()` extrinsic to propose a `RefundTransaction`, to store the details in `RefundTransactions` storage map alongside with its signature and to emit `RefundTransactionsignatureAdded` and `RefundTransactionCreated` events.
+
+4.  Other bridge validators that report the transaction later also provides their signature for that refund transaction proposal by calling same extrinsic.
+
+5.  If the majority (more than the half) of bridge validators provided their signature for a refund transaction, a `RefundTransactionReady` event is emitted as well. This check happens every time the `create_refund_transaction_or_add_sig()` extrinsic is executed by validator call.
+
+6.  The first bridge validator reporting the `RefundTransactionReady` event will handle it and query TFChain storage for the `RefundTransaction` details and the validators’ signatures. It will create a multi-signatures Stellar transaction with a [MEMO](https://developers.stellar.org/docs/encyclopedia/memos) of `RETURN` type containing the hash of the refunded transaction and submit it to Stellar network. If submitted successfully, it will call `set_refund_transaction_executed()` extrinsic (which removes the `RefundTransaction` from the `RefundTransactions` storage and adds it to `ExecutedRefundTransactions`) then emit `RefundTransactionProcessed` event.
+
+#### Overview of the TFChain Refund events
+
+1.  `tftBridgeModule.RefundTransactionCreated`: A bridge validator proposed a Refund-on-Stellar transaction after being the first to report a stellar deposit with invalid or missing cross-chain transfer information.
+2.  `tftBridgeModule.RefundTransactionsignatureAdded`: Other bridge validators reported same stellar deposit and provided signature for the refund proposal.
+3.  `tftBridgeModule.RefundTransactionReady`: Enough validators signatures were collected and stored so from now it is possible to submit the proposed stellar refund transaction.
+4.  `tftBridgeModule.RefundTransactionProcessed`: A bridge validator called `set_refund_transaction_executed()` extrinsic with a proof that the proposed stellar refund transaction was executed successfully on stellar network.
+
+### TFCHAIN -> STELLAR (Burn-and-Withdraw flow)
+
+Now, we look into the details of transferring TFT from a TFChain account to a Stellar account. On TFChain network we burn TFT and on Stellar network we withdraw TFT from TFChain.
+
+1.  To withdraw your asset back to Stellar, the TFTBridgeModule's `swap_to_stellar()` extrinsic in TFChain must be called with an amount to burn (on the TFChain side) and a Stellar account ID to receive the equivalent TFT amount (on the Stellar network side).
+
+2.  The call validates the target Stellar account ID and ensures that you have enough balance in the source account. If so, it burns the amount and transfer fees to feeAccount, increments the `BurnTransactionId` in the TFTBridgeModule storage, stores data about the transaction with empty signatures placeholder, adds it to `BurnTransactions` with the `burnId` as key, and emits `BurnTransactionCreated` event. This event contains `burn_id`, `source` account, `target` Stellar address, and burn `amount`.
+
+3.  The bridge validators are listening to this event. They extract the `burnId` and other transaction parameters, validate the Stellar address (tokens could be refunded/minted back on TFChain at this step if validation failed), then construct signed stellar transaction and extract the signature (note, the transaction can not be submitted yet to stellar network). They then call `propose_burn_transaction_or_add_sig()` extrinsic which fills their signatures and the bridge account sequence number in the `BurnTransaction` in storage that matches specified `burnId`. When this call executed, the `BurnTransactionSignatureAdded` event is emitted.
+
+4.  If the majority (more than the half) of bridge validators provided their signature for the transaction, a `BurnTransactionReady` event is emitted as well. This check happens every time the `propose_burn_transaction_or_add_sig()` extrinsic is executed by validator call.
+
+5.  The bridge will handle the event and query TFChain storage for the `BurnTransaction` details and the validators’ signatures. It will create a multi-signatures Stellar transaction and submit it to Stellar network. If submitted successfully, it will call `set_burn_transaction_executed()` extrinsic (which removes the `BurnTransaction` from the `BurnTransactions` storage and adds it to `ExecutedBurnTransactions`) then emit `BurnTransactionProcessed` event.
+
+#### Overview of the TFChain Burning events
+
+1.  `tftBridgeModule.BurnTransactionCreated`: A swap from TFChain to stellar was initiated by a call to `swap_to_stellar()` extrinsic.
+2.  `tftBridgeModule.BurnTransactionSignatureAdded`: A bridge validator handled BurnTransactionCreated TFChain event and submitted its signature for the proposed stellar transaction.
+3.  `tftBridgeModule.BurnTransactionReady`: Enough validators signatures were collected and stored so from now it is possible to submit the proposed stellar withdraw transaction.
+4.  `tftBridgeModule.BurnTransactionProcessed`: A bridge validator was the first to call `set_burn_transaction_executed()` extrinsic and the proposed stellar withdraw transaction was executed successfully on stellar network.
+
+#### When a Refund-on-TFChain occurs?
+
+A refund on TFChain is initiated when either of the following conditions is met:
+
+*   Account information cannot be retrieved from the Stellar network.
+*   The account has no trust line to TFT tokens or has a deleted one (TFT balance limit is `0`).
+
+### TFChain Retry mechanism
+
+We didn't mentioned yet a few TFChain event related to the flows discussed above, these events are:
+
+*   `tftBridgeModule.BurnTransactionExpired`: from TFCHAIN -> STELLAR burn flow, certain number of TFChain blocks passed without a `BurnTransaction` being noticed and signed by the majority of bridge validators.
+*   `tftBridgeModule.RefundTransactionExpired`: from STELLAR -> TFCHAIN refund flow, certain number of TFChain blocks passed without a `RefundTransaction` being noticed and signed by the majority of bridge validators.
+
+These expired events are typically the result of an outage of one or more bridge validators. We will explain why.
+
+TFChain has a retry mechanism built into its runtime that takes into account possible bridge validator outages. If a certain number of TFChain blocks pass without a `BurnTransaction` or `RefundTransaction` being noticed and signed by the majority of bridge validators, the stored transaction signatures are reset and a `BurnTransactionExpired` or `RefundTransactionExpired` event is emitted.
+
+These events will continue to occur until the unavailable bridge validators come back online and handle the expired events as it gets re-emitted.

bridge/docs/development.md

@@ -6,8 +6,8 @@ See [installing](./install.md)
 
 ## Prerequisites:
 
-- Install and run [tfchain](https://github.com/threefoldtech/tfchain/blob/development/docs/development/development.md)
-- Install [stellar-utils-tool](https://github.com/threefoldfoundation/tft/tree/main/bsc/bridges/stellar/utils)
+- Install and run [tfchain](https://github.com/threefoldtech/tfchain/blob/development/docs/development)
+- Install [stellar-utils-tool](https://github.com/threefoldfoundation/tft/tree/main/tools/bridgegen)
 
 ## Local single node development
 

bridge/docs/install.md

@@ -13,8 +13,8 @@ echo export PATH=\"\$PATH:\$GOPATH/bin\" >> ~/.bash_profile
 ### Get Source Code
 
 ```sh
-git clone https://github.com/threefoldtech/tfchain_tft_bridge.git
-cd tfchain_tft_bridge/tfchain_bridge
+git clone https://github.com/threefoldtech/tfchain.git
+cd bridge/tfchain_bridge
 ```
 
 ### Compile

bridge/docs/multinode.md

@@ -4,7 +4,7 @@ In this document we will explain how you can setup a local multinode instance of
 
 ## Step 1: Run tfchain
 
-See [tfchain](https://github.com/threefoldtech/tfchain/blob/development/docs/development/development.md)
+See [tfchain](https://github.com/threefoldtech/tfchain/blob/development/docs/development)
 
 Create a twin on your local chain: