New eventually-consistent staging (for local dev and gno.land)

[moul]

I suggest the creation of a special staging/testnet that resets regularly while retaining past transactions. After each reset, it will update (git pull), rebuild, and reload the examples/ directory, attempting to replay previous raw transactions. If any issues arise, it will continue without interruption. This approach combines the advantages of a constantly refreshed test environment with the consistency of older ones.

This testnet not only provides an ideal space for iterative testing but also facilitates experimentation with chain upgradability. It serves as a playground for understanding and testing migrations, which is a crucial upcoming feature for our platform.

From the perspective of a contract developer, if my new concepts don't gain immediate traction, I can simply wait a day and try again. As long as the API of my previous contracts remains consistent, most transactions are likely to remain the same, even if there are changes behind the scenes. This can be particularly beneficial for features like our r/gnoland/blog, which is currently challenging to update and may encounter URL issues during potential migrations. I believe it can also serve as an excellent platform for a more dynamic r/demo/boards.

To do

• Naming: Decide on a distinctive name for the new testnet. Consider rebranding or removing the current staging. -> gno.land
• History Storage: Develop a fast method for storing and retrieving transaction histories.
• Script Updates: Refine infrastructure scripts to cater to the unique needs of the new testnet.
• Tools to build valid genesis files (PoC for Genesis Manipulation Libraries and Tools #1189)

[dongwon8247]

Thank you, @moul, for the proposal. I like the idea because this new testnet concept will provide a stable testing environment and access to the latest features from master. It will also benefit contract developers by allowing them to continue development on a single testnet (not having to migrate to a next testnet) while retaining archived transaction histories for debugging.

I have a couple of questions:

1. What would be the time frame for a regular reset? I think it would be how often we want to sync the testnet with master. Maybe a month? but it should be done in a regular time frame so that indexers (e.g. Gnoscan) can expect a reset and respond to changes (e.g. Archive transactions and reset UIs to new block data in Gnoscan. **Archived transactions will remain searchable by transaction hashes).
2. When reset, will past transactions be included in the next genesis, or will they be saved separately for reference without inclusion in the genesis?
3. When reset, will the block number start from genesis every time? or will the testnet remember the last block and continue it there?

[albttx]

I like the idea!

Can we move to a something like a semver for the chain, and deploy upgrades to every major/minor version?

For the nodes, I can manage an infra with

• 1 validator
• 1-x RPCs
• 1 history node (with daily / weekly snapshots)

For the naming, I propose to keep the test{1...x} for other testnet and call this one testnet-x

[moul]

Let's adapt the existing staging.gno.land to make it eventually persistent.

Edit: better idea; let's make gno.land this new special testnet. Depends on #1176

[moul]

Bonuses:

• Utilize https://github.com/gnolang/tx-exports for storing and restoring.
• Subsequently, we will migrate this new testnet to "gno.land" (dependent on feat: portal loop's main contracts and gnoweb improvements #1176).
• Enable local execution of the "deploy a testnet with the latest git + try replaying history" functionality.

Assigned @zivkovicmilos so he can lead the initiative and make sure to take in consideration his work on #546.

[moul]

Hey @zivkovicmilos, I've updated the original meta issue #1108 (portal loop) and created new related sub-issues:

• META Gnoweb improvements #1187 (Gnoweb improvements) that can be tackled by anyone.
• PoC for Genesis Manipulation Libraries and Tools #1189 (Genesis manipulation libraries and tools) which could have been a part of your own issue, but I wanted to specify multiple things regarding gnosdk. It's a big enough task to have its own issue, but in my opinion, it's a sub-issue of your meta issue.

I apologize for any confusion, but the documentation is being progressively improved. Please feel free to continue. I have made the important bootstrap. Now, I suggest you continue drafting the plans with the team. Count on me for reviewing and providing feedback if you need it.

Edit: I suggest prioritizing local development improvement instead of solely focusing on production. Check out #1174. Consider reaching out to @gfanton for assistance.

[zivkovicmilos]

@moul
We have created a tool that aims to replace gnotxsync, with all the bells and whistles of gnotxsync and more:
https://github.com/gnolang/tx-archive

This tool also has the ability to support new sources of transactions (ex. the upcoming indexer), so it should be a good foundation. It falls under the History Storage: Develop a fast method for storing and retrieving transaction histories. in your top-level issue

[zivkovicmilos]

I have also created the issue that @ajnavarro has taken up in parallel for this effort, so we can track in the project board:
#1204

The idea is that we can use these Lego blocks to build out complicated functionality. The base functionality is:

• start node
• stop node
• restore txs
• backup txs

After #1203 and #1189 , the Lego blocks will also be:

• create genesis
• modify genesis

[moul]

By the way, @zivkovicmilos, I just recalled that I opened this issue on Hackerspace: gnolang/hackerspace#23.