Skip to content

Commit

Permalink
Merge pull request #981 from multiversx/remove-GCP-archives-download
Browse files Browse the repository at this point in the history
Remove option to download archives from GCP
  • Loading branch information
danidrasovean authored Sep 24, 2024
2 parents e1ab4bc + b2c899f commit 956d3e0
Showing 1 changed file with 4 additions and 34 deletions.
38 changes: 4 additions & 34 deletions docs/integrators/deep-history-squad.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,46 +117,16 @@ Apart from the flag mentioned above, the setup of a deep-history observer is ide
Never attach a non-pruned database to a regular observer (i.e. that does not have the above **operation-mode**) - unless you are not interested into the deep-history features. The regular observer irremediably removes, trucates and prunes the data (as configured, for storage efficiency).
:::

Now that we have finished with the installation part, we can proceed with populating the database from a non-pruned database archive. There are two options here:
- Download non-pruned database
- Reconstruct non-pruned database
Now that we have finished with the installation part, we can proceed to populate the non-pruned database. There are two options here:
- Reconstruct non-pruned database (recommended).
- Download non-pruned database (we can provide archives for the required epochs, on request).

[comment]: # (mx-context-auto)

## Downloading non-pruned database

Archives supporting historical lookup are available to download from a Google Cloud Storage [bucket](https://console.cloud.google.com/storage/browser/multiversx-deep-history-archives-mainnet).

In order to avoid unintentional downloads and promote careful fetching of archives, we've enabled the [requester pays](https://cloud.google.com/storage/docs/requester-pays) feature on the bucket that holds the deep-history archives for mainnet.

### Requirements

1. **Google Cloud Platform Account**: An account on Google Cloud Platform with billing enabled is required. Find out how you can manage your billing account and modify your project here:
- https://cloud.google.com/billing/docs/how-to/manage-billing-account
- https://cloud.google.com/billing/docs/how-to/modify-project

2. **Google Cloud SDK**: The Google Cloud SDK includes the `gcloud` command-line tool, which you'll use to interact with Google Cloud Storage. In order to install it, please follow the instructions provided on the [Google Cloud SDK webpage](https://cloud.google.com/sdk/docs/install).

### Downloading archives

Once you have the Google Cloud SDK installed and you're [authenticated](https://cloud.google.com/docs/authentication/gcloud), you can download archives from the Google Cloud Storage bucket using the `gcloud storage cp` command.

Here's an example command that downloads an archive from the `multiversx-deep-history-archives-mainnet` bucket:
```
gcloud storage cp gs://multiversx-deep-history-archives-mainnet/shard-0/Epoch_01200.tar ~/DOWNLOAD_LOCATION --billing-project=BILLING_PROJECT
```
Replace **BILLING_PROJECT** with the name of your billing project and **~/DOWNLOAD_LOCATION** with the directory where the archives should be downloaded.

The following example will download epochs starting with Epoch_01000.tar up to Epoch_01300.tar, for a billing project called **multiversx**:
```
gcloud storage cp gs://multiversx-deep-history-archives-mainnet/shard-0/Epoch_0{1000..1300}.tar ~/Downloads/ --billing-project=multiversx
```

[comment]: # (mx-context-auto)

## Reconstructing non-pruned databases

An alternative to downloading the non-pruned history is to reconstruct it locally (on your own infrastructure).
The recommended method for populating a non-pruned database is to reconstruct it locally (on your own infrastructure).

There are also two options for reconstructing a non-pruned database:
- Based on the **[import-db](/validators/import-db/)** feature, which re-processes past blocks - and, while doing so, retains the whole, non-pruned accounts history.
Expand Down

0 comments on commit 956d3e0

Please sign in to comment.