Skip to content

Commit

Permalink
change limit crypto list fetched to 5000 (#331)
Browse files Browse the repository at this point in the history
## PR Description:

Hello team,

This PR addresses two key issues to improve our application's
functionality and efficiency:

## Issue 1 - Increased Crypto List Fetch Limit:
Currently, our application fetches cryptocurrency data from Coin Market
Cap with a limit of 1200 assets. However, as our user base grows and the
cryptocurrency market expands, we need to provide access to a wider
range of assets. To address this, we have increased the limit for
fetching crypto lists from 1200 to 5000. This expansion ensures that our
users can access a more comprehensive list of cryptocurrencies.

## Issue 2 - Optimized Network Retrieval with Batch Concept:
In the process of fetching data about networks associated with tokens,
we have observed that there is a maximum limit of 1300 networks that can
be retrieved at once. To overcome this limitation, we have implemented a
batch concept in the getNetworkByToken function. This modification
allows us to efficiently fetch network data while staying within the
constraints imposed by external services.

## Changes Made:

- Increased the crypto list fetch limit from Coin Market Cap to 5000
assets, providing users with access to a broader range of
cryptocurrencies.
- Implemented a batch concept in the getNetworkByToken function to
optimize network data retrieval, ensuring efficient handling of a large
number of networks.


## Notes for Reviewers:

Please review the changes introduced in this PR to ensure that the
increased crypto list fetch limit and the batch concept in the
getNetworkByToken function are correctly implemented. Verify that these
changes enhance the user experience and improve the efficiency of our
application.

Your attention and contributions are highly appreciated. Feel free to
provide any feedback or suggestions you may have.

Best regards,
Louay HICHRI
  • Loading branch information
hichri-louay authored Sep 13, 2023
2 parents 090db52 + a215a7a commit 23f34fc
Showing 1 changed file with 47 additions and 21 deletions.
68 changes: 47 additions & 21 deletions web3/wallets.js
Original file line number Diff line number Diff line change
Expand Up @@ -514,30 +514,56 @@ exports.getAllWallets = async (req, res) => {
return res.status(401).end('Account not found')
}


const getNetworkByToken = async (idCrypto) => {
try {
if (
cache.get('networks') &&
Date.now() - new Date(cache.get('networks').date).getTime() < 604800000
) {
cache.get('networks') ) {
return cache.get('networks').data;
} else {
const options = {
method: 'GET',
url: process.env.CMC_CRYPTO_DETAILS,
params: {
id: idCrypto,
},
headers: {
'X-CMC_PRO_API_KEY': process.env.CMCAPIKEY,
},
};
const idCryptoArray = idCrypto.split(',');
const batchSize = 1300; // Number of cryptos per batch

// Split the idCryptoArray into batches
const batches = [];
for (let i = 0; i < idCryptoArray.length; i += batchSize) {
const batch = idCryptoArray.slice(i, i + batchSize).join(',');
batches.push(batch);
}

const results = [];

// Send requests for each batch concurrently using Promise.all
await Promise.all(
batches.map(async (batch) => {
const options = {
method: 'GET',
url: process.env.CMC_CRYPTO_DETAILS,
params: {
id: batch,
},
headers: {
'X-CMC_PRO_API_KEY': process.env.CMCAPIKEY,
},
};

try {
const result = await rp.request(options);
results.push(result);
} catch (err) {
console.error('Error fetching data for batch:', err);
// Handle the error as needed, e.g., retry or skip
}
})
);

const result = await rp.request(options);
const networksContract = Object.values(result.data.data).map((innerObj) => ({
symbol: innerObj.symbol,
contract_address: innerObj.contract_address,
}));
// Process the results
const networksContract = results.flatMap((result) =>
Object.values(result.data.data).map((innerObj) => ({
symbol: innerObj.symbol,
contract_address: innerObj.contract_address,
}))
);

const networks = { data: networksContract, date: Date.now() };
cache.put('networks', networks);
Expand All @@ -548,6 +574,7 @@ const getNetworkByToken = async (idCrypto) => {
throw new Error('Error fetching networks');
}
};

exports.getPrices = async () => {
try {
if (
Expand All @@ -562,7 +589,7 @@ exports.getPrices = async () => {
url: process.env.CMC_URl,
params: {
start: '1',
limit: '1200',
limit: '5000',
convert: 'USD',
},
headers: {
Expand Down Expand Up @@ -678,7 +705,6 @@ exports.getPrices = async () => {
var finalMap = {}

const idcrypto = priceMap.map((token) => token.id.toString());

priceMap.forEach((token) => {
finalMap[token.symbol] = { ...token, networkSupported: '' };
delete finalMap[token.symbol].symbol;
Expand Down Expand Up @@ -1017,7 +1043,7 @@ exports.getNativeBalance = async (web3Instance, walletAddress, network) => {
exports.getTronBalance = async (webTron, token, address, isTrx = false) => {
try {
if (isTrx) {

let amount = await webTron.trx.getBalance(address)
return amount.toString()
}
Expand Down

0 comments on commit 23f34fc

Please sign in to comment.