Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

V6/paranet sync command #3145

Conversation

aleksamagicka
Copy link
Contributor

No description provided.

src/commands/paranet/paranet-sync-command.js Outdated Show resolved Hide resolved
src/service/get-service.js Outdated Show resolved Hide resolved
@aleksamagicka aleksamagicka force-pushed the v6/paranet-sync-command-aleksa branch from 358e0a7 to 0946b5d Compare April 23, 2024 10:22
@aleksamagicka aleksamagicka marked this pull request as ready for review April 24, 2024 13:59
@aleksamagicka aleksamagicka force-pushed the v6/paranet-sync-command-aleksa branch from 9004f0c to 437c87f Compare April 25, 2024 11:54
src/service/blockchain-event-listener-service.js Outdated Show resolved Hide resolved
ot-node.js Outdated Show resolved Hide resolved
ot-node.js Outdated Show resolved Hide resolved
ot-node.js Outdated Show resolved Hide resolved
src/commands/paranet/paranet-sync-command.js Show resolved Hide resolved
src/commands/paranet/paranet-sync-command.js Outdated Show resolved Hide resolved
src/commands/paranet/paranet-sync-command.js Outdated Show resolved Hide resolved
src/commands/paranet/paranet-sync-command.js Outdated Show resolved Hide resolved
src/commands/paranet/paranet-sync-command.js Outdated Show resolved Hide resolved
src/commands/paranet/paranet-sync-command.js Outdated Show resolved Hide resolved
src/commands/paranet/start-paranet-sync-commands.js Outdated Show resolved Hide resolved
};

promises.append(
this.commandExecutor.add({
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that it would be better to change the logic here. This command will potentially schedule a lot of paranet-sync-commands (if we have a lot of assets in Paranet and node started syncing it for the first time.). Previously we had a problem when scheduling a lot of commands in command executor - operational db become too slow...
I recommend to change the logic so that start-paranet-sync-command is scheduled as permanent command and executed just once during node start. It's purpose is to schedule new paranet-sync-command for each Paranet user defined in configuration.
paranet-sync-command should have logic to get and store all assets connected to that paranet.
We did logic like this in epoch-check-command/blockchain-epoch-check-command and get-latest-service-agreement-data/blockchain-get-latest-service-agreement-data please check it out.

await this.repositoryModuleManager.updateParanetKaCount(paranetId, contractKaCount);
});

await Promise.all(promises);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What will happen here if one of the promises fails, we will update Paranet ka count in database but not schedule all commands, so we will miss some token ids

src/commands/paranet/paranet-sync-command.js Outdated Show resolved Hide resolved
async syncAsset(blockchain, contract, tokenId, assertionIds, stateIndex, paranetId, paranetRepository, latestAsset, deleteFromEarlier) {
try {
if (
await this.repositoryModuleManager.isStateSynced(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are missing implementation for couple of methods in repository module used in this command

src/service/get-service.js Outdated Show resolved Hide resolved
src/service/paranet-id-service.js Outdated Show resolved Hide resolved
);

// Go through all except the last one
for (let stateIndex = assertionIds.length - 2; stateIndex > 0; stateIndex -= 1) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should loop condition be >0, if we have assertionIds.length equal to 2, would we want to get asset stateIndex 0?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or state index starts from 1?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initial code was this, so I'm not sure

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initial state index of the asset is 0, if there is one update it's assertionIds.length will be 2. When we call this command it won't get into the lope, as stateIndex will be 0 at beginning of the loop, and it should process this old asset and put it historical, but with current condition we won't do it?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should test this :D

src/commands/paranet/paranet-sync-command.js Outdated Show resolved Hide resolved
@@ -96,6 +101,60 @@ class GetService extends OperationService {
tokenId,
keyword,
);

if (paranetSync) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally not sure if this is the right place for sync logic.

I think it shouldn't be the responsibility of getting service to do asset sync logic, I would suggest moving this logic to paranet-sync-command and just getting the result of the network get from there, and continue inserting and moving data.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC Djordje said it's OK, as it's already syncing assets above?

@aleksamagicka aleksamagicka force-pushed the v6/paranet-sync-command-aleksa branch from 18143ea to 3c90e3b Compare April 30, 2024 08:47
@Mihajlo-Pavlovic
Copy link
Collaborator

Where are we creating new paranets repositories in triple store?

@aleksamagicka
Copy link
Contributor Author

Where are we creating new paranets repositories in triple store?

In ot-node.js

@Mihajlo-Pavlovic Mihajlo-Pavlovic changed the base branch from v6/develop to v6/paranet-support May 14, 2024 13:48
@Mihajlo-Pavlovic Mihajlo-Pavlovic changed the base branch from v6/paranet-support to v6/paranets May 14, 2024 13:49
@Mihajlo-Pavlovic Mihajlo-Pavlovic merged commit fe88963 into OriginTrail:v6/paranets May 14, 2024
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants