Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Store Forked Indexer when Publishing #724

Merged
merged 8 commits into from
May 8, 2024
Merged

Conversation

darunrs
Copy link
Collaborator

@darunrs darunrs commented May 2, 2024

Maintaining what Indexer was forked during development is useful for various usage statistics. I've updated the frontend to now store what indexer was forked from and store that information in the contract after publishing. We specifically store the indexer which was directly forked.

With this information, we can construct a graph using the "forked_from" as edge information, and calculate various other statistics such as how many indexers used some other indexer as a base by doing path counts. This can be done by creating an indexer which indexes the QueryApi contract.

There are some bugs in the workflow currently. Specifically, refreshing while in the forked page will cause the loss of whatever code was written. You need to re-fork the Indexer. While this benefits to ensuring the forked from field is correct, if we end up fixing this behavior, we need to then ensure forked from is stored persistently.

As part of updating the contract, I've removed various old functions which are no longer necessary.

@darunrs darunrs marked this pull request as ready for review May 6, 2024 20:28
@darunrs darunrs requested a review from a team as a code owner May 6, 2024 20:29
@@ -455,44 +361,16 @@ impl Contract {
}
}

pub fn list_indexer_functions(&self, account_id: Option<String>) -> OldAccountOrAllIndexers {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just double check these aren't used anywhere in the codebase (probably just frontend) to ensure this isn't a breaking change.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we do not use it anywhere. But, Gabe's account status component does use it. However, his component is only interested in the 2D map structure. He picks the Map corresponding to the account, and then gets the keys from the inner map. So, that's the reason I still kept this API. I just modified it to use our new structures, and the unit tests interestingly all passed with minimal changes.

@@ -281,7 +280,7 @@ impl Contract {
affected_account_id,
..
} => {
if affected_account_id == "*" {
if affected_account_id.contains("*") {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would prevent *.something.near which isn't what we want? I think we still want == "*"?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Yeah in Block Streamer, we actually have a hardcoded check where if the filter contains any of ["*", "*.near", "*.kaiching", "*.tg"] then we skip lake filtering. But the contract only checks Owner role for *. Even though *.near is basically just *.

I guess I can for now I can make them match. We'll just need to remember to evolve the contract if we make changes to the restrictions in the future.

registry/contract/src/lib.rs Outdated Show resolved Hide resolved
let account_id = env::signer_account_id();
let account_id = match account_id {
Some(account_id) => {
self.assert_roles(vec![Role::Owner]);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should only make this assertion if env::signer_account_id is different from what's passed 🤔

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we usually don't provide it at all. At least, the frontend omits it. But I can add a condition over the assert to only assert if it's different. Not too hard to do.

@@ -14,123 +14,6 @@ use serde::{Deserialize, Serialize};

type FunctionName = String;

#[derive(Clone, Debug, Serialize, Deserialize, BorshSerialize, BorshDeserialize, PartialEq, Eq)]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These types are shared across block-streamer and coordinator, can you try compile those just to ensure they still work. I'm pretty sure there will be no impact, just want to sanity check.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the call out. I did a cargo build on both and they succeeded building. I don't believe I need to do any runtime checks right?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, compile check is enough.

@darunrs darunrs merged commit 43c60f5 into main May 8, 2024
3 checks passed
@darunrs darunrs deleted the store-forked-indexer branch May 8, 2024 23:10
@darunrs darunrs linked an issue May 14, 2024 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Track forks of indexers
2 participants