Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: updated the logic for variable update queue #6586

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

SagarRajput-7
Copy link
Contributor

@SagarRajput-7 SagarRajput-7 commented Dec 4, 2024

Summary

Under this change I have introduced the following things -

  • A new approach using topological sort to replace the current variable queue approach, which contains many issues and redundancy
  • With the new approach, multiple customer issues around the dashboard not loading are fixed. (issues mentioned below)

For more info refer to the Technical Analysis and new approach block of this notion doc - https://www.notion.so/signoz/Dashboard-Variables-8783d02224c0483b9e8a4b8218cb5ddc?pvs=4#2c0617c157a141f5a09ba261d9d15d4d

Test cases -

image image

Related Issues / PR's

Screenshots

Comparison between local and prod env

  • Staging Env - (No throttling) -
Screen.Recording.2024-12-04.at.10.15.42.AM.mov

Affected Areas and Manually Tested Areas

Tested before and after of -


Important

Replaces variable update queue with a dependency graph in dashboard variable management, adding new utility functions and tests.

  • Behavior:
    • Replaces variable update queue with dependency graph in DashboardVariableSelection.tsx.
    • Updates onValueUpdate to handle mounted and non-mounted calls.
    • Removes onVarChanged function.
  • Functions:
    • Adds buildDependencies, buildDependencyGraph, buildParentDependencyGraph, and onUpdateVariableNode in util.ts.
    • Updates validVariableUpdate logic in VariableItem.tsx.
  • Tests:
    • Adds tests for new utility functions in dashboardVariables.test.tsx.
  • Misc:
    • Adds useRef for initialization checks in DashboardVariableSelection.tsx and VariableItem.tsx.
    • Minor refactoring in VariableItem.tsx.

This description was created by Ellipsis for 1ee2cc7. It will automatically update as commits are pushed.

@github-actions github-actions bot added docs required enhancement New feature or request labels Dec 4, 2024
Copy link

github-actions bot commented Dec 4, 2024

Build Error! No Linked Issue found. Please link an issue or mention it in the body using #<issue_id>

2 similar comments
Copy link

github-actions bot commented Dec 4, 2024

Build Error! No Linked Issue found. Please link an issue or mention it in the body using #<issue_id>

Copy link

github-actions bot commented Dec 4, 2024

Build Error! No Linked Issue found. Please link an issue or mention it in the body using #<issue_id>

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Changes requested. Reviewed everything up to fa4aeae in 1 minute and 38 seconds

More details
  • Looked at 307 lines of code in 3 files
  • Skipped 0 files when reviewing.
  • Skipped posting 4 drafted comments based on config settings.
1. frontend/src/container/NewDashboard/DashboardVariablesSelection/util.ts:33
  • Draft comment:
    The getDependentVariables function is duplicated in multiple files. Consider defining it in a single location and importing it where needed to avoid redundancy and potential inconsistencies.
  • Reason this comment was not posted:
    Decided after close inspection that this draft comment was likely wrong and/or not actionable:
    The comment is speculative because it assumes the function is duplicated in other files, but the diff does not provide evidence of this. Without evidence, the comment does not meet the criteria for a necessary code change.
    I might be missing context from other files that are not shown in the diff. The function could indeed be duplicated elsewhere, but I have no way to verify this from the current information.
    Given the rules, I should only consider the information presented in the diff. Without evidence of duplication in the diff, the comment should be removed.
    Remove the comment because it is speculative and not based on evidence from the diff.
2. frontend/src/container/NewDashboard/DashboardVariablesSelection/VariableItem.tsx:99
  • Draft comment:
    The validVariableUpdate function's logic for checking if the component is mounted is inverted. It should return true if the component is mounted and the variable is in the update queue, not the other way around. Consider revising the logic.
  • Reason this comment was not posted:
    Comment looked like it was already resolved.
3. frontend/src/container/NewDashboard/DashboardVariablesSelection/VariableItem.tsx:95
  • Draft comment:
    The useEffect that sets isMounted.current to true is missing a dependency array. Add an empty dependency array to ensure this effect only runs once when the component mounts.
  • Reason this comment was not posted:
    Comment looked like it was already resolved.
4. frontend/src/container/NewDashboard/DashboardVariablesSelection/util.ts:33
  • Draft comment:
    This function is duplicated. Consider importing and reusing the existing implementation instead.

  • function getDependentVariables (VariableItem.tsx)

  • Reason this comment was not posted:
    Marked as duplicate.

Workflow ID: wflow_so46fFL9ZJXaSHHK


Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

Copy link

github-actions bot commented Dec 4, 2024

Build Error! No Linked Issue found. Please link an issue or mention it in the body using #<issue_id>

@cxposition
Copy link

I merged the three files based on version 0.57.0 of the code, and the query interface worked, but the query_range interface reported an error.
image
image

@SagarRajput-7
Copy link
Contributor Author

@cxposition - Hey, it seems like there is some issue with composite queries, and it seems like you are missing configs there, so can you please share the URL of this above scenario? It will help with the debugging.

@cxposition
Copy link

@cxposition - Hey, it seems like there is some issue with composite queries, and it seems like you are missing configs there, so can you please share the URL of this above scenario? It will help with the debugging.

I'm sorry that the company's Intranet environment cannot be provided, maybe it is our configuration problem, we will investigate and solve it first, and I will share with you as soon as there is any progress, thank you for your reply.

@cxposition
Copy link

@cxposition - Hey, it seems like there is some issue with composite queries, and it seems like you are missing configs there, so can you please share the URL of this above scenario? It will help with the debugging.

The problem is resolved. It is a configuration problem.Thanks for your reply.

Copy link

Build Error! No Linked Issue found. Please link an issue or mention it in the body using #<issue_id>

@@ -21,6 +26,11 @@ function DashboardVariableSelection(): JSX.Element | null {

const [variablesTableData, setVariablesTableData] = useState<any>([]);

const [dependencyData, setDependencyData] = useState<{
order: string[];
graph: VariableGraph;
Copy link
Collaborator

@vikrantgupta25 vikrantgupta25 Dec 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extract out to independent type , IDependencyData

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

const initializationRef = useRef(false);

useEffect(() => {
if (variablesTableData.length > 0 && !initializationRef.current) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this required ?

  • since the initial state of dependency data is null, a null check would be sufficient to know the first load ?
  • on similar lines we do not want to re-calculate the deps graph / order when variablesTableData is updated ? when adding or removing variables from the dashboard ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done, removed ref and added a check on the dependencyGraph as discussed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also tested variable addition and deletion

@@ -88,18 +89,26 @@ function VariableItem({
(state) => state.globalTime,
);

// logic to detect if its a rerender or a new render/mount
const isMounted = useRef(false);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is onMount handling different ? onMount should be handled via the root triggers ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed and added root trigger handling

@@ -29,3 +29,110 @@ export const convertVariablesToDbFormat = (
result[id] = obj;
return result;
}, {});

const getDependentVariables = (queryValue: string): string[] => {
const variableRegexPattern = /\{\{\s*?\.([^\s}]+)\s*?\}\}/g;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we do support a bunch of other variable formats as well, please do include them as well

}

if (topologicalOrder.length !== Object.keys(dependencies).length) {
throw new Error('Cycle detected in the dependency graph!');
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we want to break the UI here ? maybe show some user consumable error here ? or detect this while saving the variables so that this doesn't occur ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added console.error - to help debugging and created a ticket to work on a better informative UI for this -
#6715

allSelected,
isMounted.current,
);
setVariablesToGetUpdated((prev) =>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

onValueUpdate should be responsible to remove the variable from variablesToGetUpdated ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed

@pranay01 pranay01 deleted the branch main December 19, 2024 13:03
@pranay01 pranay01 closed this Dec 19, 2024
@SagarRajput-7 SagarRajput-7 reopened this Dec 20, 2024
@SagarRajput-7 SagarRajput-7 changed the base branch from develop to main December 20, 2024 02:54
Copy link

Build Error! No Linked Issue found. Please link an issue or mention it in the body using #<issue_id>

SagarRajput-7 and others added 3 commits December 24, 2024 11:44
… variables (#6609)

* feat: added API limiting to reduce unneccesary api call for dashboard variables

* feat: fixed dropdown open triggering the api calls for single-select and misc
…sors - dashboardVariables (#6621)

* feat: added API limiting to reduce unneccesary api call for dashboard variables

* feat: fixed dropdown open triggering the api calls for single-select and misc

* feat: add jest test cases for new logic's utils, functions and processors - dashboardVariables

* feat: added test for checkAPIInvocation

* feat: refactor code

* feat: added more test on graph utilities
Copy link

Build Error! No Linked Issue found. Please link an issue or mention it in the body using #<issue_id>

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Changes requested. Incremental review on 5097375 in 32 seconds

More details
  • Looked at 518 lines of code in 3 files
  • Skipped 0 files when reviewing.
  • Skipped posting 1 drafted comments based on config settings.
1. frontend/src/container/NewDashboard/DashboardVariablesSelection/util.ts:49
  • Draft comment:
    Remove the console.log statement to clean up the code.

  • Reason this comment was not posted:
    Marked as duplicate.

Workflow ID: wflow_opWqhmMAiMrrkLH0


Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

export const buildDependencies = (
variables: IDashboardVariable[],
): VariableGraph => {
console.log('buildDependencies', variables);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the console.log statement to avoid unnecessary console output.

Suggested change
console.log('buildDependencies', variables);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@srikanthccv srikanthccv self-assigned this Dec 24, 2024
@srikanthccv
Copy link
Member

@srikanthccv
Copy link
Member

@SagarRajput-7 is this point about not resetting already selected values https://www.notion.so/signoz/Dashboard-Variables-8783d02224c0483b9e8a4b8218cb5ddc?pvs=4#144fcc6bcd1980dda2c2ede018f29914. Is it part of this PR? or is it yet to be implemented?

@SagarRajput-7
Copy link
Contributor Author

https://www.loom.com/share/a958edbb6f2f42b1881d60658fe8cbef

Thanks for pointing this, fixed in the latest commit -

Screen.Recording.2024-12-26.at.10.39.21.AM.mov

Copy link

Build Error! No Linked Issue found. Please link an issue or mention it in the body using #<issue_id>

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on f2bb467 in 19 seconds

More details
  • Looked at 284 lines of code in 3 files
  • Skipped 0 files when reviewing.
  • Skipped posting 3 drafted comments based on config settings.
1. frontend/src/container/NewDashboard/DashboardVariablesSelection/util.ts:115
  • Draft comment:
    Consider using a more robust error handling mechanism instead of console.error for cycle detection in the dependency graph.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The error message for cycle detection in the dependency graph is logged using console.error, which is appropriate for debugging but might not be suitable for production environments. Consider using a more robust error handling mechanism.
2. frontend/src/container/NewDashboard/DashboardVariablesSelection/util.ts:171
  • Draft comment:
    Revisit the logic in checkAPIInvocation where it returns false if parentDependencyGraph is empty. Ensure this aligns with the intended behavior for variables without dependencies.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The checkAPIInvocation function has a condition that returns false if parentDependencyGraph is empty. This might lead to unexpected behavior if the graph is supposed to be empty for certain variables. Consider revisiting this logic.
3. frontend/src/container/NewDashboard/DashboardVariablesSelection/util.ts:175
  • Draft comment:
    Revisit the logic in checkAPIInvocation where it returns true if there are no dependencies. Ensure this aligns with the intended behavior for API invocation checks.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The checkAPIInvocation function has a condition that returns true if there are no dependencies. This might not be the intended behavior if the function is supposed to check for dependencies before invocation. Consider revisiting this logic.

Workflow ID: wflow_1dZFStJAISA2kCLX


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

@SagarRajput-7
Copy link
Contributor Author

@SagarRajput-7 is this point about not resetting already selected values https://www.notion.so/signoz/Dashboard-Variables-8783d02224c0483b9e8a4b8218cb5ddc?pvs=4#144fcc6bcd1980dda2c2ede018f29914. Is it part of this PR? or is it yet to be implemented?

No, this is part of the dependent task section, as it will be affected by the new proposal and, hence, not implemented yet.

Copy link

Build Error! No Linked Issue found. Please link an issue or mention it in the body using #<issue_id>

Copy link

Build Error! No Linked Issue found. Please link an issue or mention it in the body using #<issue_id>

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on 1ee2cc7 in 35 seconds

More details
  • Looked at 126 lines of code in 3 files
  • Skipped 0 files when reviewing.
  • Skipped posting 8 drafted comments based on config settings.
1. frontend/src/container/NewDashboard/DashboardVariablesSelection/util.ts:40
  • Draft comment:
    The regex pattern in getDependentVariables is complex and could be simplified for better readability. Consider breaking it down or adding comments to explain each part.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The function getDependentVariables uses a regex pattern to match variable names in different formats. However, the regex pattern is complex and could be simplified for better readability and maintainability. Additionally, the function uses a map to extract variable names, which could be optimized.
2. frontend/src/container/NewDashboard/DashboardVariablesSelection/util.ts:34
  • Draft comment:
    The regex pattern in getDependentVariables is complex and could be simplified for better readability. Consider breaking it down or adding comments to explain each part.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The getDependentVariables function uses a regex pattern to match variable names in different formats. However, the regex pattern is complex and could be simplified for better readability and maintainability. Additionally, the function uses a map to extract variable names, which could be optimized.
3. frontend/src/container/NewDashboard/DashboardVariablesSelection/__test__/dashboardVariables.test.tsx:106
  • Draft comment:
    The test case expects checkAPIInvocation to return true when parentDependencyGraph is empty, but the implementation returns false. Verify the logic in checkAPIInvocation.
  • Reason this comment was not posted:
    Comment did not seem useful.
4. frontend/src/container/NewDashboard/DashboardVariablesSelection/VariableItem.test.tsx:49
  • Draft comment:
    Avoid using inline styles in React components. Use external stylesheets, CSS classes, or styled components instead. This applies to all similar instances in this file.
  • Reason this comment was not posted:
    Comment was on unchanged code.
5. frontend/src/container/NewDashboard/DashboardVariablesSelection/VariableItem.test.tsx:70
  • Draft comment:
    Avoid using inline styles in React components. Use external stylesheets, CSS classes, or styled components instead. This applies to all similar instances in this file.
  • Reason this comment was not posted:
    Marked as duplicate.
6. frontend/src/container/NewDashboard/DashboardVariablesSelection/VariableItem.test.tsx:90
  • Draft comment:
    Avoid using inline styles in React components. Use external stylesheets, CSS classes, or styled components instead. This applies to all similar instances in this file.
  • Reason this comment was not posted:
    Marked as duplicate.
7. frontend/src/container/NewDashboard/DashboardVariablesSelection/VariableItem.test.tsx:124
  • Draft comment:
    Avoid using inline styles in React components. Use external stylesheets, CSS classes, or styled components instead. This applies to all similar instances in this file.
  • Reason this comment was not posted:
    Marked as duplicate.
8. frontend/src/container/NewDashboard/DashboardVariablesSelection/VariableItem.test.tsx:153
  • Draft comment:
    Avoid using inline styles in React components. Use external stylesheets, CSS classes, or styled components instead. This applies to all similar instances in this file.
  • Reason this comment was not posted:
    Marked as duplicate.

Workflow ID: wflow_DZXcrgw61mV7PQPe


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs not required enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants