Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added two env vars for enabling query performance #4917

Merged
merged 5 commits into from
Oct 14, 2024
Merged

Conversation

minhtuev
Copy link
Contributor

@minhtuev minhtuev commented Oct 10, 2024

What changes are proposed in this pull request?

Added two env vars that control the default behaviors for query performance mode

How is this patch tested? If it is not, please explain why.

  • Manual testing

Release Notes

Is this a user-facing change that should be mentioned in the release notes?

  • No. You can skip the rest of this section.
  • Yes. Give a description of this change to be included in the release
    notes for FiftyOne users.

(Details in 1-2 sentences. You can just refer to another PR with a description
if this PR is part of a larger change.)

What areas of FiftyOne does this PR affect?

  • App: FiftyOne application changes
  • Build: Build and test infrastructure changes
  • Core: Core fiftyone Python library changes
  • Documentation: FiftyOne documentation changes
  • Other

Summary by CodeRabbit

  • New Features

    • Updated terminology in the Options component from "Lightning mode" to "Query Performance mode."
    • Introduced new configuration options related to query performance.
  • Bug Fixes

    • Improved exception handling in the saved_views method for better reliability.
  • Documentation

    • Updated GraphQL schema and fragments to include new fields for query performance settings.
  • Chores

    • Marked the lightning_threshold attribute for deprecation in the configuration settings.

self.lightning_threshold = self.parse_int(
d,
"lightning_threshold",
env_var="FIFTYONE_APP_LIGHTNING_THRESHOLD",
default=None,
)
self.enable_query_performance = self.parse_bool(
d, "enable_query_performance",
env_var="FIFTYONE_APP_ENABLE_QUERY_PERFORMANCE", default=True
Copy link
Contributor

@kaixi-wang kaixi-wang Oct 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we defaulting to true for both enable and default?

Also the naming is super confusing. I have no idea what query performance true/false actually does without digging into implementation detail

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kaixi-wang : do you have a suggestion for name change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FIFTYONE_ENABLE_QUERY_PERFORMANCE=true/false: whether to disable the entire query performance feature, including the panel, toasts, etc; default: true

FIFTYONE_DEFAULT_QUERY_PERFORMANCE=true/false: whether to enable query performance by default for each dataset (if query performance is enabled systemwide per first env var); default: true

@minhtuev minhtuev marked this pull request as ready for review October 11, 2024 20:21
Copy link
Contributor

coderabbitai bot commented Oct 11, 2024

Walkthrough

The changes involve updates to various components and configurations related to query performance. The Options component's terminology has been revised, and it now conditionally renders subcomponents based on application state. The GraphQL schema has been expanded with new fields for query performance in the configFragment, and additional modifications were made to several types. The AppConfig class in Python has also been updated to include new attributes for query performance, while marking the old lightning_threshold attribute for deprecation. Overall, these changes enhance the application's configuration and rendering capabilities.

Changes

File Change Summary
app/packages/core/src/components/Actions/Options.tsx Updated terminology in Options component; conditional rendering based on state; modified method signature and type definition.
app/packages/relay/src/fragments/configFragment.ts Added enableQueryPerformance and defaultQueryPerformance fields to configFragment.
app/schema.graphql Updated GraphQL schema with new fields in Aggregate, AppConfig, and Dataset; reformatted union types and method signatures.
fiftyone/core/config.py Updated AppConfig class with new boolean attributes for query performance; deprecated lightning_threshold.
fiftyone/server/query.py Updated AppConfig and Query classes with new fields and improved exception handling in saved_views.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant OptionsComponent
    participant RecoilState
    participant SubComponents

    User->>OptionsComponent: Interacts with Options
    OptionsComponent->>RecoilState: Check modal state
    RecoilState-->>OptionsComponent: Return modal state
    OptionsComponent->>SubComponents: Render based on state
    SubComponents-->>OptionsComponent: Display appropriate components
Loading

🐰 "In the land of code where changes bloom,
New options sprout, dispelling gloom.
Query performance takes the stage,
As we turn a new, exciting page.
With every click, the data flows,
A rabbit's joy in all that grows!" 🐇


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (5)
app/packages/core/src/components/Actions/Options.tsx (1)

230-230: Approved: Terminology update from "Lightning mode" to "Query Performance mode"

This change aligns well with the PR objectives and provides a more descriptive term for the feature.

Consider updating any related documentation or user guides to reflect this terminology change for consistency across the application.

app/schema.graphql (2)

Line range hint 1-5: LGTM: New histogram aggregation capability added

The addition of the histogramValues field to the Aggregate input type enhances the aggregation capabilities, allowing for histogram-based data analysis. This is a valuable feature for data visualization and analysis.

Consider updating the documentation to explain the usage and benefits of this new histogram aggregation feature.


Line range hint 437-441: LGTM: New exclude field added to LightningPathInput

The addition of the exclude field to LightningPathInput enhances the flexibility of lightning queries by allowing specific values to be excluded from the results. The included description clearly explains the field's purpose.

Consider updating any related documentation or examples to demonstrate the usage of this new exclude field in lightning queries.

fiftyone/server/query.py (1)

378-379: LGTM! Consider updating documentation.

The addition of enable_query_performance and default_query_performance fields aligns well with the PR objectives to introduce environment variables for controlling query performance mode. This change enhances the configurability of the application.

Consider updating the relevant documentation to reflect these new configuration options and their impact on query performance. This will help users understand how to leverage these new features effectively.

fiftyone/core/config.py (1)

351-357: Enhance the deprecation notice for lightning_threshold.

While it's good to see a comment explaining the reason for deprecation, consider adding more information to help users migrate:

  1. Add a deprecation warning using the warnings module.
  2. Provide information about when this attribute will be removed (e.g., in which version).
  3. If possible, add a brief explanation or link to documentation about the new timer-based approach.

Example:

import warnings

# deprecate lightning threshold in favor of a timer-based approach
self.lightning_threshold = self.parse_int(
    d,
    "lightning_threshold",
    env_var="FIFTYONE_APP_LIGHTNING_THRESHOLD",
    default=None,
)
warnings.warn(
    "The 'lightning_threshold' attribute is deprecated and will be removed in version X.Y. "
    "Use the new timer-based approach instead. See [DOC_LINK] for more information.",
    DeprecationWarning,
    stacklevel=2
)
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between 6f5a12c and 4314f90.

⛔ Files ignored due to path filters (5)
  • app/packages/app/src/pages/__generated__/IndexPageQuery.graphql.ts is excluded by !**/__generated__/**, !**/__generated__/**
  • app/packages/app/src/pages/datasets/__generated__/DatasetPageQuery.graphql.ts is excluded by !**/__generated__/**, !**/__generated__/**
  • app/packages/relay/src/fragments/__generated__/configFragment.graphql.ts is excluded by !**/__generated__/**, !**/__generated__/**
  • app/packages/relay/src/fragments/__generated__/estimatedCountsFragment.graphql.ts is excluded by !**/__generated__/**, !**/__generated__/**
  • app/packages/relay/src/queries/__generated__/datasetQuery.graphql.ts is excluded by !**/__generated__/**, !**/__generated__/**
📒 Files selected for processing (5)
  • app/packages/core/src/components/Actions/Options.tsx (1 hunks)
  • app/packages/relay/src/fragments/configFragment.ts (1 hunks)
  • app/schema.graphql (8 hunks)
  • fiftyone/core/config.py (1 hunks)
  • fiftyone/server/query.py (1 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
app/packages/core/src/components/Actions/Options.tsx (1)

Pattern **/*.{ts,tsx}: Review the Typescript and React code for conformity with best practices in React, Recoil, Graphql, and Typescript. Highlight any deviations.

app/packages/relay/src/fragments/configFragment.ts (1)

Pattern **/*.{ts,tsx}: Review the Typescript and React code for conformity with best practices in React, Recoil, Graphql, and Typescript. Highlight any deviations.

🔇 Additional comments (11)
app/packages/relay/src/fragments/configFragment.ts (1)

14-15: LGTM: New fields align with PR objectives

The addition of enableQueryPerformance and defaultQueryPerformance fields to the config object in the configFragment is well-implemented and aligns perfectly with the PR objectives. These new fields will allow for better control over query performance modes, as intended.

app/packages/core/src/components/Actions/Options.tsx (2)

232-232: Approved: Consistent terminology update in title

This change maintains consistency with the previous update, ensuring a coherent user interface.


Line range hint 1-424: Summary: Terminology update enhances clarity

The changes in this file are focused on updating the terminology from "Lightning mode" to "Query Performance mode". This update aligns with the PR objectives and improves clarity for users without altering the component's functionality.

To ensure consistency across the codebase, please run the following script to check for any remaining instances of "Lightning mode" that may need updating:

app/schema.graphql (6)

7-7: LGTM: Improved formatting for union types

The formatting changes to AggregateResult and AggregationResponses union types improve readability and maintain consistency in the schema definition. These changes don't affect functionality and are a good practice for schema maintenance.

Also applies to: 31-31


473-485: LGTM: Improved formatting for mutation definitions

The formatting changes to the Mutation type definitions improve readability and maintain a consistent style across the schema. These changes don't affect functionality and contribute to better schema maintenance.


509-511: LGTM: Improved formatting for query definitions

The formatting changes to the Query type definitions enhance readability and maintain a consistent style across the schema. These changes are cosmetic and don't impact functionality, contributing to better schema maintenance.

Also applies to: 520-521, 526-526


447-447: LGTM: Improved formatting for LightningResults union type

The formatting change to the LightningResults union type improves readability and maintains a consistent style across the schema. This change is cosmetic and doesn't affect functionality.


229-229: Change in nullability for estimatedSampleCount field

The estimatedSampleCount field in the Dataset type has been changed from non-nullable (Int!) to nullable (Int). This change provides more flexibility when the count might not be available or calculated.

Ensure that client-side code is updated to handle potential null values for this field. Run the following script to identify potential usage of this field:

#!/bin/bash
# Description: Check for estimatedSampleCount usage in client-side code

# Search for estimatedSampleCount in JavaScript/TypeScript files
rg --type js 'estimatedSampleCount'

39-40: LGTM: New query performance configuration fields added

The addition of enableQueryPerformance and defaultQueryPerformance fields to the AppConfig type aligns with the PR objective of introducing environment variables for query performance control. These required fields will ensure consistent configuration across the application.

Please verify that existing queries and mutations involving the AppConfig type have been updated to include these new required fields. Run the following script to check for potential issues:

✅ Verification successful

LGTM: Verified Query Performance Configuration Fields

The addition of enableQueryPerformance and defaultQueryPerformance fields to the AppConfig type has been successfully verified. No existing queries or mutations were found that require updates to include these new required fields.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check for AppConfig usage in queries and mutations

# Search for AppConfig in GraphQL operations
rg --type graphql 'AppConfig.*\{' -A 10

Length of output: 35


Script:

#!/bin/bash
# Description: Search for all usages of AppConfig in GraphQL operations

# Search for any occurrence of AppConfig in GraphQL files
rg --type graphql 'AppConfig' -A 10

Length of output: 67


Script:

#!/bin/bash
# Description: Search for all usages of AppConfig in GraphQL operations

# Search for any occurrence of AppConfig in .graphql and .gql files
rg 'AppConfig' --glob '*.graphql' --glob '*.gql' -A 10

Length of output: 2242

fiftyone/server/query.py (1)

378-379: Verify handling of new fields in related code

While the addition of enable_query_performance and default_query_performance fields to AppConfig is straightforward, it's important to ensure that these new fields are properly handled in related parts of the code.

Please run the following script to check if the Query class and any serialization/deserialization logic for AppConfig have been updated to handle these new fields:

If the script doesn't return any results, consider updating the Query class and any serialization/deserialization logic to handle the new fields.

fiftyone/core/config.py (1)

358-365: ⚠️ Potential issue

Review query performance attributes and consider renaming.

The addition of enable_query_performance and default_query_performance attributes is noted. However, there are some concerns:

  1. Both attributes default to True. Consider if this is the desired default behavior, especially for enable_query_performance. Enabling new performance features by default might affect existing systems.

  2. The naming of default_query_performance is ambiguous. It's not clear what a boolean value represents for "default" performance. Consider a more descriptive name that clearly indicates its purpose, such as use_high_performance_query_mode or optimize_query_performance.

  3. Add comments explaining the purpose and impact of these attributes to help users understand how they affect the system's behavior.

To verify the impact and usage of these new attributes, let's search for their usage in the codebase:

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Outside diff range and nitpick comments (1)
fiftyone/core/config.py (1)

351-357: Consider adding more details about the deprecation of lightning_threshold.

While it's good to mark lightning_threshold as deprecated, it would be helpful to provide more information:

  1. Add a deprecation warning using the warnings module.
  2. Include information about when this attribute will be removed.
  3. Provide guidance on what users should use instead (the mentioned "timer-based approach").

This will help users prepare for the future removal of this attribute and understand how to migrate their code.

Here's a suggested implementation:

import warnings

# At the top of the file, add:
from fiftyone.core.config import TIMER_BASED_APPROACH_DOCS_URL  # Assuming this constant exists

# Then modify the existing code:
warnings.warn(
    "The 'lightning_threshold' attribute is deprecated and will be removed in version X.Y. "
    f"Use the new timer-based approach instead. See {TIMER_BASED_APPROACH_DOCS_URL} for details.",
    DeprecationWarning,
    stacklevel=2
)
self.lightning_threshold = self.parse_int(
    d,
    "lightning_threshold",
    env_var="FIFTYONE_APP_LIGHTNING_THRESHOLD",
    default=None,
)
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between 4314f90 and d0518fa.

📒 Files selected for processing (2)
  • app/schema.graphql (3 hunks)
  • fiftyone/core/config.py (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • app/schema.graphql
🧰 Additional context used

Comment on lines +358 to +369
self.enable_query_performance = self.parse_bool(
d,
"enable_query_performance",
env_var="FIFTYONE_APP_ENABLE_QUERY_PERFORMANCE",
default=True,
)
self.default_query_performance = self.parse_bool(
d,
"default_query_performance",
env_var="FIFTYONE_APP_DEFAULT_QUERY_PERFORMANCE",
default=True,
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Reconsider naming and default values for new query performance attributes.

The addition of enable_query_performance and default_query_performance is a good step towards improving query performance. However, there are a few points to consider:

  1. The naming of these attributes could be more descriptive. For example, enable_query_performance_optimization and use_query_performance_optimization_by_default would be clearer.

  2. Both attributes default to True, which might change behavior for existing users. Consider defaulting default_query_performance to False to maintain backwards compatibility.

  3. The purpose and interaction of these two attributes are not immediately clear from their names. Consider adding comments to explain their roles and how they affect query performance.

Here's a suggested implementation with improved naming and comments:

# Enable the query performance optimization feature
self.enable_query_performance_optimization = self.parse_bool(
    d,
    "enable_query_performance_optimization",
    env_var="FIFTYONE_APP_ENABLE_QUERY_PERFORMANCE_OPTIMIZATION",
    default=True,
)

# Use query performance optimization by default when enabled
self.use_query_performance_optimization_by_default = self.parse_bool(
    d,
    "use_query_performance_optimization_by_default",
    env_var="FIFTYONE_APP_USE_QUERY_PERFORMANCE_OPTIMIZATION_BY_DEFAULT",
    default=False,  # Default to False for backwards compatibility
)

Also, consider adding documentation for these new attributes to help users understand their purpose and usage.

Copy link
Contributor

@benjaminpkane benjaminpkane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@minhtuev minhtuev merged commit 16f55b0 into develop Oct 14, 2024
13 checks passed
@minhtuev minhtuev deleted the feat/qp-env-config branch October 14, 2024 18:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants