Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(opensearch): add profile_id and organization_id to /search APIs #5705

Merged
merged 18 commits into from
Sep 11, 2024

Conversation

tsdk02
Copy link
Contributor

@tsdk02 tsdk02 commented Aug 27, 2024

Type of Change

  • Bugfix
  • New feature
  • Enhancement
  • Refactoring
  • Dependency updates
  • Documentation
  • CI/CD

Description

Global Search is currently being done through the merchant_id authentication.
Enhancing the global search to search through new hierarchical structure of OrgLevel, MerchantLevel and ProfileLevel.
Structure of the enum used:

pub enum AuthInfo {
    OrgLevel {
        org_id: id_type::OrganizationId,
    },
    MerchantLevel {
        org_id: id_type::OrganizationId,
        merchant_ids: Vec<id_type::MerchantId>,
    },
    ProfileLevel {
        org_id: id_type::OrganizationId,
        merchant_id: id_type::MerchantId,
        profile_ids: Vec<id_type::ProfileId>,
    },
}
  • Modified the query-building for opensearch/elasticsearch by adding the orgLevel and ProfileLevel queries.
  • Results will now be returned if any search query is matched, and for all matched queries

Additional Changes

  • This PR modifies the API contract
  • This PR modifies the database schema
  • This PR modifies application configuration/environment variables

Motivation and Context

OrgLevel, MerchantLevel and ProfileLevel provide for enhanced analytics insights while using global search to the users

How did you test it?

  • Sample Query Structure
    image

  • Hit the /search API (using query, filters and timeRange).

curl --location 'http://localhost:8080/analytics/v1/search' \
--header 'Accept: */*' \
--header 'Accept-Language: en-US,en;q=0.9' \
--header 'Connection: keep-alive' \
--header 'Content-Type: application/json' \
--header 'Origin: http://localhost:9000' \
--header 'Referer: http://localhost:9000/' \
--header 'Sec-Fetch-Dest: empty' \
--header 'Sec-Fetch-Mode: cors' \
--header 'Sec-Fetch-Site: same-site' \
--header 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36' \
--header 'api-key: hyperswitch' \
--header 'authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkIjoiY2M3M2ZiY2EtYTExMS00Nzk5LWI3MDEtN2E4NmU0OWIxMzY1IiwibWVyY2hhbnRfaWQiOiJtZXJjaGFudF8xNzI1MjY3Mzk3Iiwicm9sZV9pZCI6Im9yZ19hZG1pbiIsImV4cCI6MTcyNTUxNzc2Nywib3JnX2lkIjoib3JnX0lpbTRxOHpPdlA3aGp3QTRxTkpoIiwicHJvZmlsZV9pZCI6bnVsbH0.7vZllJaSLEw1EsU4tgXkEcpAjRmjlukk6swzZfa6uU0' \
--header 'sec-ch-ua: "Not/A)Brand";v="8", "Chromium";v="126", "Google Chrome";v="126"' \
--header 'sec-ch-ua-mobile: ?0' \
--header 'sec-ch-ua-platform: "macOS"' \
--data '{
    "query": "usd",
    "filters": {
        "status": [
            "succeeded", "charged", "success"
        ]
    },
    "timeRange": {
        "startTime": "2024-09-02T00:30:00Z",
        "endTime": "2024-09-04T21:45:00Z"
    }
}'
  • Sample Response:
[
    {
        "count": 2,
        "index": "payment_attempts",
        "hits": [
            {
                "payment_id": "pay_mUtaBhUnlC2b7kTtxTJ0",
                "merchant_id": "merchant_1725267397",
                "attempt_id": "pay_mUtaBhUnlC2b7kTtxTJ0_1",
                "status": "charged",
                "amount": 6540,
                "currency": "USD",
                "save_to_locker": null,
                "connector": "stripe_test",
                "error_message": null,
                "offer_amount": null,
                "surcharge_amount": null,
                "tax_amount": null,
                "payment_method_id": null,
                "payment_method": "card",
                "connector_transaction_id": "pay_XO5KGoaMl5y7R8fM3Xya",
                "capture_method": "automatic",
                "capture_on": 1662804672,
                "confirm": true,
                "authentication_type": "no_three_ds",
                "created_at": 1725339016,
                "modified_at": 1725339017,
                "last_synced": 1725339016,
                "cancellation_reason": null,
                "amount_to_capture": 6540,
                "mandate_id": null,
                "browser_info": null,
                "error_code": null,
                "connector_metadata": null,
                "payment_experience": null,
                "payment_method_type": "credit",
                "payment_method_data": "{\"card\":{\"last4\":\"4242\",\"bank_code\":null,\"card_isin\":\"424242\",\"card_type\":null,\"card_issuer\":null,\"card_network\":null,\"card_exp_year\":\"25\",\"card_exp_month\":\"10\",\"payment_checks\":null,\"card_holder_name\":null,\"card_extended_bin\":null,\"authentication_data\":null,\"card_issuing_country\":null}}",
                "error_reason": null,
                "multiple_capture_count": null,
                "amount_capturable": 0,
                "merchant_connector_id": "mca_zPBTVwk8mQPMvW9rrW8n",
                "net_amount": 6540,
                "unified_code": null,
                "unified_message": null,
                "mandate_data": null,
                "client_source": null,
                "client_version": null,
                "profile_id": "pro_GOyb8ecA2QA4pXnJTvAL",
                "organization_id": "org_Iim4q8zOvP7hjwA4qNJh",
                "card_network": null,
                "sign_flag": 1,
                "tenant_id": "public",
                "clickhouse_database": "default",
                "@timestamp": "2024-09-03T04:50:16.000000000+00:00"
            },
            {
                "payment_id": "pay_CW0tpqeKk7F6q2TVKffH",
                "merchant_id": "merchant_1725267397",
                "attempt_id": "pay_CW0tpqeKk7F6q2TVKffH_1",
                "status": "charged",
                "amount": 6540,
                "currency": "USD",
                "save_to_locker": null,
                "connector": "stripe_test",
                "error_message": null,
                "offer_amount": null,
                "surcharge_amount": null,
                "tax_amount": null,
                "payment_method_id": null,
                "payment_method": "card",
                "connector_transaction_id": "pay_w5UHXOqOiLqRieOUQL63",
                "capture_method": "automatic",
                "capture_on": 1662804672,
                "confirm": true,
                "authentication_type": "no_three_ds",
                "created_at": 1725283209,
                "modified_at": 1725283210,
                "last_synced": 1725283209,
                "cancellation_reason": null,
                "amount_to_capture": 6540,
                "mandate_id": null,
                "browser_info": null,
                "error_code": null,
                "connector_metadata": null,
                "payment_experience": null,
                "payment_method_type": "credit",
                "payment_method_data": "{\"card\":{\"last4\":\"4242\",\"bank_code\":null,\"card_isin\":\"424242\",\"card_type\":null,\"card_issuer\":null,\"card_network\":null,\"card_exp_year\":\"25\",\"card_exp_month\":\"10\",\"payment_checks\":null,\"card_holder_name\":null,\"card_extended_bin\":null,\"authentication_data\":null,\"card_issuing_country\":null}}",
                "error_reason": null,
                "multiple_capture_count": null,
                "amount_capturable": 0,
                "merchant_connector_id": "mca_zPBTVwk8mQPMvW9rrW8n",
                "net_amount": 6540,
                "unified_code": null,
                "unified_message": null,
                "mandate_data": null,
                "client_source": null,
                "client_version": null,
                "profile_id": "pro_GOyb8ecA2QA4pXnJTvAL",
                "organization_id": "org_Iim4q8zOvP7hjwA4qNJh",
                "card_network": null,
                "sign_flag": 1,
                "tenant_id": "public",
                "clickhouse_database": "default",
                "@timestamp": "2024-09-02T13:20:09.000000000+00:00"
            }
        ],
        "status": "Success"
    },
    {
        "count": 2,
        "index": "payment_intents",
        "hits": [
            {
                "payment_id": "pay_mUtaBhUnlC2b7kTtxTJ0",
                "merchant_id": "merchant_1725267397",
                "status": "succeeded",
                "amount": 6540,
                "currency": "USD",
                "amount_captured": 6540,
                "customer_id": "StripeCustomer",
                "description": "Its my first payment request",
                "return_url": "https://google.com/",
                "metadata": "{\"data2\":\"camel\",\"login_date\":\"2019-09-10T10:11:12Z\",\"new_customer\":\"true\"}",
                "connector_id": null,
                "statement_descriptor_name": "joseph",
                "statement_descriptor_suffix": "JS",
                "created_at": 1725339016,
                "modified_at": 1725339017,
                "last_synced": 1725339016,
                "setup_future_usage": null,
                "off_session": null,
                "client_secret": "pay_mUtaBhUnlC2b7kTtxTJ0_secret_wiZ5NQnCZ2i9ajdAYyAJ",
                "active_attempt_id": "pay_mUtaBhUnlC2b7kTtxTJ0_1",
                "business_country": null,
                "business_label": "default",
                "attempt_count": 1,
                "profile_id": "pro_GOyb8ecA2QA4pXnJTvAL",
                "payment_confirm_source": null,
                "billing_details": null,
                "shipping_details": null,
                "customer_email": "f6cd0753a831880835a0e0d23136ef66318d8303605bd37fa80b4ecf248bd303",
                "feature_metadata": {
                    "redirect_response": null,
                    "search_tags": [
                        "9c202f2fabd12f758a12b449944f5cdaeadf39132680667543033158023bc805",
                        "ffd1c37eb18f3459e3da6f21f739866bce89a334061c248656cb71d83d2375f4"
                    ]
                },
                "merchant_order_reference_id": null,
                "organization_id": "org_Iim4q8zOvP7hjwA4qNJh",
                "sign_flag": 1,
                "tenant_id": "public",
                "clickhouse_database": "default",
                "@timestamp": "2024-09-03T04:50:16.000000000+00:00"
            },
            {
                "payment_id": "pay_CW0tpqeKk7F6q2TVKffH",
                "merchant_id": "merchant_1725267397",
                "status": "succeeded",
                "amount": 6540,
                "currency": "USD",
                "amount_captured": 6540,
                "customer_id": "StripeCustomer",
                "description": "Its my first payment request",
                "return_url": "https://google.com/",
                "metadata": "{\"data2\":\"camel\",\"login_date\":\"2019-09-10T10:11:12Z\",\"new_customer\":\"true\"}",
                "connector_id": null,
                "statement_descriptor_name": "joseph",
                "statement_descriptor_suffix": "JS",
                "created_at": 1725283209,
                "modified_at": 1725283210,
                "last_synced": 1725283209,
                "setup_future_usage": null,
                "off_session": null,
                "client_secret": "pay_CW0tpqeKk7F6q2TVKffH_secret_rZIT5OsQzbhkE7rhsUrq",
                "active_attempt_id": "pay_CW0tpqeKk7F6q2TVKffH_1",
                "business_country": null,
                "business_label": "default",
                "attempt_count": 1,
                "profile_id": "pro_GOyb8ecA2QA4pXnJTvAL",
                "payment_confirm_source": null,
                "billing_details": null,
                "shipping_details": null,
                "customer_email": "f6cd0753a831880835a0e0d23136ef66318d8303605bd37fa80b4ecf248bd303",
                "feature_metadata": {
                    "redirect_response": null,
                    "search_tags": [
                        "9c202f2fabd12f758a12b449944f5cdaeadf39132680667543033158023bc805",
                        "ffd1c37eb18f3459e3da6f21f739866bce89a334061c248656cb71d83d2375f4"
                    ]
                },
                "merchant_order_reference_id": null,
                "organization_id": "org_Iim4q8zOvP7hjwA4qNJh",
                "sign_flag": 1,
                "tenant_id": "public",
                "clickhouse_database": "default",
                "@timestamp": "2024-09-02T13:20:09.000000000+00:00"
            }
        ],
        "status": "Success"
    },
    {
        "count": 1,
        "index": "refunds",
        "hits": [
            {
                "internal_reference_id": "refid_yWIsLDexIn3LvMoEvXds",
                "refund_id": "ref_WBSqj48rx3gaOuz1kUJv",
                "payment_id": "pay_mUtaBhUnlC2b7kTtxTJ0",
                "merchant_id": "merchant_1725267397",
                "connector_transaction_id": "pay_XO5KGoaMl5y7R8fM3Xya",
                "connector": "stripe_test",
                "connector_refund_id": "dummy_ref_DA5VW0KwaiCvnTaeBtcA",
                "external_reference_id": "ref_WBSqj48rx3gaOuz1kUJv",
                "refund_type": "instant_refund",
                "total_amount": 6540,
                "currency": "USD",
                "refund_amount": 600,
                "refund_status": "success",
                "sent_to_gateway": true,
                "refund_error_message": null,
                "refund_arn": "",
                "created_at": 1725339184,
                "modified_at": 1725339185,
                "description": "Customer returned product",
                "attempt_id": "pay_mUtaBhUnlC2b7kTtxTJ0_1",
                "refund_reason": "Customer returned product",
                "refund_error_code": null,
                "profile_id": "pro_GOyb8ecA2QA4pXnJTvAL",
                "organization_id": "org_Iim4q8zOvP7hjwA4qNJh",
                "sign_flag": 1,
                "tenant_id": "public",
                "clickhouse_database": "default",
                "@timestamp": "2024-09-03T04:53:04.000000000+00:00"
            }
        ],
        "status": "Success"
    },
    {
        "count": 0,
        "index": "disputes",
        "hits": [],
        "status": "Failure"
    }
]

Checklist

  • I formatted the code cargo +nightly fmt --all
  • I addressed lints thrown by cargo clippy
  • I reviewed the submitted code
  • I added unit tests for my changes where possible

@tsdk02 tsdk02 added C-feature Category: Feature request or enhancement A-Analytics labels Aug 27, 2024
@tsdk02 tsdk02 self-assigned this Aug 27, 2024
@tsdk02 tsdk02 requested review from a team as code owners August 27, 2024 05:25
Copy link

semanticdiff-com bot commented Aug 27, 2024

Review changes with SemanticDiff.

Analyzed 3 of 3 files.

Overall, the semantic diff is 17% smaller than the GitHub diff.

Filename Status
✔️ crates/router/src/analytics.rs 10.69% smaller
✔️ crates/analytics/src/opensearch.rs 6.1% smaller
✔️ crates/analytics/src/search.rs 43.63% smaller

@tsdk02 tsdk02 linked an issue Aug 27, 2024 that may be closed by this pull request
@tsdk02 tsdk02 requested review from lsampras, a team and ThisIsMani September 4, 2024 04:04
ThisIsMani
ThisIsMani previously approved these changes Sep 4, 2024
@Gnanasundari24 Gnanasundari24 added this pull request to the merge queue Sep 11, 2024
Merged via the queue into main with commit 7eb5354 Sep 11, 2024
18 checks passed
@Gnanasundari24 Gnanasundari24 deleted the accept-profile-id-org-id-search-api branch September 11, 2024 16:03
pixincreate added a commit that referenced this pull request Sep 13, 2024
* 'main' of github.com:juspay/hyperswitch: (51 commits)
  feat(connector): [DEUTSCHEBANK] Integrate SEPA Payments (#5826)
  feat(payments_v2): payment intent diesel and domain models changes v2 (#5783)
  feat(connector): [Fiuu] ADD Wasm Configs (#5874)
  chore(version): 2024.09.13.0
  refactor(core): Update shipping_cost and order_tax_amount to net_amount of payment_attempt (#5844)
  refactor: return optional request body from build_request_v2 in ConnectorIntegrationV2 trait (#5865)
  feat(refunds): Refunds aggregate api (#5795)
  refactor: handle redirections for iframed content (#5591)
  refactor(payment_links): Update API contract for dynamic transaction details and upgrade UI (#5849)
  fix(router): add payment_method check in `get_mandate_type` (#5828)
  fix(connector): [ZSL] compare consr_paid_amt with the total amount for identifying partial payments (#5873)
  feat(connector): [Novalnet] add Payment flows for cards (#5726)
  chore(version): 2024.09.12.0
  fix(router): return `collect_billing_details_from_wallet_connector` if `always_collect_billing_details_from_wallet_connector ` is false in merchant payment method list (#5854)
  feat(opensearch): add profile_id and organization_id to /search APIs (#5705)
  build(deps): bump `sqlx` to `0.8.2` (#5859)
  refactor: Remove unwanted commented lines (#5851)
  feat(payments): add support for profile aggregates (#5845)
  Feat(connector): [Fiuu] Add DuitNow/FPX PaymentMethod (#5841)
  chore: remove Connectors enum dependency from ConnectorIntegrationV2 trait (#5840)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-Analytics C-feature Category: Feature request or enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

feat(opensearch): add profile_id and organization_id to /search APIs
4 participants