feat(webhooks): add support for custom outgoing webhook http headers #5275

srujanchikke · 2024-07-10T12:07:49Z

Type of Change

Description

This pr adds support for custom outgoing webhook http headers. If we pass custom_outgoing_webhook_http_headers to business profile these headers will be forwaded to all outgoing webhooks request headers.

Additional Changes

This PR modifies the API contract
This PR modifies the database schema
This PR modifies application configuration/environment variables

Motivation and Context

How did you test it?

Test case 1 :
Create or Update the business profile with custom http headers

curl --location 'http://localhost:8080/account/merchant_1720605139/business_profile/pro_vGz02PKyssUnujEpTwee' \
--header 'Content-Type: application/json' \
--header 'api-key: test_admin' \
--data '{
  "profile_name": "food",
  "outgoing_webhook_custom_http_headers": {
    "authentication" : "this is test key header",
    "client_id" : "askdjfhaksjhdf",
    "client_key" : "asdkfjhakdjh"
  }

}'

Create payment you should should be able to see custom headers in the configure test webhook site.

curl --location 'http://localhost:8080/payments' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--header 'api-key: dev_WYTknsPs0yvmcht1il3zk5bwvieOxmpk43I1O17xUrGDMgFoKYqUErBeafZw43Tg' \
--data-raw '{
    "amount":1000,
    "currency": "EUR",
    "confirm": true,
    "capture_method": "automatic",
    "capture_on": "2029-09-10T10:11:12Z",
    "amount_to_capture": 1000,
    "customer_id": "AdyenCustomer",
    "email": "guest@example.com",
    "name": "John Doe",
    "phone": "999999999",
    "phone_country_code": "+1",
    "description": "Its my first payment request",
    "authentication_type": "no_three_ds",
    "return_url": "https://google.com",
    "payment_method_data": {
        "card": {
            "card_number": "4111111145551142",
            "card_exp_month": "03",
            "card_exp_year": "30",
            "card_holder_name": "chikke.srujan@juspay.in",
            "card_cvc": "737",
            "card_network": "Visa",
            "card_type": null,
            "card_issuing_country": null,
            "bank_code": null
        },
        "billing": {
            "address": {
                "line1": "1467",
                "line2": "CA",
                "line3": "CA",
                "city": "San Fransico",
                "state": "California",
                "zip": "94122",
                "country": "NL",
                "first_name": "joseph",
                "last_name": "Doe"
            },
            "phone": {
                "number": "8056594427",
                "country_code": "+91"
            }
        }
    },
    "payment_method": "card",
    "payment_method_type" :"credit",
    "customer_acceptance": {
        "acceptance_type": "online",
        "accepted_at": "2026-07-04T10:59:42.232Z",
        "online": {
            "ip_address": null,
            "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36 Edg/126.0.0.0"
        }
    },
    "browser_info": {
        "color_depth": 24,
        "java_enabled": true,
        "java_script_enabled": true,
        "language": "en-GB",
        "screen_height": 720,
        "screen_width": 1280,
        "time_zone": -330,
        "ip_address": "208.127.127.193",
        "accept_header": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8",
        "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36 Edg/126.0.0.0"
    },
    "shipping": {
        "address": {
            "line1": "1467",
            "line2": "CA",
            "line3": "CA",
            "city": "San Fransico",
            "state": "California",
            "zip": "94122",
            "country": "US",
            "first_name": "joseph",
            "last_name": "Doe"
        },
        "phone": {
            "number": "8056594427",
            "country_code": "+91"
        }
    },
    "metadata": {
        "udf1": "value1",
        "new_customer": "true",
        "login_date": "2019-09-10T10:11:12Z"
    }
}'

Test case 2 : Pass random string to the field custom_outgoing_webhook_http_headers , this should not trigger custom headers to outgoing webhook, but this will show warning in the logs

Testcase 3 : Check how data is stored in db, outgoing_webhook_custom_http_headers should be encrypted in db.

Test case 4: check whether retry for outgoing webhooks have custom headers . You can verify it by passing invalid return url to create 4xx webhook.

Checklist

I formatted the code cargo +nightly fmt --all
I addressed lints thrown by cargo clippy
I reviewed the submitted code
I added unit tests for my changes where possible

sai-harsha-vardhan · 2024-07-10T13:58:45Z

api-reference/openapi_spec.json

@@ -6991,6 +6991,11 @@
              }
            ],
            "nullable": true
+          },
+          "custom_outgoing_webhook_http_headers": {


Suggested change

"custom_outgoing_webhook_http_headers": {

"outgoing_webhook_custom_http_headers": {

sai-harsha-vardhan · 2024-07-10T14:02:34Z

crates/api_models/src/admin.rs

+
+    /// These key-value pairs are sent as additional custom headers in the outgoing webhook request. It is recommended not to use more than four key-value pairs.  
+    #[schema(value_type = Option<Object>, example = r#"{ "key1": "value-1", "key2": "value-2" }"#)]
+    pub custom_outgoing_webhook_http_headers: Option<pii::SecretSerdeValue>,


We should accept it as an Hashmap here right?

sai-harsha-vardhan · 2024-07-10T14:14:40Z

crates/router/src/core/webhooks/outgoing.rs

+        let custom_outgoing_webhook_http_headers = business_profile
+            .custom_outgoing_webhook_http_headers
+            .clone();
+        let hashset = utils::convert_serde_json_to_hashset(custom_outgoing_webhook_http_headers)


Can we use serde_json::from_value here instead?

crates/router/src/core/admin.rs

SanchithHegde · 2024-07-16T10:29:30Z

crates/router/src/core/webhooks/outgoing.rs

    .change_context(errors::ApiErrorResponse::WebhookProcessingFailure)
-    .attach_printable("Failed to construct outgoing webhook request content")?;
+    .attach_printable("Failed to construct outgoing webhook custom HTTP headers")?;


Can you revert this change?

SanchithHegde · 2024-07-16T10:45:01Z

crates/router/src/core/webhooks/outgoing.rs

+        let custom_headers = decrypt::<serde_json::Value, masking::WithType>(
+            business_profile
+                .outgoing_webhook_custom_http_headers
+                .clone(),
+            key_store.key.get_inner().peek(),
+        )


We'd preferably want to have this implementation as part of domain type implementation rather than encrypting/decrypting it individually in separate locations. We can take up the domain type implementation in a separate PR.

Sure will take this in separate pr .

SanchithHegde · 2024-07-16T10:53:14Z

crates/router/src/types/api/admin.rs

+        routing_algorithm: Some(serde_json::json!({
+            "algorithm_id": null,
+            "timestamp": 0
+        })),


Do we still need this hardcoded value?

@SanchithHegde this was hardcoded for backward compatibility.
cc: @Aprabhat19

* 'main' of github.com:juspay/hyperswitch: (25 commits) fix(logs): ignore request headers while logging (#5273) feat(webhooks): add support for custom outgoing webhook http headers (#5275) fix(payment_methods): set `requires_cvv` to false when either `connector_mandate_details` or `network_transaction_id` is present during MITs (#5331) chore: create justfile for running commands for v1 and v2 migrations (#5325) fix(routing): do not update `perform_session_flow_routing` output if the `SessionRoutingChoice` is none (#5336) fix(database): modified_at updated for every state change for Payment Attempts (#5312) feat(mca): Added recipient connector call for open banking connectors (#3758) chore(version): 2024.07.16.0 refactor(connector): [Mifinity] add a field language_preference in payment request for mifinity payment method data (#5326) fix(router): store `customer_acceptance` in payment_attempt, use it in confirm flow for delayed authorizations like external 3ds flow (#5308) feat(proxy): add support to pass proxy bypass urls from configs (#5322) Docs: Updating Error codes in API-ref (#5296) feat(core): [Payouts] Add retrieve flow for payouts (#4936) fix(connector): [AUTHORIZEDOTNET] Populate error reason for failure transactions (#5319) chore(version): 2024.07.15.0 feat(logging): Emit a setup error when a restricted keys are used for logging default keys (#5185) feat(payment_methods): add support to migrate existing customer PMs from processor to hyperswitch (#5306) feat(connector): [DATATRANS] Implement card payments (#5028) chore: making of function create_encrypted_data (#5251) fix(payments): populate merchant order ref id in list (#5310) ...

add custom outgoing webhook http headers

98e055c

srujanchikke self-assigned this Jul 10, 2024

srujanchikke requested review from a team as code owners July 10, 2024 12:07

hyperswitch-bot bot added the M-database-changes Metadata: This PR involves database schema changes label Jul 10, 2024

docs(openapi): re-generate OpenAPI specification

1f255e1

hyperswitch-bot bot added the M-api-contract-changes Metadata: This PR involves API contract changes label Jul 10, 2024

srujanchikke added A-core Area: Core flows C-feature Category: Feature request or enhancement S-waiting-on-review Status: This PR has been implemented and needs to be reviewed labels Jul 10, 2024

srujanchikke requested a review from sai-harsha-vardhan July 10, 2024 12:16

srujanchikke requested review from a team as code owners July 10, 2024 13:24

srujanchikke force-pushed the custom_outgoint_webhook_http_headers branch from 6aef41c to 1f255e1 Compare July 10, 2024 13:26

srujanchikke and others added 5 commits July 10, 2024 18:56

Merge branch 'main' into custom_outgoint_webhook_http_headers

02befe5

update api reference comment

0d0b464

docs(openapi): re-generate OpenAPI specification

61f7e12

resolve comments

8f82e70

schema fix

b3aa411

sai-harsha-vardhan reviewed Jul 11, 2024

View reviewed changes

srujanchikke and others added 2 commits July 11, 2024 13:50

Merge branch 'main' into custom_outgoint_webhook_http_headers

a0f31af

resolve comments

4fbf406

srujanchikke removed request for a team July 11, 2024 17:59

Chikke Srujan and others added 3 commits July 16, 2024 13:54

postman check fix

00437b9

merge main

b792ff3

chore: run formatter

207fc92

SanchithHegde reviewed Jul 16, 2024

View reviewed changes

Chikke Srujan added 2 commits July 16, 2024 16:30

resolve conficts

221974f

resolve comments

579813e

SanchithHegde previously approved these changes Jul 16, 2024

View reviewed changes

sai-harsha-vardhan previously approved these changes Jul 16, 2024

View reviewed changes

Aprabhat19 previously approved these changes Jul 16, 2024

View reviewed changes

Gnanasundari24 added this pull request to the merge queue Jul 16, 2024

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jul 16, 2024

srujanchikke and others added 2 commits July 16, 2024 19:48

Merge branch 'main' into custom_outgoint_webhook_http_headers

acf447b

update_schema_v2

bb9f3b5

srujanchikke dismissed stale reviews from sai-harsha-vardhan, Aprabhat19, and SanchithHegde via bb9f3b5 July 16, 2024 14:42

Aprabhat19 previously approved these changes Jul 16, 2024

View reviewed changes

fix schema_v2

9ae3694

srujanchikke dismissed Aprabhat19’s stale review via 9ae3694 July 16, 2024 14:43

SanchithHegde approved these changes Jul 16, 2024

View reviewed changes

Aprabhat19 approved these changes Jul 16, 2024

View reviewed changes

sai-harsha-vardhan approved these changes Jul 16, 2024

View reviewed changes

Gnanasundari24 enabled auto-merge July 16, 2024 15:18

Gnanasundari24 added this pull request to the merge queue Jul 16, 2024

Merged via the queue into main with commit 101b21f Jul 16, 2024
17 checks passed

Gnanasundari24 deleted the custom_outgoint_webhook_http_headers branch July 16, 2024 16:37

pixincreate removed the S-waiting-on-review Status: This PR has been implemented and needs to be reviewed label Jul 16, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(webhooks): add support for custom outgoing webhook http headers #5275

feat(webhooks): add support for custom outgoing webhook http headers #5275

srujanchikke commented Jul 10, 2024 •

edited

Loading

sai-harsha-vardhan Jul 10, 2024

sai-harsha-vardhan Jul 10, 2024

sai-harsha-vardhan Jul 10, 2024

SanchithHegde Jul 16, 2024

SanchithHegde Jul 16, 2024 •

edited

Loading

srujanchikke Jul 16, 2024 •

edited

Loading

SanchithHegde Jul 16, 2024

srujanchikke Jul 16, 2024

	"custom_outgoing_webhook_http_headers": {
	"outgoing_webhook_custom_http_headers": {

feat(webhooks): add support for custom outgoing webhook http headers #5275

feat(webhooks): add support for custom outgoing webhook http headers #5275

Conversation

srujanchikke commented Jul 10, 2024 • edited Loading

Type of Change

Description

Additional Changes

Motivation and Context

How did you test it?

Checklist

sai-harsha-vardhan Jul 10, 2024

Choose a reason for hiding this comment

sai-harsha-vardhan Jul 10, 2024

Choose a reason for hiding this comment

sai-harsha-vardhan Jul 10, 2024

Choose a reason for hiding this comment

SanchithHegde Jul 16, 2024

Choose a reason for hiding this comment

SanchithHegde Jul 16, 2024 • edited Loading

Choose a reason for hiding this comment

srujanchikke Jul 16, 2024 • edited Loading

Choose a reason for hiding this comment

SanchithHegde Jul 16, 2024

Choose a reason for hiding this comment

srujanchikke Jul 16, 2024

Choose a reason for hiding this comment

srujanchikke commented Jul 10, 2024 •

edited

Loading

SanchithHegde Jul 16, 2024 •

edited

Loading

srujanchikke Jul 16, 2024 •

edited

Loading