Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pattern Validation - Max Length Issue Fix #1426

Merged
merged 27 commits into from
Dec 15, 2024
Merged

Conversation

samyakOO7
Copy link
Collaborator

@samyakOO7 samyakOO7 commented Nov 14, 2024

Pattern validation was taking priority in response schema for stub and request schema for test over max length parameter due to random() method generating value more than default

Fixed the use case putting a constraint of max length on random() method so that it never generates value for than max length keeping the current logic intact

Test Spec File - validate-regex-spec.yaml.zip

Backward Compatibility Spec (v2.0.36 release check) -
validate-regex-spec.yaml.zip

Logic used -

Default length - 5 (remains intact)

If minimum and maximum is both supplied, pattern length would be in range of (min to max)

If minimum length is not supplied in spec -> and maximum is less than default (5), pattern would be of length (1 i.e. generex min value, max)
Say max = 4 is only supplied
pattern length would be in range (1-4)

If minimum length is not supplied in spec -> and maximum is greater than default (5), pattern would be of length (5 (default, max)
Say max = 15 is only supplied
pattern length would be in range (5-15)

if max is not supplied and min value exist we will assume to take pattern length from (min , generex max value)

Test for maximum length conditions :
Request : {minLength : 2, maxLength: 11}
Response : {minLength : 2, maxLength: 6}

Before Stub :

curl -X POST -H "Content-Type: application/json" -d '{ "number": "12000000" }' http://0.0.0.0:49272/validate
{
    "number": "873.13863"
}

Generated value : 9 digit

Before Test:
Details:

Scenario: POST /validate -> 200 has FAILED
In scenario "Validate the input number. Response: Input is valid"
API: POST /validate -> 200

  In scenario "Validate the input number. Response: Input is valid"
     API: POST /validate -> 200
   
       REQUEST.BODY.number Contract expected string with maxLength 2 but request contained "2.8837947"

After Issue Fixed Stub:

curl -X POST -H "Content-Type: application/json" -d '{ "number": "12000000" }' http://0.0.0.0:9000/validate
{
    "number": "8.9162"
}

After Issue Fixed Test:

Request:
POST /validate
Accept-Charset: UTF-8
Accept: */*
Content-Type: application/json

{
    "number": "3."
}

Response:
200 OK
Vary: Origin
X-Specmatic-Result: success
X-Specmatic-Type: random
Content-Length: 26
Content-Type: application/json
Connection: keep-alive

{
    "number": "3618.9"
}

Test for Minimum Length conditions:
Request : {minLength : 2, maxLength: 11}
Response : {minLength : 2, maxLength: 6}

Stub:
Before Fix (The output generated was defaulting to 5 as minimum for num<5 (breaching min validation):

curl -X POST -H "Content-Type: application/json" -d '{ "number": "1" }' http://0.0.0.0:54811/validate
{
    "number": "2.866"
}
 curl -X POST -H "Content-Type: application/json" -d '{ "number": "1" }' http://0.0.0.0:54811/validate
{
    "number": "235.4"
}
curl -X POST -H "Content-Type: application/json" -d '{ "number": "1" }' http://0.0.0.0:54811/validate
{
    "number": "8.150"
}

After Fix (Min validation is respected):

curl -X POST -H "Content-Type: application/json" -d '{ "number": "1" }' http://0.0.0.0:9000/validate
{
    "number": "2."
}
 curl -X POST -H "Content-Type: application/json" -d '{ "number": "1" }' http://0.0.0.0:9000/validate
{
    "number": "5.1"
}
curl -X POST -H "Content-Type: application/json" -d '{ "number": "1" }' http://0.0.0.0:54811/validate
{
    "number": "8.150"
}
 curl -X POST -H "Content-Type: application/json" -d '{ "number": "1" }' http://0.0.0.0:9000/validate
{
    "number": "39"
}

Test :
Before Fix:

Request:
POST /validate
Accept-Charset: UTF-8
Accept: */*
Content-Type: application/json

{
    "number": "6.7293812038"
}


Response:
400 Bad Request
Vary: Origin
X-Specmatic-Result: failure
Content-Length: 411
Content-Type: text/plain
Connection: keep-alive

In scenario "Validate the input number. Response: Input is valid"
API: POST /validate -> 200

  >> REQUEST.BODY.number
  
     Contract expected string with maxLength 2 but request contained "6.7293812038"

After Fix :

Request:
POST /validate
Accept-Charset: UTF-8
Accept: */*
Content-Type: application/json

{
    "number": "9."
}
Response:
200 OK
Vary: Origin
X-Specmatic-Result: success
X-Specmatic-Type: random
Content-Length: 23
Content-Type: application/json
Connection: keep-alive

{
    "number": "407"
}

Some edge cases taken care of defaults -


Request { maxLength = 4 }
Response { maxLength = 4 }

min = null and max (4) < default (5) [pattern (min ‎ =  Generex(min val = 1), max = 4)]

Stub-

curl -X POST -H "Content-Type: application/json" -d '{ "number": "1" }' http://0.0.0.0:9000/validate
{
    "number": "7"
}
curl -X POST -H "Content-Type: application/json" -d '{ "number": "1" }' http://0.0.0.0:9000/validate
{
    "number": "3.47"
}

Test -

Request:
POST /validate
Accept-Charset: UTF-8
Accept: */*
Content-Type: application/json

{
    "number": "65.0"
}
Response:
200 OK
Vary: Origin
X-Specmatic-Result: success
X-Specmatic-Type: random
Content-Length: 23
Content-Type: application/json
Connection: keep-alive

{
    "number": "4.72”
}

Request { maxLength = 15 }
Response { maxLength = 15 }
min length = null and max length > default (5) [pattern (min ‎ =  default(5), max = 15)]

Stub-

curl -X POST -H "Content-Type: application/json" -d '{ "number": "1" }' http://0.0.0.0:9000/validate
{
    "number": "8.915"
}
curl -X POST -H "Content-Type: application/json" -d '{ "number": "1" }' http://0.0.0.0:9000/validate
{
    "number": "9.609"
}
curl -X POST -H "Content-Type: application/json" -d '{ "number": "1" }' http://0.0.0.0:9000/validate
{
    "number": "7.09024600"
}

Test -

Request:
POST /validate
Accept-Charset: UTF-8
Accept: */*
Content-Type: application/json

{
    "number": "4.22214"
}
Response:
200 OK
Vary: Origin
X-Specmatic-Result: success
X-Specmatic-Type: random
Content-Length: 25
Content-Type: application/json
Connection: keep-alive

{
    "number": "2.648"
}


Clear outputs respecting validation schema are found with multiple test of min and max length

Thanks

Samy and others added 8 commits November 16, 2024 11:00
- Removing comments and instead renaming variables for readability
- Removing redundant rows in parameterised test for min and max length based String generation
- Adding test for regex based generation
- Extracting regex generation code to separate method
Copy link
Member

@harikrishnan83 harikrishnan83 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

Samy and others added 7 commits November 27, 2024 12:53
* main: (124 commits)
  Fix schema regex and generation
  2.0.43
  Generate 4xx responses as per the schema in virtual-service be it any pattern containing a json object pattern
  Ignore a seed data entry which involves attribute selection during the seed data load stage of the virtual service
  Ignore a seed data entry with an id which is pre-existing in the cache of the virtual service
  Fix failing tests due to stale config.
  Add test for ExamplePreProcessor
  Add tests for Asserts.
  Cleanup on amber validation.
  2.0.42
  Changes for GoBack
  Added Details Pre Div
  Removed highlighting of specmatic errors in editor
  Editor syntax highlighting
  Fix full pattern validation with ListPattern.
  Add tests for partially valid examples.
  Better mismatchMessages for missing optional keys.
  better variable namings and partial handling
  Interactive Server partially valid examples.
  Modify result logic when all keys are mandatory.
  ...
- adding tests for contradicting length constraints and regex
@harikrishnan83 harikrishnan83 merged commit 5446ced into main Dec 15, 2024
2 checks passed
@harikrishnan83 harikrishnan83 deleted the pattern_max_length_fix branch December 15, 2024 05:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants