Discrepancy in Validation Failure Output

[mikeradka]

I encountered a Schema Validation issue with our Github Actions. Despite repeatedly receiving the message FAILED: Required keys are present, the validation problem stemmed from a missing attribute in the dictionary. Upon adding the attribute, the failure message ceased. Could we explore enhancing the output of our validation process? With the discrepancy in messaging, there were recurrent validation failures without a clear path to resolution, as the message implied a required field needed fixing.

---

• Example Failure ('required keys' message): https://github.com/ocsf/ocsf-schema/actions/runs/8729744683/job/23952309619#step:5:112
• What fixed it (added a dictionary attribute): 0f432f7

[alanisaac]

There is also an earlier error in the GitHub action run above here:

TESTING: Required keys are present
   ERROR: Missing required key `OcsfAttr.caption` at `attributes.log_file` in events/system/event_log.json

But that error also does not point out how to resolve it.

Enhancements we could make:

• Provide resolutions for any error where a common resolution is known.
• Only summarize errors (not passes) on failure, and include the error messages to make them more obvious. That would be a similar pattern to how many code testing frameworks work, for example.
• (nice to have): colorize output in GitHub actions to make all messages (errors, warnings, etc.) in the log more prominent and easier to scan

[alanisaac]

I'm going to leave the second bullet alone for now, and close this as is. The other two bullets have been implemented, and should help with both making the error more prominent and the resolution more clear.