Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename some files and functions to be more standard #13

Merged
merged 1 commit into from
Sep 4, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 23 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
[![Custom badge](https://img.shields.io/endpoint?color=%2374b9ff&url=https%3A%2F%2Fraw.luolix.top%2unfoldingWord%2Fcontent-validation%2Fmaster%2Fcoverage%2Fshields.json)]()
[![Install, Build & Run Cypress](https://github.com/unfoldingWord/door43-content-validation/workflows/Install,%20Build%20&%20Run%20Cypress/badge.svg)]()
[![Install, Build & Run Cypress](https://github.com/unfoldingWord/uw-content-validation/workflows/Install,%20Build%20&%20Run%20Cypress/badge.svg)]()

# uW Content/Resource Validation functions

GH Pages: https://unfoldingword.github.io/door43-content-validation/
GH Pages: https://unfoldingword.github.io/uw-content-validation/

This repository contains JavaScript functions for validating/checking for errors in text that is passed to the functions. This text might be a line in a file (especially a TSV file when a line contains a number of distinct fields), or the entire text of a file that's perhaps open in an editor in the enclosing app.

Expand All @@ -18,7 +18,7 @@ There are three sample notice processing functions that show how to:
1. Divide the noticeList into a list of severe, medium, and low priority warnings,
1. Convert the noticeList into a list of warnings sorted by priority,

In addition, there are Styleguidist pages viewable at https://unfoldingword.github.io/door43-content-validation/ which show how these core functions may be used, effectively producing a primitive app that checks Door43.org files, repositories (repos), book packages, etc. as well as demonstrating the basic functions.
In addition, there are Styleguidist pages viewable at https://unfoldingword.github.io/uw-content-validation/ which show how these core functions may be used, effectively producing a primitive app that checks Door43.org files, repositories (repos), book packages, etc. as well as demonstrating the basic functions.

## The Stack

Expand Down Expand Up @@ -49,15 +49,22 @@ The top-level checking demonstrations:

However, the lower-level checking functions provide only the list of success message strings and one list of `notices` (i.e., warnings/errors combined) typically consisting of an object with some or all of the following fields (as available/relevant):

1. A notice priority number in the range 1-1000. Each different type of warning/error has a unique number (but not each instance of those warnings/errors). By default, notice priority numbers 700 and over are considered `errors` and 0-699 are considered `warnings`.
1. The actual general descriptive text of the notice
1. The 3-character UPPERCASE [book identifier](http://ubsicap.github.io/usfm/identification/books.html) or [OBS](https://www.openbiblestories.org/) (if relevant)
1. The chapter number or story number (if relevant)
1. The verse number or frame number (if relevant)
1. A one-based line number in the file (if available)
1. A zero-based integer character index which indicates the position of the error in the given text (line or file) (if relevant)
1. An extract (if available) of the checked text which indicates the area containing the problem. Where helpful, some character substitutions have already been made, for example, if the notice is about spaces, it is generally helpful to display spaces as a visible character in an attempt to best highlight the issue to the user. (The length of the extract defaults to ten characters, but is settable as an option.)
1. A string indicating the context of the notice, e.g., `in line 17 of 'someBook.usfm'`.
There are two compulsory fields in all of these notice objects:

1. `priority`: A notice priority number in the range 1-1000. Each different type of warning/error has a unique number (but not each instance of those warnings/errors). By default, notice priority numbers 700 and over are considered `errors` and 0-699 are considered `warnings`.
1. `message`: The actual general descriptive text of the notice

All of the following fields may be missing or undefined, i.e., they're all optional:

1. `bookID`: The 3-character UPPERCASE [book identifier](http://ubsicap.github.io/usfm/identification/books.html) or [OBS](https://www.openbiblestories.org/) (if relevant)
1. `C`: The chapter number or story number (if relevant)
1. `V`: The verse number or frame number (if relevant)
1. `filename`: filename string (if available)
1. `repoName`: repository name (if available)
1. `lineNumber`: A one-based line number in the file (if available)
1. `characterIndex`: A zero-based integer character index which indicates the position of the error in the given text (line or field) (if available)
1. `extract`: An extract (if available) of the checked text which indicates the area containing the problem. Where helpful, some character substitutions have already been made, for example, if the notice is about spaces, it is generally helpful to display spaces as a visible character in an attempt to best highlight the issue to the user. (The length of the extract defaults to ten characters, but is settable as an option.)
1. `location`: A string indicating the context of the notice, e.g., "in line 17 of 'someBook.usfm'". (Still not completely sure what should be in this string now that we have added optional `filename`, `repoName`, `lineNumber` fields.)


Keeping our notices in this format, rather than the simplicity of just saving an array of single strings, allows the above *notice components* to be processed at a higher level, e.g., to allow user-controlled filtering, sorting, etc. The default is to funnel them all through the supplied `processNoticesToErrorsWarnings` function (in core/notice-processing-functions.fs) which does the following:
Expand All @@ -75,15 +82,15 @@ Still unfinished (in rough priority order):
1. Finish adding lineNumber, fileName, repoName as separate optional notice fields
1. Consider fetching TA and TW as zip files when checking links to those resources
1. Standardise parameters according to best practice (i.e., dereferencing, etc.)
1. Document the API
1. Checking of general markdown and naked links
1. Document the API with (JsDoc)
1. Checking of general markdown and naked links (esp. in plain text and markdown files)
1. Testing and fine-tuning of error messages (e.g., comparing with tX), especially suppression of false alarms
1. Improve general documentation
1. Improve general documentation in the code and readMe files
1. Optimise various different file fetches and caches (incl. using zips) for the demos
1. Is our `RepoCheck` the same as `ResourceContainerCheck`? Or is the latter more specific?
1. Understand and standardise React stuff in the demos, e.g., e.g., withStyles, etc.
1. Write unit tests (especially for the core functions) and get them passing, including on GitHub push -- is it Cypress or Jest that's the right tool for this? Or Jest for the core functions? Cypress for some basic tests of the demos?
1. Write the correct checks for the new TSV annotation formats
1. Write the correct checks for the forthcoming new TSV annotation formats
1. Check for and remove left-over (but unused) code from the source projects that the original code was copied from
1. Remove all debug code and console logging, and to consider possible speed and memory optimizations
1. Add a Scripture Burrito check (once Door43 has that available)
Expand Down
2 changes: 1 addition & 1 deletion noticeList.txt
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ Got 96 notices:
158, "Unexpected space(s) beside ellipse characters", ix, extract
158, "Unexpected space(s) beside ellipse character", ix, extract
148, "'checking' key is missing", -1, ""
111, `Bad options for doBasicLinkChecks: expectedCount=${linkOptions.expectedCount} but allowedCount=${linkOptions.allowedCount}`, -1, "", ""
111, `Bad options for checkFieldLinks: expectedCount=${linkOptions.expectedCount} but allowedCount=${linkOptions.allowedCount}`, -1, "", ""
107, "Unexpected leading break", 0, extract
106, "Unexpected leading space" + (fieldText[1] === ' ' ? "s" : ""), 0, extract
105, "Unexpected trailing space(s)", fieldText.length - 1, extract
Expand Down
8 changes: 4 additions & 4 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
{
"name": "uw-content-validation",
"description": "Functions for Checking uW Content/Resources.",
"version": "0.8.4",
"description": "Functions for Checking Door43.org Scriptural Content/Resources.",
"version": "0.8.5",
"private": false,
"homepage": "https://unfoldingword.github.io/content-validation/",
"homepage": "https://unfoldingword.github.io/uw-content-validation/",
"repository": {
"type": "git",
"url": "https://github.com/unfoldingWord/content-validation/"
"url": "https://github.com/unfoldingWord/uw-content-validation/"
},
"license": "MIT",
"engines": {
Expand Down
2 changes: 1 addition & 1 deletion src/core/BCS-usfm-grammar-check.js
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ export function checkUSFMGrammar(bookID, strictnessString, filename, givenText,
* @description - adds a new notice entry, adding bookID,C,V to the given fields
* @param {Number} priority - notice priority from 1 (lowest) to 999 (highest)
* @param {String} message - the text of the notice message
* @param {Number} characterIndex - where the issue occurs in the line (or -1 if unknown)
* @param {Number} characterIndex - where the issue occurs in the line
* @param {String} extract - short extract from the line centred on the problem (if available)
* @param {String} location - description of where the issue is located
*/
Expand Down
4 changes: 2 additions & 2 deletions src/core/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@

This non-component function can have playgrounds to test them out.

`doBasicTextChecks()` does basic checks of a text field.
`checkTextField()` does basic checks of a text field.

`doBasicLinkChecks()` checks text fields that might either be a link or contain one or more links.
`checkFieldLinks()` checks text fields that might either be a link or contain one or more links.

`checkTN_TSVDataRow()` checks a line of tab-separated fields.

Expand Down
Loading