From be9abe367ae9e06fcb2ca3135ba12bce31db76fd Mon Sep 17 00:00:00 2001 From: Jack Vander Pol Date: Fri, 1 Nov 2024 10:16:37 -0400 Subject: [PATCH 01/10] Initial rough draft commit, does not support subexpressions yet --- .../independent-definitions-schema.xsd | 235 ++++++++++++++++++ ...ependent-system-characteristics-schema.xsd | 102 ++++++++ 2 files changed, 337 insertions(+) diff --git a/oval-schemas/independent-definitions-schema.xsd b/oval-schemas/independent-definitions-schema.xsd index a80648a..aea2dea 100644 --- a/oval-schemas/independent-definitions-schema.xsd +++ b/oval-schemas/independent-definitions-schema.xsd @@ -969,6 +969,153 @@ + + + + + The shellcommand_test is used to check the values produced by the running of the command (or embedded script, not an external script file) found in the object 'command' element. It extends the standard TestType as defined in the oval-definitions-schema and one should refer to the TestType description for more information. The required object element references a shellcommand_object and the optional state element references a shellcommand_state that specifies the information to check. + +Since this test requires the running of code supplied by content and since SCAP applications commonly run with elevated privileges, significant responsibilty falls to the content author to do no harm. This also requires that any content stream that employs this test MUST be from a known trusted source and be digitally signed. The use of any executables that are not supplied by the installed operating system is highly discouraged. + + + shellcommand_test + shellcommand_object + shellcommand_state + shellcommand_item + + + + + + - the object child element of a shellcommand_test must reference a shellcommand_object + + + - the state child element of a shellcommand_test must reference a shellcommand_state + + + + + + + + + + + + + + + + + + + The shellcommand_object is used by a shellcommand_test to define a command (or shell script) to be run and a pattern to filter result lines. Each object extends the standard ObjectType as defined in the oval-definitions-schema and one should refer to the ObjectType description for more information. The common set element allows complex objects to be created using filters and set logic. + + + + + + + + + + State referenced in filter for '' is of the wrong type. + + + + + + + + + + + + + + The shell entity defines the specific shell interpeter use. Any tool looking to collect information about this object will need to know the shell in order to use it properly. + + + + + The command element specifies the command string to be run on the target system. Since this command or script will be executed on the target system and since SCAP applications commonly run with elevated privileges, significant responsibilty falls to the content author to do no harm. This also requires that any content stream that employs this test MUST be from a known trusted source and be digitally signed. The use of executables that are not supplied by the installed operating system is highly discouraged. + + + + + The 'pattern' is a regular expression that identifies lines in 'command' results that are to produce OVAL items. Any result line that matches the pattern is kept as an OVAL item. Any that do not are discarded. If the pattern element is empty or does not exist, all results lines are kept. Importantly, this test does not support the capture of subexpressions as the textfilecontent test do. The shell command or script should produce concise results obfuscating the need for subexpression. + Note that when using regular expressions, OVAL supports a common subset of the regular expression character classes, operations, expressions and other lexical tokens defined within Perl 5's regular expression specification. For more information on the supported regular expression syntax in OVAL see: http://oval.mitre.org/language/about/re_support_5.6.html. + + + + - operation attribute for the pattern entity of a textfilecontent54_object should be 'pattern match' + + + + + + + + + + + + + + + + + The shellcommand_state contains the entity that are used to check the value(s) returned by the shellcommand_object. Note that only the 'result' element is meaningful in OVAL item to state comparisons. All other state entities (label, command, and pattern) are echoed, verbatim, from the same elements in the associated shellcommand_object. + + + + + + + + The shell element contains the shell used to perform the command. + + + + + + The 'command' element specifies the command string to be run on the target system and must match, verbatim, the same element in the associated shellcommand_object. + + + + + The 'pattern' is a regular expression that identifies lines in 'command' results that are to produce OVAL items and must match, verbatim, the same element in the associated shellcommand_object. + + + + - operation attribute for the pattern entity of a textfilecontent54_object should be 'pattern match' + + + + + + + + The exit status value from the command + + + + + The result element contains the standard output of a successful run of the command string. Each result line should produce an OVAL item. The object pattern can be used to filter (exclude) unwanted lines. + + + + + The result element contains the standard error of a run of the command string. Each result line should produce an OVAL item. The object pattern can be used to filter (exclude) unwanted lines. + + + + + + + + + @@ -2491,6 +2638,94 @@ SERVERPROPERTY('IsClustered') AS [is_clustered] + + + The EntityObjectShellType complex type defines a string entity value that is restricted to a set of command shells. The empty string is also allowed to support empty elements associated with variable references. + + + + + + The borne shell (sh) + + + + + The gnu borne again shell (bash). + + + + + The C shell (csh). + + + + + The korn shell (ksh). + + + + + The Microsoft Windows command prompt (cmd). + + + + + The Microsoft Powershell prompt (powershell). + + + + + The empty string value is permitted here to allow for empty elements associated with variable references. + + + + + + + + The EntityStateShellType complex type defines a string entity value that is restricted to a set of command shells. The empty string is also allowed to support empty elements associated with variable references. + + + + + + The borne shell (sh) + + + + + The gnu borne again shell (bash). + + + + + The C shell (csh). + + + + + The korn shell (ksh). + + + + + The Microsoft Windows command prompt (cmd). + + + + + The Microsoft Powershell prompt (powershell). + + + + + The empty string value is permitted here to allow for empty elements associated with variable references. + + + + + The EntityObjectEngineType complex type defines a string entity value that is restricted to a set of enumerations. Each valid enumeration is a valid database engine. The empty string is also allowed to support empty elements associated with variable references. diff --git a/oval-schemas/independent-system-characteristics-schema.xsd b/oval-schemas/independent-system-characteristics-schema.xsd index 37cff6b..a6ce217 100644 --- a/oval-schemas/independent-system-characteristics-schema.xsd +++ b/oval-schemas/independent-system-characteristics-schema.xsd @@ -325,6 +325,55 @@ + + + + + + The shellcommand_item stores information retrieved from the local system that result from the running of the command or embedded script in the associated object command element. + + + + + + + + The shell element contains the shell used to perform the command. + + + + + The command element specifies the command string that was run on the target system and should be taken, verbatim, from the associated object 'command' element.. + + + + + + The pattern element is simply an echo of the same element in the OVAL object and is supplied in the item to aid in end user interpretation and should be taken, verbatim, from the associated object 'pattern' element.. + + + + + The exit status value from the command + + + + + Each line of standard output from the shell command which matches the line selection + + + + + Each line of standard error output from the shell command. All standard error lines are always included in any shellcommand_item generated. + + + + + + + + + @@ -704,6 +753,59 @@ + + + + + The EntityItemShellType restricts a string value to a specific set of shell commands. The empty string is also allowed to support empty elements associated with error conditions. + + + + + + The borne shell (sh) + + + + + The gnu borne again shell (bash). + + + + + The C shell (csh). + + + + + The korn shell (ksh). + + + + + The Microsoft Windows command prompt (cmd). + + + + + The Microsoft Powershell prompt (powershell). + + + + + The empty string value is permitted here to allow for empty elements associated with variable references. + + + + + The empty string value is permitted here to allow for detailed error reporting. + + + + + + + From 36f2bfa339ed2f176b293710fdff7ddf293cd763 Mon Sep 17 00:00:00 2001 From: John Ulmer <60975458+johnulmer-oval@users.noreply.github.com> Date: Thu, 7 Nov 2024 11:12:53 -0500 Subject: [PATCH 02/10] Update independent-definitions-schema.xsd Modified to refine shellcommand_test. Removed 'label' element from object and state. Added 'shell' element to object and state. Removed 'result' element from state and replaced with 'exist_status', 'stdout_line', and 'stderror_line'. There is still some question about how to handle stderr output as the stderr info will not necessarily be directly related to a specific item. --- .../independent-definitions-schema.xsd | 30 ++++++++++++------- 1 file changed, 19 insertions(+), 11 deletions(-) diff --git a/oval-schemas/independent-definitions-schema.xsd b/oval-schemas/independent-definitions-schema.xsd index aea2dea..0b28909 100644 --- a/oval-schemas/independent-definitions-schema.xsd +++ b/oval-schemas/independent-definitions-schema.xsd @@ -968,6 +968,7 @@ + @@ -975,7 +976,7 @@ The shellcommand_test is used to check the values produced by the running of the command (or embedded script, not an external script file) found in the object 'command' element. It extends the standard TestType as defined in the oval-definitions-schema and one should refer to the TestType description for more information. The required object element references a shellcommand_object and the optional state element references a shellcommand_state that specifies the information to check. -Since this test requires the running of code supplied by content and since SCAP applications commonly run with elevated privileges, significant responsibilty falls to the content author to do no harm. This also requires that any content stream that employs this test MUST be from a known trusted source and be digitally signed. The use of any executables that are not supplied by the installed operating system is highly discouraged. + Since this test requires the running of code supplied by content and since SCAP applications commonly run with elevated privileges, significant responsibilty falls to the content author to do no harm. This also requires that any content stream that employs this test MUST be from a known trusted source and be digitally signed. The use of any executables that are not supplied by the installed operating system is highly discouraged. shellcommand_test @@ -1033,7 +1034,7 @@ Since this test requires the running of code supplied by content and since SCAP - The shell entity defines the specific shell interpeter use. Any tool looking to collect information about this object will need to know the shell in order to use it properly. + The shell entity defines the specific shell interpreter to use. Any tool looking to collect information about this object will need to know the shell in order to use it properly. @@ -1043,12 +1044,15 @@ Since this test requires the running of code supplied by content and since SCAP - The 'pattern' is a regular expression that identifies lines in 'command' results that are to produce OVAL items. Any result line that matches the pattern is kept as an OVAL item. Any that do not are discarded. If the pattern element is empty or does not exist, all results lines are kept. Importantly, this test does not support the capture of subexpressions as the textfilecontent test do. The shell command or script should produce concise results obfuscating the need for subexpression. + The 'pattern' is a regular expression that identifies lines in 'command' results that are to produce OVAL items. Any result line that matches the pattern is kept as an OVAL item. Any that do not are discarded. If the pattern element is empty or does not exist, all results lines are kept. + + A subexpression (using parentheses) can call out a piece of the matched stdout_line to test. For example, the pattern abc(.*)xyz would look for a block of text in the file that starts with abc and ends with xyz, with the subexpression being all the characters that exist in between. The value of the subexpression can then be tested using the subexpression entity of a shellcommand_state. Note that if the pattern, starting at the same point in the line, matches more than one block of text, then it matches the longest. For example, given a file with abcdefxyzxyzabc, then the pattern abc(.*)xyz would match the block abcdefxyzxyz. Subexpressions also match the longest possible substrings, subject to the constraint that the whole match be as long as possible, with subexpressions starting earlier in the pattern taking priority over ones starting later. + Note that when using regular expressions, OVAL supports a common subset of the regular expression character classes, operations, expressions and other lexical tokens defined within Perl 5's regular expression specification. For more information on the supported regular expression syntax in OVAL see: http://oval.mitre.org/language/about/re_support_5.6.html. - - operation attribute for the pattern entity of a textfilecontent54_object should be 'pattern match' + - operation attribute for the pattern entity of a shellcommand_object should be 'pattern match' @@ -1065,7 +1069,7 @@ Since this test requires the running of code supplied by content and since SCAP - The shellcommand_state contains the entity that are used to check the value(s) returned by the shellcommand_object. Note that only the 'result' element is meaningful in OVAL item to state comparisons. All other state entities (label, command, and pattern) are echoed, verbatim, from the same elements in the associated shellcommand_object. + The shellcommand_state contains the entities that are used to check the values returned by the shellcommand_object. Note that the state entities shell, command, and pattern are echoed, verbatim, from the same elements in the associated shellcommand_object. @@ -1088,7 +1092,7 @@ Since this test requires the running of code supplied by content and since SCAP - - operation attribute for the pattern entity of a textfilecontent54_object should be 'pattern match' + - operation attribute for the pattern entity of a shellcommand_object should be 'pattern match' @@ -1096,17 +1100,19 @@ Since this test requires the running of code supplied by content and since SCAP - The exit status value from the command + The exit_status entity represents the exist status returned by the system for the execution of the object command. - + - The result element contains the standard output of a successful run of the command string. Each result line should produce an OVAL item. The object pattern can be used to filter (exclude) unwanted lines. + The stdout_line entity represents a line from the STDOUT output of a successful run of the command string that matched the specified object pattern. - + - The result element contains the standard error of a run of the command string. Each result line should produce an OVAL item. The object pattern can be used to filter (exclude) unwanted lines. + The stderr_line element represents a line from the STDERR output of a run of the command string that matched the specified object pattern. Each line of STDERR produced by the command execution will produce a single shellcommand item. + + The stderr(_line) element contains any output to STDERR from a run of the object command. @@ -1115,6 +1121,8 @@ Since this test requires the running of code supplied by content and since SCAP + + From dda347449db6e4b53b740fe53c41ae6ef1e2c047 Mon Sep 17 00:00:00 2001 From: John Ulmer <60975458+johnulmer-oval@users.noreply.github.com> Date: Thu, 7 Nov 2024 11:17:54 -0500 Subject: [PATCH 03/10] Update independent-system-characteristics-schema.xsd Changes to shellcommand_item. Added 'shell' element. Removed 'label' element. Added 'exit_status, 'stdout_line', and 'stderr_line'. There is still some question about how to handle stderr output as the stderr info will not necessarily be directly related to a specific item. --- .../independent-system-characteristics-schema.xsd | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/oval-schemas/independent-system-characteristics-schema.xsd b/oval-schemas/independent-system-characteristics-schema.xsd index a6ce217..cd4d8b9 100644 --- a/oval-schemas/independent-system-characteristics-schema.xsd +++ b/oval-schemas/independent-system-characteristics-schema.xsd @@ -330,7 +330,7 @@ - The shellcommand_item stores information retrieved from the local system that result from the running of the command or embedded script in the associated object command element. + The shellcommand_item stores information retrieved from the local system that results from the running of the command or embedded script in the associated object command element. @@ -338,7 +338,7 @@ - The shell element contains the shell used to perform the command. + The shell element contains the shell used (e.g. bash or powershell) to perform the command and should be taken, verbatim, from the associated object 'shell' element. @@ -354,17 +354,19 @@ - The exit status value from the command + The exit_status entity represents the exist status returned by the system for the execution of the object command. OVAL Item status should match the exit status of the system call. - Each line of standard output from the shell command which matches the line selection + The stdout_line entity contains a line from the STDOUT output of a successful run of the command string that matched the specified object pattern. Each line created by the execution of the object command should create an item. - Each line of standard error output from the shell command. All standard error lines are always included in any shellcommand_item generated. + The stderr_line element represents a line from the STDERR output of a run of the command string that matched the specified object pattern. Each line of STDERR produced by the command execution will produce a single shellcommand item. + + The stderr_line element represents any and all output to STDERR from a run of the command string. From b5efc178244737e2e713e03f7fb996971d0c2318 Mon Sep 17 00:00:00 2001 From: John Ulmer <60975458+johnulmer-oval@users.noreply.github.com> Date: Wed, 20 Nov 2024 14:31:43 -0500 Subject: [PATCH 04/10] Update independent-definitions-schema.xsd In SHELLCOMMAND_TEST: - Added 'shell' element to object. - Removed 'label' elements from object and state. - Added 'stdout_line' element to object and state. - Added 'exit_status' element to object and state. - Added 'stderr' element to object and state. --- .../independent-definitions-schema.xsd | 51 +++++++++++-------- 1 file changed, 31 insertions(+), 20 deletions(-) diff --git a/oval-schemas/independent-definitions-schema.xsd b/oval-schemas/independent-definitions-schema.xsd index 0b28909..0072fa8 100644 --- a/oval-schemas/independent-definitions-schema.xsd +++ b/oval-schemas/independent-definitions-schema.xsd @@ -968,15 +968,17 @@ - + - The shellcommand_test is used to check the values produced by the running of the command (or embedded script, not an external script file) found in the object 'command' element. It extends the standard TestType as defined in the oval-definitions-schema and one should refer to the TestType description for more information. The required object element references a shellcommand_object and the optional state element references a shellcommand_state that specifies the information to check. + The shellcommand_test is used to check the values produced by the running of the 'command' (or script, but not an external script file) found in the object 'command' element. It extends the standard TestType as defined in the oval-definitions-schema and one should refer to the TestType description for more information. The required object element references a shellcommand_object and the optional state element references a shellcommand_state that specifies the information to check. + +Since this test runs the command string supplied in the object command element, the content author should avoid writing command strings that may produce large amounts of output or that may be fragile causing errors and thus produce large amounts of error output. The command should produce well formed output that will result in one OVAL item foreach line of output produced by the object evaluation. - Since this test requires the running of code supplied by content and since SCAP applications commonly run with elevated privileges, significant responsibilty falls to the content author to do no harm. This also requires that any content stream that employs this test MUST be from a known trusted source and be digitally signed. The use of any executables that are not supplied by the installed operating system is highly discouraged. +IMPORTANT! - Since this test requires the running of code supplied by content and since OVAL interpreters commonly run with elevated privileges, significant responsibilty falls to the content author to DO NO HARM to the target system. This also requires that any content stream that employs this test MUST be from a known trusted source and be digitally signed. The use of any executables that are not supplied by the installed operating system is highly discouraged. shellcommand_test @@ -1010,7 +1012,13 @@ - The shellcommand_object is used by a shellcommand_test to define a command (or shell script) to be run and a pattern to filter result lines. Each object extends the standard ObjectType as defined in the oval-definitions-schema and one should refer to the ObjectType description for more information. The common set element allows complex objects to be created using filters and set logic. + The shellcommand_object is used by a shellcommand_test to define a shell to use (e.g. sh, bash, ksh, etc.), a command (or shell script) to be run, and a pattern to filter result lines. The default shell is bash. Each object extends the standard ObjectType as defined in the oval-definitions-schema and one should refer to the ObjectType description for more information. The common set element allows complex objects to be created using filters and set logic. + +The evaluation of the object should always produce at least one item. If the object evaluation does not produce output that should result in an item, one should be created with a status of 'does not exist' and, if possible, the exit_status and any STDERR from the system call should be captured in that item. + +Also, note that as lines from STDERR often cannot be related to a specific line of STDOUT, each item will contain all of the output from STDERR. Content authors should be aware that this could be highly redundant and voluminous and should design object command strings accordingly. + + @@ -1032,21 +1040,22 @@ - + - The shell entity defines the specific shell interpreter to use. Any tool looking to collect information about this object will need to know the shell in order to use it properly. + The shell entity defines the specific shell to use (e.g. bash, csh, ksh, etc.). Any tool collecting information for this object will need to know the shell in order to use it properly. - The command element specifies the command string to be run on the target system. Since this command or script will be executed on the target system and since SCAP applications commonly run with elevated privileges, significant responsibilty falls to the content author to do no harm. This also requires that any content stream that employs this test MUST be from a known trusted source and be digitally signed. The use of executables that are not supplied by the installed operating system is highly discouraged. + The command element specifies the command string to be run on the target system. Since this command string will be executed on the target system and since OVAL interpreters commonly run with elevated privileges, significant responsibilty falls to the content author to DO NO HARM. This also requires that any content stream that employs this test MUST be from a known trusted source and be digitally signed. The use of executables that are not supplied by the installed operating system is highly discouraged. The 'pattern' is a regular expression that identifies lines in 'command' results that are to produce OVAL items. Any result line that matches the pattern is kept as an OVAL item. Any that do not are discarded. If the pattern element is empty or does not exist, all results lines are kept. - A subexpression (using parentheses) can call out a piece of the matched stdout_line to test. For example, the pattern abc(.*)xyz would look for a block of text in the file that starts with abc and ends with xyz, with the subexpression being all the characters that exist in between. The value of the subexpression can then be tested using the subexpression entity of a shellcommand_state. Note that if the pattern, starting at the same point in the line, matches more than one block of text, then it matches the longest. For example, given a file with abcdefxyzxyzabc, then the pattern abc(.*)xyz would match the block abcdefxyzxyz. Subexpressions also match the longest possible substrings, subject to the constraint that the whole match be as long as possible, with subexpressions starting earlier in the pattern taking priority over ones starting later. +A subexpression (using parentheses) can call out a piece of the matched stdout_line to test. For example, the pattern abc(.*)xyz would look for a block of text in the file that starts with abc and ends with xyz, with the subexpression being all the characters that exist in between. The value of the subexpression can then be tested using the subexpression entity of a shellcommand_state. Note that if the pattern, starting at the same point in the line, matches more than one block of text, then it matches the longest. For example, given a file with abcdefxyzxyzabc, then the pattern abc(.*)xyz would match the block abcdefxyzxyz. Subexpressions also match the longest possible substrings, subject to the constraint that the whole match be as long as possible, with subexpressions starting earlier in the pattern taking priority over ones starting later. + Note that when using regular expressions, OVAL supports a common subset of the regular expression character classes, operations, expressions and other lexical tokens defined within Perl 5's regular expression specification. For more information on the supported regular expression syntax in OVAL see: http://oval.mitre.org/language/about/re_support_5.6.html. @@ -1075,20 +1084,20 @@ - + - The shell element contains the shell used to perform the command. + The 'shell' element contains the shell used to perform the command and must match the value in the associated object, verbatim. - The 'command' element specifies the command string to be run on the target system and must match, verbatim, the same element in the associated shellcommand_object. + The 'command' element specifies the command string to be run on the target system and must match the same element in the associated shellcommand_object, verbatim. - The 'pattern' is a regular expression that identifies lines in 'command' results that are to produce OVAL items and must match, verbatim, the same element in the associated shellcommand_object. + The 'pattern' is a regular expression that identifies lines in 'command' results that are to produce OVAL items and must match the same element in the associated shellcommand_object, verbatim. @@ -1100,28 +1109,30 @@ - The exit_status entity represents the exist status returned by the system for the execution of the object command. + The 'exit_status' entity represents the exist status returned by the system for the execution of the object command. - The stdout_line entity represents a line from the STDOUT output of a successful run of the command string that matched the specified object pattern. + The 'stdout_line' entity represents a line from the STDOUT output of a successful run of the command string that matched the specified object pattern. - + - The stderr_line element represents a line from the STDERR output of a run of the command string that matched the specified object pattern. Each line of STDERR produced by the command execution will produce a single shellcommand item. - - The stderr(_line) element contains any output to STDERR from a run of the object command. + The subexpression entity represents a value to test against the subexpression in the specified pattern. If multiple subexpressions are specified in the pattern, this value is tested against all of them. For example, if the pattern abc(.*)mno(.*)xyp was supplied, and the state specifies a subexpression value of enabled, then the test would check that both (or at least one, none, etc. depending on the entity_check attribute) of the subexpressions have a value of enabled. + + + + + The 'stderr' element contains any and all output to STDERR from a run of the object command. - - + From 17b580b2d30975a77883116111f401cd0f063eb0 Mon Sep 17 00:00:00 2001 From: John Ulmer <60975458+johnulmer-oval@users.noreply.github.com> Date: Wed, 20 Nov 2024 14:34:25 -0500 Subject: [PATCH 05/10] Update independent-system-characteristics-schema.xsd In SHELLCOMMAND_ITEM: - Added 'shell' element. - Removed 'label' element. - Added 'stdout_line' element. - Added 'exit_status' element. - Added 'stderr' element. --- ...ependent-system-characteristics-schema.xsd | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/oval-schemas/independent-system-characteristics-schema.xsd b/oval-schemas/independent-system-characteristics-schema.xsd index cd4d8b9..ee44ceb 100644 --- a/oval-schemas/independent-system-characteristics-schema.xsd +++ b/oval-schemas/independent-system-characteristics-schema.xsd @@ -325,18 +325,22 @@ + + - The shellcommand_item stores information retrieved from the local system that results from the running of the command or embedded script in the associated object command element. + The shellcommand_item stores information retrieved from the local system that results from the running of the command or embedded script in the associated object command element. + +The evaluation of the object should always produce at least one item. If the object evaluation does not produce output that should result in an item, one should be created with a status of 'does not exist'. This facilitates that capture of the exit_status and stderr from the system call. - + The shell element contains the shell used (e.g. bash or powershell) to perform the command and should be taken, verbatim, from the associated object 'shell' element. @@ -362,11 +366,14 @@ The stdout_line entity contains a line from the STDOUT output of a successful run of the command string that matched the specified object pattern. Each line created by the execution of the object command should create an item. - + + + The subexpression entity represents the value of a subexpression in the specified pattern. If multiple subexpressions are specified in the pattern, then multiple entities are presented. Note that the textfilecontent_state in the definition schema only allows a single subexpression entity. This means that the test will check that all (or at least one, none, etc.) the subexpressions pass the same check. This means that the order of multiple subexpression entities in the item does not matter. + + + - The stderr_line element represents a line from the STDERR output of a run of the command string that matched the specified object pattern. Each line of STDERR produced by the command execution will produce a single shellcommand item. - - The stderr_line element represents any and all output to STDERR from a run of the command string. + The stderr element represents any and all output to STDERR from a run of the command string. From b39866132869a5cf5197f3b19e2d9c6cf4070778 Mon Sep 17 00:00:00 2001 From: John Ulmer <60975458+johnulmer-oval@users.noreply.github.com> Date: Fri, 22 Nov 2024 16:43:59 -0500 Subject: [PATCH 06/10] Update independent-system-characteristics-schema.xsd Modified shellcommand item so that each line of STDOUT produces a new item 'stdout_line' element and each line of STDERR produces a new item 'stderr_line' element. Corrected some documentation and min and max occurs to reflect these changes. --- .../independent-system-characteristics-schema.xsd | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/oval-schemas/independent-system-characteristics-schema.xsd b/oval-schemas/independent-system-characteristics-schema.xsd index ee44ceb..7ea752c 100644 --- a/oval-schemas/independent-system-characteristics-schema.xsd +++ b/oval-schemas/independent-system-characteristics-schema.xsd @@ -334,7 +334,7 @@ The shellcommand_item stores information retrieved from the local system that results from the running of the command or embedded script in the associated object command element. -The evaluation of the object should always produce at least one item. If the object evaluation does not produce output that should result in an item, one should be created with a status of 'does not exist'. This facilitates that capture of the exit_status and stderr from the system call. +The evaluation of the object should always produce at least one item. If the object evaluation does not produce output via STDOUT that should result in an item, one should be created with a status of 'does not exist'. This facilitates that capture of the exit_status and stderr from the system call. @@ -356,14 +356,14 @@ The evaluation of the object should always produce at least one item. If the ob The pattern element is simply an echo of the same element in the OVAL object and is supplied in the item to aid in end user interpretation and should be taken, verbatim, from the associated object 'pattern' element.. - + The exit_status entity represents the exist status returned by the system for the execution of the object command. OVAL Item status should match the exit status of the system call. - The stdout_line entity contains a line from the STDOUT output of a successful run of the command string that matched the specified object pattern. Each line created by the execution of the object command should create an item. + The stdout_line entity contains a line from the STDOUT output of a successful run of the command string that matched the specified object pattern. Each line created by the execution of the object command should create an item 'stdout_line' element. @@ -371,9 +371,9 @@ The evaluation of the object should always produce at least one item. If the ob The subexpression entity represents the value of a subexpression in the specified pattern. If multiple subexpressions are specified in the pattern, then multiple entities are presented. Note that the textfilecontent_state in the definition schema only allows a single subexpression entity. This means that the test will check that all (or at least one, none, etc.) the subexpressions pass the same check. This means that the order of multiple subexpression entities in the item does not matter. - + - The stderr element represents any and all output to STDERR from a run of the command string. + The 'stderr_line' element contains a single line of any output from STDERR. From ddb4def886039d9ad9451b2db4b6b92575381eb4 Mon Sep 17 00:00:00 2001 From: John Ulmer <60975458+johnulmer-oval@users.noreply.github.com> Date: Fri, 22 Nov 2024 16:46:33 -0500 Subject: [PATCH 07/10] Update independent-definitions-schema.xsd Changed to reflect that each line of STDOUT will produce an item 'stdout_line' element and each line of STDERR will produce an item 'stderr_line' element. --- oval-schemas/independent-definitions-schema.xsd | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/oval-schemas/independent-definitions-schema.xsd b/oval-schemas/independent-definitions-schema.xsd index 0072fa8..cc48ec0 100644 --- a/oval-schemas/independent-definitions-schema.xsd +++ b/oval-schemas/independent-definitions-schema.xsd @@ -976,7 +976,9 @@ The shellcommand_test is used to check the values produced by the running of the 'command' (or script, but not an external script file) found in the object 'command' element. It extends the standard TestType as defined in the oval-definitions-schema and one should refer to the TestType description for more information. The required object element references a shellcommand_object and the optional state element references a shellcommand_state that specifies the information to check. -Since this test runs the command string supplied in the object command element, the content author should avoid writing command strings that may produce large amounts of output or that may be fragile causing errors and thus produce large amounts of error output. The command should produce well formed output that will result in one OVAL item foreach line of output produced by the object evaluation. +Since this test runs the command string supplied in the object command element, the content author should avoid writing command strings that may produce large amounts of output or that may be fragile causing errors and thus produce large amounts of error output. + +The command should produce well formed output that will result in one item stdout_line element for each line of output via STDOUT by the object evaluation. Similarly, in the item, for any output to STDERR, a stderr_line element will be created. IMPORTANT! - Since this test requires the running of code supplied by content and since OVAL interpreters commonly run with elevated privileges, significant responsibilty falls to the content author to DO NO HARM to the target system. This also requires that any content stream that employs this test MUST be from a known trusted source and be digitally signed. The use of any executables that are not supplied by the installed operating system is highly discouraged. @@ -1015,9 +1017,6 @@ IMPORTANT! - Since this test requires the running of code supplied by content an The shellcommand_object is used by a shellcommand_test to define a shell to use (e.g. sh, bash, ksh, etc.), a command (or shell script) to be run, and a pattern to filter result lines. The default shell is bash. Each object extends the standard ObjectType as defined in the oval-definitions-schema and one should refer to the ObjectType description for more information. The common set element allows complex objects to be created using filters and set logic. The evaluation of the object should always produce at least one item. If the object evaluation does not produce output that should result in an item, one should be created with a status of 'does not exist' and, if possible, the exit_status and any STDERR from the system call should be captured in that item. - -Also, note that as lines from STDERR often cannot be related to a specific line of STDOUT, each item will contain all of the output from STDERR. Content authors should be aware that this could be highly redundant and voluminous and should design object command strings accordingly. - @@ -1052,7 +1051,7 @@ Also, note that as lines from STDERR often cannot be related to a specific line - The 'pattern' is a regular expression that identifies lines in 'command' results that are to produce OVAL items. Any result line that matches the pattern is kept as an OVAL item. Any that do not are discarded. If the pattern element is empty or does not exist, all results lines are kept. + The 'pattern' is a regular expression that identifies lines in 'command' results that are to produce OVAL items. Any result line via STDOUT that matches the pattern is kept as an item stdout_line element. Any that do not are discarded. If the pattern element is empty or does not exist, all results lines are kept. A subexpression (using parentheses) can call out a piece of the matched stdout_line to test. For example, the pattern abc(.*)xyz would look for a block of text in the file that starts with abc and ends with xyz, with the subexpression being all the characters that exist in between. The value of the subexpression can then be tested using the subexpression entity of a shellcommand_state. Note that if the pattern, starting at the same point in the line, matches more than one block of text, then it matches the longest. For example, given a file with abcdefxyzxyzabc, then the pattern abc(.*)xyz would match the block abcdefxyzxyz. Subexpressions also match the longest possible substrings, subject to the constraint that the whole match be as long as possible, with subexpressions starting earlier in the pattern taking priority over ones starting later. @@ -1112,7 +1111,7 @@ A subexpression (using parentheses) can call out a piece of the matched stdout_l The 'exit_status' entity represents the exist status returned by the system for the execution of the object command. - + The 'stdout_line' entity represents a line from the STDOUT output of a successful run of the command string that matched the specified object pattern. @@ -1122,9 +1121,9 @@ A subexpression (using parentheses) can call out a piece of the matched stdout_l The subexpression entity represents a value to test against the subexpression in the specified pattern. If multiple subexpressions are specified in the pattern, this value is tested against all of them. For example, if the pattern abc(.*)mno(.*)xyp was supplied, and the state specifies a subexpression value of enabled, then the test would check that both (or at least one, none, etc. depending on the entity_check attribute) of the subexpressions have a value of enabled. - + - The 'stderr' element contains any and all output to STDERR from a run of the object command. + The 'stderr_line' element contains any and all output to STDERR from a run of the object command. Each line of STDERR should create an additional 'stderr_line' element. From ad214eea29234614d8ec2c770c5486796eeef957 Mon Sep 17 00:00:00 2001 From: Jack Vander Pol Date: Tue, 26 Nov 2024 08:29:19 -0500 Subject: [PATCH 08/10] Several updates based on review from @solind Fixed shell to use be ShellType Added platform specific default shell Added Zsh for macOS (and some linux) Fixed subexpression notes from 'file' to 'output' --- .../independent-definitions-schema.xsd | 18 ++++++++++++++---- ...dependent-system-characteristics-schema.xsd | 9 +++++++-- 2 files changed, 21 insertions(+), 6 deletions(-) diff --git a/oval-schemas/independent-definitions-schema.xsd b/oval-schemas/independent-definitions-schema.xsd index cc48ec0..a1f5222 100644 --- a/oval-schemas/independent-definitions-schema.xsd +++ b/oval-schemas/independent-definitions-schema.xsd @@ -1039,7 +1039,7 @@ The evaluation of the object should always produce at least one item. If the ob - + The shell entity defines the specific shell to use (e.g. bash, csh, ksh, etc.). Any tool collecting information for this object will need to know the shell in order to use it properly. @@ -1053,7 +1053,7 @@ The evaluation of the object should always produce at least one item. If the ob The 'pattern' is a regular expression that identifies lines in 'command' results that are to produce OVAL items. Any result line via STDOUT that matches the pattern is kept as an item stdout_line element. Any that do not are discarded. If the pattern element is empty or does not exist, all results lines are kept. -A subexpression (using parentheses) can call out a piece of the matched stdout_line to test. For example, the pattern abc(.*)xyz would look for a block of text in the file that starts with abc and ends with xyz, with the subexpression being all the characters that exist in between. The value of the subexpression can then be tested using the subexpression entity of a shellcommand_state. Note that if the pattern, starting at the same point in the line, matches more than one block of text, then it matches the longest. For example, given a file with abcdefxyzxyzabc, then the pattern abc(.*)xyz would match the block abcdefxyzxyz. Subexpressions also match the longest possible substrings, subject to the constraint that the whole match be as long as possible, with subexpressions starting earlier in the pattern taking priority over ones starting later. +A subexpression (using parentheses) can call out a piece of the matched stdout_line to test. For example, the pattern abc(.*)xyz would look for a block of text in the output that starts with abc and ends with xyz, with the subexpression being all the characters that exist in between. The value of the subexpression can then be tested using the subexpression entity of a shellcommand_state. Note that if the pattern, starting at the same point in the line, matches more than one block of text, then it matches the longest. For example, given output with abcdefxyzxyzabc, then the pattern abc(.*)xyz would match the block abcdefxyzxyz. Subexpressions also match the longest possible substrings, subject to the constraint that the whole match be as long as possible, with subexpressions starting earlier in the pattern taking priority over ones starting later. Note that when using regular expressions, OVAL supports a common subset of the regular expression character classes, operations, expressions and other lexical tokens defined within Perl 5's regular expression specification. For more information on the supported regular expression syntax in OVAL see: http://oval.mitre.org/language/about/re_support_5.6.html. @@ -1083,7 +1083,7 @@ A subexpression (using parentheses) can call out a piece of the matched stdout_l - + The 'shell' element contains the shell used to perform the command and must match the value in the associated object, verbatim. @@ -1121,7 +1121,7 @@ A subexpression (using parentheses) can call out a piece of the matched stdout_l The subexpression entity represents a value to test against the subexpression in the specified pattern. If multiple subexpressions are specified in the pattern, this value is tested against all of them. For example, if the pattern abc(.*)mno(.*)xyp was supplied, and the state specifies a subexpression value of enabled, then the test would check that both (or at least one, none, etc. depending on the entity_check attribute) of the subexpressions have a value of enabled. - + The 'stderr_line' element contains any and all output to STDERR from a run of the object command. Each line of STDERR should create an additional 'stderr_line' element. @@ -2682,6 +2682,11 @@ SERVERPROPERTY('IsClustered') AS [is_clustered] The korn shell (ksh). + + + The Z shell (Zsh). + + The Microsoft Windows command prompt (cmd). @@ -2726,6 +2731,11 @@ SERVERPROPERTY('IsClustered') AS [is_clustered] The korn shell (ksh). + + + The Z shell (Zsh). + + The Microsoft Windows command prompt (cmd). diff --git a/oval-schemas/independent-system-characteristics-schema.xsd b/oval-schemas/independent-system-characteristics-schema.xsd index 7ea752c..29918f7 100644 --- a/oval-schemas/independent-system-characteristics-schema.xsd +++ b/oval-schemas/independent-system-characteristics-schema.xsd @@ -334,13 +334,13 @@ The shellcommand_item stores information retrieved from the local system that results from the running of the command or embedded script in the associated object command element. -The evaluation of the object should always produce at least one item. If the object evaluation does not produce output via STDOUT that should result in an item, one should be created with a status of 'does not exist'. This facilitates that capture of the exit_status and stderr from the system call. +The evaluation of the object should always produce one item. If the object evaluation does not produce output via STDOUT that should result in an item, one should be created with a status of 'does not exist'. This facilitates that capture of the exit_status and stderr from the system call. - + The shell element contains the shell used (e.g. bash or powershell) to perform the command and should be taken, verbatim, from the associated object 'shell' element. @@ -790,6 +790,11 @@ The evaluation of the object should always produce at least one item. If the ob The korn shell (ksh). + + + The Z shell (Zsh). + + The Microsoft Windows command prompt (cmd). From 97dc656ca72617a70db999dee2c54367714b4d5d Mon Sep 17 00:00:00 2001 From: Jack Vander Pol Date: Tue, 26 Nov 2024 13:55:48 -0500 Subject: [PATCH 09/10] Changed case of Zsh to zsh, removed default from shell --- oval-schemas/independent-definitions-schema.xsd | 10 +++++----- .../independent-system-characteristics-schema.xsd | 4 ++-- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/oval-schemas/independent-definitions-schema.xsd b/oval-schemas/independent-definitions-schema.xsd index a1f5222..06b8729 100644 --- a/oval-schemas/independent-definitions-schema.xsd +++ b/oval-schemas/independent-definitions-schema.xsd @@ -1039,7 +1039,7 @@ The evaluation of the object should always produce at least one item. If the ob - + The shell entity defines the specific shell to use (e.g. bash, csh, ksh, etc.). Any tool collecting information for this object will need to know the shell in order to use it properly. @@ -2682,9 +2682,9 @@ SERVERPROPERTY('IsClustered') AS [is_clustered] The korn shell (ksh). - + - The Z shell (Zsh). + The Z shell (zsh). @@ -2731,9 +2731,9 @@ SERVERPROPERTY('IsClustered') AS [is_clustered] The korn shell (ksh). - + - The Z shell (Zsh). + The Z shell (zsh). diff --git a/oval-schemas/independent-system-characteristics-schema.xsd b/oval-schemas/independent-system-characteristics-schema.xsd index 29918f7..50bb5e3 100644 --- a/oval-schemas/independent-system-characteristics-schema.xsd +++ b/oval-schemas/independent-system-characteristics-schema.xsd @@ -790,9 +790,9 @@ The evaluation of the object should always produce one item. If the object eval The korn shell (ksh). - + - The Z shell (Zsh). + The Z shell (zsh). From fd146988290fa35e2ae8467e778b6db44e8a5e71 Mon Sep 17 00:00:00 2001 From: Jack Vander Pol Date: Tue, 26 Nov 2024 15:36:17 -0500 Subject: [PATCH 10/10] Updated documentation based on feedback from @solind --- oval-schemas/independent-definitions-schema.xsd | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/oval-schemas/independent-definitions-schema.xsd b/oval-schemas/independent-definitions-schema.xsd index 06b8729..457bb3a 100644 --- a/oval-schemas/independent-definitions-schema.xsd +++ b/oval-schemas/independent-definitions-schema.xsd @@ -1016,8 +1016,8 @@ IMPORTANT! - Since this test requires the running of code supplied by content an The shellcommand_object is used by a shellcommand_test to define a shell to use (e.g. sh, bash, ksh, etc.), a command (or shell script) to be run, and a pattern to filter result lines. The default shell is bash. Each object extends the standard ObjectType as defined in the oval-definitions-schema and one should refer to the ObjectType description for more information. The common set element allows complex objects to be created using filters and set logic. -The evaluation of the object should always produce at least one item. If the object evaluation does not produce output that should result in an item, one should be created with a status of 'does not exist' and, if possible, the exit_status and any STDERR from the system call should be captured in that item. - +The evaluation of the object should always produce one item. If the command execution does not produce output, an item should still be created with the exit_status (AKA process exit code), a stdout entity with a status of 'does not exist', and any STDERR from the execution captured in stderr_line entities. +