Skip to content

(Indexed) cmi5 Conformance Testing Requirements Draft

Jump to bottom Edit New page
Bill McDonald edited this page Sep 15, 2017 · 21 revisions

Draft as of Sept 15th, 2017

Structure

Prerequisite xAPI Test (LMS/LRS)

Testing Scenarios (AU and LMS)

Test Scenario #1 – xxx

  • Prerequisites
  • Spec Requirements
  • Procedure/Steps
    • When this is done, this happens, this is verified

Testing Requirements List (AU and LMS)

Testing Requirements Document

1.0 Overview

1.1 Scope

2.0 References

3.0 Definitions

  • cmi5_3.0_1 Internationalized Resource Identifier (IRI): All id's must conform to RFC 3987

  • cmi5_3.0_2 Learning Management System (LMS): Must be xAPI conformant (must pass ADL LRS conformance tests.)

  • cmi5_3.0_3 Learning Management System (LMS): LRS must be integrated with LRS (comment – may be indirectly tested)

3.1 Abbreviations and Acronyms

4.0 Conformance

4.1 Assignable Unit (AU)

4.2 Learning Management Systems (LMS)

  • cmi5_4.2_1 Learning Management System (LMS): The LMS MUST (have access to) be able to retrieve all Resource data (from the Statement API, etc., including attachments and extensions) about another distinct user across multiple sessions for that user.

Reference the following Requirements:

4.3 Optional JSON Values

4.4 Courses

5.0 Conceptual Model: Informative

6.0 LMS Requirements

6.1 Course Structures

  • cmi5_6.1_1 Learning Management System (LMS): The LMS MUST be able to import and process course structures containing more than 1000 AU's.

6.2 LMS State API Requirements

6.3 LMS Statement API Requirements

  • cmi5_6.3_1 The LMS MUST NOT provide permissions/credentials which allow the AU to issue voiding Statements.

  • cmi5_6.3_2 The LMS MUST Void statements that are NOT rejected AND conflict with the "Statement API" requirements as defined in Section 9

  • cmi5_6.3_3 All "cmi5 allowed" and "cmi5 defined" statements issued by the LMS MUST include the "content template" as defined in section 10.0 xAPI State Data Model.

7.0 AU Requirements

7.1 AU Statement API Requirements

7.1.1 First Statement API Call

  • cmi5_7.1.1_1 Assignable Unit (AU): The AU MUST issue an initialized statement before issuing any other statements in a session.

7.1.2 Last Statement Call

  • cmi5_7.1.2_1 Assignable Unit (AU): The AU MUST issue a Terminated statement as the last statement in a session.

7.1.3 Types of Statements

  • cmi5_7.1.3_1 If "cmi5 allowed" statements are posted by the AU, they MUST occur between cmi5 "initialized" and "terminated" statements in the same session.

  • cmi5_7.1.3_2 (Place holder) "cmi5 allowed" statements are not considered in cmi5 defined session management and completion rules.

  • cmi5_7.1.3_3 All "cmi5 allowed" and "cmi5 defined" statements issued by the AU must include the "content template" as defined in section 10.0 xAPI State Data Model.

8.0 Content Launch Mechanisms

8.1 Launch Method

  • cmi5_8.1_1 Learning Management System (LMS): The AU MUST be launched by the LMS using either a new browser window or redirecting the existing browser window to a new location.

  • cmi5_8.1_2 Learning Management System (LMS): The AU MUST be launched by the LMS using the method specified in the course structure metadata. (new window, redirect existing window, or any window)

  • cmi5_8.1_3 Learning Management System (LMS): The AU MUST be launched by the LMS with a launch URL containing query string parameters (endpoint, fetch, actor, registration, and activityId).

  • cmi5_8.1_4 endpoint - Learning Management System (LMS): The AU MUST be launched by the LMS with a launch URL containing a "endpoint" name/value pair in the query string containing the endpoint to the LRS.

  • cmi5_8.1_5 endpoint - Assignable Unit (AU): The AU MUST get the endpoint value from the query string and use the endpoint value as the Base Endpoint for xAPI requests.

  • cmi5_8.1_6 fetch - Learning Management System (LMS): the LMS MUST place the fetch name/value pair in the query string.

  • cmi5_8.1_7 fetch - Assignable Unit (AU): The AU MUST retrieve the fetch value from the query string.

  • cmi5_8.1_8 actor - Learning Management System (LMS): The LMS MUST populate the actor with a valid value.

  • cmi5_8.1_9 actor - Assignable Unit (AU): The AU MUST get the actor value from the query string. The AU MUST use the actor value in API calls that require an "actor" property when sending xAPI requests.

  • cmi5_8.1_10 registration - Learning Management System (LMS): The LMS MUST place a UUID value for registration id in the query string.

  • cmi5_8.1_11 registration - Assignable Unit (AU): The AU MUST use the registration id provided query string in API calls that require a "registration id" when sending xAPI requests.

  • cmi5_8.1_12 activityId- Learning Management System (LMS): The LMS MUST generate an activityId for the AU and place its value in the query string.

  • cmi5_8.1_13 activityId- Learning Management System (LMS): The activityId generated MUST NOT match the AU's id (publisher id) from the course structure

  • cmi5_8.1_14 activityId- Learning Management System (LMS): The LMS MUST use the same generated activityId on all subsequent launches (for the same AU) within the same registration. The LMS SHOULD use the same generated activityId (for the same AU) for all registrations.

  • cmi5_8.1_13 activityId- Assignable Unit (AU): The AU MUST get the activityId value from the query string and use the activityId value in API calls that require an "activity id" when issuing cmi5 defined statements.

8.2 Authorization Token Fetch URL

8.2.1 Overview

  • cmi5_8.2.1_1 Learning Management System (LMS): The LMS MUST include a _ fetch _ name/value pair in the launch URL.

  • cmi5_8.2.1_2 Learning Management System (LMS): The _ fetch _ URL MUST accept POST requests and MUST NOT accept GET requests.

  • cmi5_8.2.1_3 Assignable Unit (AU): The AU MUST issue only one request to the _ fetch _ URL.

  • cmi5_8.2.1_4 Assignable Unit (AU): The AU MUST NOT issue any type of HTTP requests other than POST to the _ fetch _ URL.

  • cmi5_8.2.1_5 Learning Management System (LMS): The LMS must return an auth token in a the JSON structure specified from a valid fetch URL call.

8.2.2 Definition: auth-token

  • cmi5_8.2.2_1 Learning Management System (LMS): The LMS MUST return an HTTP status code of "200" for a valid (POST) request to the fetch URL.

  • cmi5_8.2.2_2 Assignable Unit (AU): The AU MUST place the authorization token returned from the fetch URL in the Authorization headers of all HTTP requests made to the LRS endpoint.

8.2.3 Errors

8.2.3.1 Duplicate call to fetch URL

8.2.3.2 Error Values

8.3 Other Launch Environments

9.0 xAPI Statement Data Model

9.1 Statement ID

  • cmi5_9.1_1 Assignable Unit (AU): The AU MUST assign a statement id property in UUID format for all statements it issues.

9.2 Actor

  • cmi5_9.2_1 Learning Management System (LMS): The LMS MUST provide an actor objectwith an objectType of "Agent" and MUST contain an "account" as defined in the xAPI specification.

  • cmi5_9.2_2 Learning Management System (LMS): The Actor object defined by the LMS must be used as the actor in all cmi5 defined statements made by the LMS.

  • cmi5_9.2_3 Assignable Unit (AU): The Actor object defined by the LMS must be used as the actor in all cmi5 defined statements made by the AU.

9.3 Verbs

  • cmi5_9.3_1 Assignable Unit (AU): The AU MUST NOT issue duplicate verbs (in cmi5 defined statements) within an AU session. (see cmi5 verbs…)

  • cmi5_9.3_2 Assignable Unit (AU): The AU MUST NOT use a Passed and a Failed verb (in cmi5 defined statements) within an AU session.

  • cmi5_9.3_4 Assignable Unit (AU): The AU MUST issue a Terminated (cmi5 defined) statement as its last statement in a session.

  • cmi5_9.3_6 Assignable Unit (AU): The AU MUST issue only one Passed (cmi5 defined) statement per ActivityId per registration.

  • cmi5_9.3_7 Assignable Unit (AU): The AU MUST NOT issue a Failed (cmi5 defined) statement following a Passed (cmi5 defined) statement per ActivityId per registration.

  • cmi5_9.3_8 Learning Management System (LMS): All statements from AUs must be recorded to the LRS.

  • cmi5_9.3_9 Learning Management System (LMS): LMS MUST NOT issue more than one abandoned (cmi5 defined) statement for a session id.

  • cmi5_9.3_10 Learning Management System (LMS): LMS MUST NOT issue more than one waived statement per session per AU and MUST not issue more than one waived statement per registration per AU.

9.3.1 Launched

  • cmi5_9.3.1_1 Learning Management System (LMS): The LMS MUST issue a Launched (cmi5 defined) statement before launching an AU.

  • cmi5_9.3.1_2 Learning Management System (LMS): The LMS MUST NOT issue multiple Launched (cmi5 defined) statements for given a session id.

9.3.2 Initialized

  • cmi5_9.3.2_1 Assignable Unit (AU): The AU MUST issue a Initialized (cmi5 defined) statement as its first statement in a session.

  • cmi5_9.3.2_2 Assignable Unit (AU): The AU MUST NOT issue more than one Initialized (cmi5 defined) statement in a session.

9.3.3 Completed

  • cmi5_9.3.3_1 Assignable Unit (AU): The AU MUST issue only one Completed (cmi5 defined) statement per ActivityId per registration.

  • cmi5_9.3.3_2 _Assignable Unit (__AU): _The AU MUST issue a (cmi5 defined) statement containing the "completed" verb when the learner has experienced all relevant material in an AU. (testing is subject to AU instructions – human inspection – not automatable)

9.3.4 Passed

  • cmi5_9.3.4_1 _Assignable Unit (__AU): _The AU MUST record a (cmi5 defined) statement containing the "passed" verb when the learner has attempted and passed the AU. (testing is subject to AU instruction – human inspection – not automatable)

  • cmi5_9.3.4_2 _Assignable Unit (__AU): _. If a "passed" (cmi5 defined) statement contains a (scaled) score and the LMS Launch Data contains a "masteryScore", then the (scaled) score MUST be equal to or greater than the masteryScore

9.3.5 Failed

  • cmi5_9.3.5_1 _Assignable Unit (__AU): _The AU MUST record a (cmi5 defined) statement containing the "failed" verb when the learner has attempted and failed the AU.. (testing is subject to AU instruction – human inspection – not automatable)

  • cmi5_9.3.5_2 _Assignable Unit (__AU): _. If a " failed" (cmi5 defined) statement contains a (scaled) score and the LMS Launch Data contains a "masteryScore", then the (scaled) score MUST be less than the masteryScore

9.3.6 Abandoned

  • cmi5_9.3.6_1 Learning Management System (LMS): The LMS MUST record an "abandoned" (cmi5 defined) statement on behalf of an given AU if the previous session (with the same learner/course registration) of that AU did not include a "terminated" statement(cmi5 defined).

  • cmi5_9.3.6_2 Learning Management System (LMS): After recording an "abandoned" (cmi5 defined) statement for a given session, the LMS MUST NOT allow any additional statements to be recorded for that session.

9.3.7 Waived

  • cmi5_9.3.7_1 Learning Management System (LMS): A (cmi5 defined) statement containing a "waived" verb MUST include a "reason" in the extension property of the Statement Result.

  • cmi5_9.3.7_2 Learning Management System (LMS): The LMS MUST generate a unique session id for a (cmi5 defined) statement containing a "waived" verb and MUST NOT issue any other statements (except for cmi5 defined statements with the "satisfied" verb) using that session id.

  • cmi5_9.3.7_3 Learning Management System (LMS): The LMS MUST NOT issue multiple (cmi5 defined) statements with "waived" for the same AU within a course registration.

9.3.8 Terminated

  • cmi5_9.3.8_1 Assignable Unit (AU): The AU MUST record a (cmi5 defined) statement containing the "terminated" verb as the last statement in a session.

  • cmi5_9.3.8_2 Learning Management System (LMS): Upon receiving a (cmi5 defined) "terminated" statement, the LMS MUST reject statements (of any kind) for the AU session received after its (LMS defined) specified period of time.

9.3.9 Satisfied

  • cmi5_9.3.9 _1 Learning Management System (LMS): The LMS MUST generate a unique block id for each block in the course structure. The generated Block id MUST NOT match the publisher's ID from the course structure.

  • cmi5_9.3.9 _2 Learning Management System (LMS): The LMS MUST issue a (cmi5 defined) "satisfied" statement when the learner has met the moveOn criteria of all AU's in a (course structure) block. The "satisfied" statement must include the block id generated by the LMS and use "https://w3id.org/xapi/cmi5/activitytype/block" as the value of the "type" property in the Object's Definition.

  • cmi5_9.3.9 _1 Learning Management System (LMS): The LMS MUST generate a unique course id for the course structure. The generated course id MUST NOT match the publisher's ID from the course structure.

  • cmi5_9.3.9 _3 Learning Management System (LMS): The LMS MUST issue a (cmi5 defined) "satisfied" statement when the learner has met the moveOn criteria of all AU's in a course. The "satisfied" statement must include the course id generated by the LMS and use "https://w3id.org/xapi/cmi5/activitytype/block" as the value of the "type" property in the Object's Definition.

  • cmi5_9.3.9 _4 Learning Management System (LMS): For all "satisfied" statements triggered as a result of an AU launch session, the LMS MUST use the session id from the AU launch in the statements.

9.4 Object

  • cmi5_9.4_1 Assignable Unit (AU): The AU must provide an object in all (cmi5 defined) statements.

  • cmi5_9.4_2 Assignable Unit (AU): For all (cmi5 defined) statements issued by the AU, the Object's "id" property must match the activityId defined in the launch URL.

  • cmi5_9.4._3 Learning Management System (LMS): The LMS must provide an object in all (cmi5 defined) statements.

  • cmi5_9.4._4 Learning Management System (LMS): For launched (cmi5 defined) statements issued by the LMS, the Object's "id" property must match the ActivityId provided by the LMS for the AU launched.

  • cmi5_9.4._5 Learning Management System (LMS): For abandoned (cmi5 defined) statements issued by the LMS, the Object's "id" property must match the ActivityId provided by the LMS for the AU abandoned.

  • cmi5_9.4._6 Learning Management System (LMS): For waived (cmi5 defined) statements issued by the LMS, the Object's "id" property for must match the ActivityId provided by the LMS for the AU waived.

  • cmi5_9.4._7 Learning Management System (LMS): For satisfied (cmi5 defined) statements for blocks issued by the LMS, the Object's "id" property must match the LMS generated block id.

  • cmi5_9.4._8 Learning Management System (LMS): For satisfied (cmi5 defined) statements for blocks issued by the LMS, the Object's "id" property must match the LMS generated course id.

9.5 Result

9.5.1 Score

  • cmi5_9.5.1_1 Assignable Unit (AU): The AU must not report a score for (cmi5 defined) statements other than passed or failed.

  • cmi5_9.5.1_2 Assignable Unit (AU): If the AU provides a "raw" value for score it MUST also provide the "min" and "max" values for score.

(Other score requirements are covered in passed, failed, and MasteryScore)

9.5.2 Success

  • cmi5_9.5.2_1 Assignable Unit (AU): The AU must set the "success" property of the result to true for passed (cmi5 defined) statements.

  • cmi5_9.5.2_2 Assignable Unit (AU): The AU must set the "success" property of the result to false for failed (cmi5 defined) statements.

  • cmi5_9.5.2_3 Assignable Unit (AU): The AU must not include the success property for (cmi5 defined) statements other passed and failed.

  • cmi5_9.5.2_4 Learning Management System (LMS): The LMS must set the "success" property of the result to true for waived (cmi5 defined) statements.

  • cmi5_9.5.2_5 Learning Management System (LMS): The LMS must not include the success property for (cmi5 defined) statements other waived.

9.5.3 Completion

  • cmi5_9.5.3_1 Assignable Unit (AU): The AU must set the "completion" property of the result to true for passed (cmi5 defined) statements.

  • cmi5_9.5.3_2 Assignable Unit (AU): The AU must not include the "completion" property for (cmi5 defined) statements other completed.

  • cmi5_9.5.3_3 Learning Management System (LMS): The LMS must set the "completion" property of the result to true for waived (cmi5 defined) statements.

  • cmi5_9.5.3_4 Learning Management System (LMS): The LMS must not include the " completion" property for (cmi5 defined) statements other waived.

9.5.4 Duration

9.5.4.1 AU statements that include duration

  • cmi5_9.5.4.1_1 Assignable Unit (AU): The AU MUST include the "duration" property in "terminated" statements.

  • cmi5_9.5.4.1_2 Assignable Unit (AU): The AU MUST include the "duration" property in "completed" statements.

  • cmi5_9.5.4.1_3 Assignable Unit (AU): The AU MUST include the "duration" property in "passed" statements.

  • cmi5_9.5.4.1_3 Assignable Unit (AU): The AU MUST include the "duration" property in "failed" statements.

9.5.4.2 LMS statements that include duration

  • cmi5_9.5.4.2_1 Learning Management System (LMS): The duration property MUST be included in "Abandoned" statements.

  • cmi5_9.5.4.2_2 Learning Management System (LMS): The duration property MUST be at least set to the total session time, calculated as the time between the "Launched" statement and the last statement (of any kind) issued by the AU.

9.5.5 Extensions

9.5.5.1 progress

  • cmi5_9.5.5.1_1 Assignable Unit (AU): The progress extensions, if provided, must be an integer 0 to 100 inclusive.

9.5.5.2 reason

  • cmi5_9.5.5.2_1 Learning Management System (LMS): The LMS MUST set the "reason" extension value in "waived" statements.

9.6 Context

9.6.1 Registration

  • cmi5_9.6.1_1 Learning Management System (LMS): At the time of registration, the LMS MUST evaluate moveOn criteria in the course structure and issue satisfied statements for AU with moveOn of "Not Applicable".

  • cmi5_9.6.1_2 Learning Management System (LMS): For all satisfied statements triggered outside of an AU launch session, the LMS MUST generate a unique session id and use it for those statements.

9.6.2 ContextActivities

9.6.2.1 cmi5 Category Activity

9.6.2.2 moveOn Category Activity

  • cmi5_9.6.2.2_1 Learning Management System (LMS): Waived (cmi5 defined) statements issued by the LMS MUST have an Activity object with an "id" of "https://w3id.org/xapi/cmi5/context/categories/moveon" in the "category" context activities list. Other statements issued by the LMS MUST NOT include this Activity.

  • cmi5_9.6.2.2_2 Assignable Unit (AU): Passed and Completed (cmi5 defined) statements issued by the AU MUST have an Activity object with an "id" of "https://w3id.org/xapi/cmi5/context/categories/moveon" in the "category" context activities list. Other statements issued by the AU MUST NOT include this Activity.

9.6.2.3 Publisher ID Grouping Activity

  • cmi5_9.6.2.3_1 Learning Management System (LMS): The LMS MUST include an Activity object in the "grouping" context activities list with an "id" property whose value matches the AU's id attribute from the course structure in the "contextTemplate" of the State API document LMS.launchdata. (Written to the State API prior to launching the AU)

9.6.3 Extensions

9.6.3.1 session ID

  • cmi5_9.6.3.1_1 Learning Management System (LMS): The LMS MUST generate a unique string identifier ("session id") for each AU launch session.

  • cmi5_9.6.3.1_2 Learning Management System (LMS): The LMS MUST record the session ID in context template of LMS.launchdata in the State API (per Section 10) prior to launching an AU.

9.6.3.2 masteryScore

  • cmi5_9.6.3.2_1 Learning Management System (LMS): LMS MUST add masteryScore as an extension of context in a "Launched" statement when masteryScore is provided in the LMS.launchdata.

  • cmi5_9.6.3.2_2 Assignable Unit (AU): An AU MUST include the "masteryScore" value when provided in the LMS.launchdata as a context extension for "passed"/"failed" Statements.

9.6.3.3 launchMode

  • cmi5_9.6.3.3_1 Learning Management System (LMS): LMS MUST add launchMode as an extension of context in a "Launched" statement when launchMode is provided in the LMS.launchdata.

9.6.3.4 launchURL

  • cmi5_9.6.3.4_1 Learning Management System (LMS): The LMS MUST put a fully qualified URL equivalent to the one that the LMS used to launch the AU without the name/value pairs included as defined in section 8.1 as a context extension of the "Launched" statement.

9.6.3.5 publisherId

NA

9.6.3.6 moveOn

  • cmi5_9.6.3.6_1 Learning Management System (LMS): LMS MUST add moveOn as an extension of context in a Launched statement when moveOn is provided in LMS.launchdata.

9.6.3.7 launchParameters

  • cmi5_9.6.3.7_1 Learning Management System (LMS): LMS MUST add launchParametersas an extension of context in a Launched statement when launchParameters is provided in LMS.launchdata.

9.7 Timestamp

  • cmi5_9.7_1 Learning Management System (LMS): All statements issued by the LMS MUST include a timestamp property in UTC time.

  • cmi5_9.7_2 Assignable Unit (AU): All statements issued by the AU MUST include a timestamp property in UTC time.

10.0 xAPI State Data Model

  • cmi5_10_1 Learning Management System (LMS):

11.0 xAPI Agent Profile Data Model

11.1 languagePreference

11.2 audioPreference

12.0 xAPI Activity Profile Data Model

13.0 Course Structure Data Requirements

13.1 Course Structure Data Model

13.1.1 Course Level Metadata

13.1.2 Block Metadata

13.1.3 Objective Metadata

13.1.4 AU Metadata

13.1.5 Vendor Specific Metadata (Extensions)

13.2 Course Structure XSD

14.0 Course Package

14.1 Course Packages in ZIP Format

14.2 Course Structure XML Without a ZIP File Package

cmi5 Working Group Wiki

Home

Meeting Minutes

Working Documents

Clone this wiki locally