Contest_API.mw

'''This is the development draft for the Contest API. See also [[Contest_API_2020|the version that will be used at WF 2020]].'''

== Introduction ==

This page describes an API for accessing information provided by a [[Contest_Control_System|Contest Control System]] or [[CDS|Contest Data Server]].
Such an API can be used by a multitude of clients:
* an external scoreboard
* a scoreboard resolver application
* contest analysis software, such as the [[ICAT]] toolset
* another "shadow" CCS, providing forwarding of submissions and all relevant information
* internally, to interface between the CCS server and judging instances

This API is meant to be useful, not only at the ICPC World Finals, but more generally in any ICPC-style contest setup. It is meant to incorporate and supersede the [[JSON_Scoreboard_2016|JSON Scoreboard]], the [[Draft_2014_REST_interface_for_source_code_fetching|REST interface for source code fetching]], and the [[Contest_Start_Interface|Contest start interface]]. This REST interface is specified in conjunction with a new [[#Event feed|NDJSON event feed]], which provides all changes to this interface as CRUD-style events and is meant to supersede the old XML [[Event Feed]].

== General design principles ==

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [https://www.ietf.org/rfc/rfc2119.txt RFC 2119].

The interface is implemented as a HTTP REST interface that outputs information in [https://en.wikipedia.org/wiki/JSON JSON] format ([https://tools.ietf.org/html/rfc7159 RFC]). This REST interface should be provided over HTTPS to guard against eavesdropping on sensitive contest data and authentication credentials (see roles below).

=== Endpoint URLs ===

The specific base URL of this API will be dependent on the server (e.g. main CCS or CDS) providing the service; in the specification we only indicate the relative paths of API endpoints with respect to a '''baseurl'''. In all the examples below the baseurl is <tt>https://example.com/api</tt>.

We follow standard REST practices so that a whole collection can be requested, e.g. at the URL path
  GET https://example.com/api/contests/wf14/teams
while an element with specific ID is requested as
  GET https://example.com/api/contests/wf14/teams/10

A collection is always returned as a JSON list of objects. Every object in the list represents a single element (and always includes the ID). When requesting a single element the exact same object is returned. E.g. the URL path
 GET baseurl/<collection>
returns
 [ { "id":<id1>, <element specific data for id1>},
   { "id":<id2>, <element specific data for id2>},
      ...
 ]
while the URL path
 GET baseurl/<collection>/<id1>
returns
 { "id":<id1>, <element specific data for id1>}

=== HTTP headers ===

A server should allow cross-origin requests by setting the <tt>Access-Control-Allow-Origin</tt> HTTP header:

 Access-Control-Allow-Origin: *

A server should specify how clients should cache file downloads by setting the <tt>Cache-Control</tt> or <tt>Expires</tt> HTTP headers:

 Cache-Control: public, max-age=3600, s-maxage=18000

 Expires: Wed, 18 Jul 2018 07:28:00 GMT

=== HTTP methods ===

The current version of this specification only requires support for the '''GET''' method, unless explicitly specified otherwise in an endpoint below (see [[#PATCH_start_time|PATCH start_time]]).
However, for future compatibility below are already listed other methods with their expected behavior, if implemented.
;GET
: Read data. This method is idempotent and does not modify any data. It can be used to request a whole collection or a specific element.
;POST
: Create a new element. This can only be called on a collection endpoint. No '''id''' attribute should be specified as it is up to the server to assign one, which is returned in the location header.
;PUT
: Replaces a specific element. This method is idempotent and can only be called on a specific element and replaces its contents with the data provided. The payload data must be complete, i.e. no partial updates are allowed. The '''id''' attribute cannot be changed: it does not need to be specified (other than in the URL) and if specified different from in the URL, a '''409 Conflict''' HTTP code should be returned.
;PATCH
: Updates/modifies a specific element. Similar to '''PUT''' but allows partial updates by providing only that data, for example:
: <code>PATCH https://example.com/api/contests/wf14/teams/10</code>
: with JSON contents
: <code>{"name":"Our cool new team name"}</code>
: No updates of the '''id''' attribute are allowed either.
;DELETE
: Delete a specific element. Idempotent, but may return a 404 status code when repeated. Any provided data is ignored. Example:
: <code>DELETE https://example.com/api/contests/wf14/teams/8</code>
: Note that deletes must keep [[#Referential integrity|referential integrity]] intact.

Standard [https://en.wikipedia.org/wiki/List_of_HTTP_status_codes HTTP status codes] are returned to indicate success or failure.

=== Roles ===

Access to this API is controlled via user roles. The API provider must require authentication to access each role except for optionally the public role.
The API provider must support [https://en.wikipedia.org/wiki/Basic_access_authentication HTTP basic authentication] ([https://tools.ietf.org/html/rfc7617 RFC]). This provides a standard and flexible method; besides HTTP basic auth, other forms of authentication can be offered as well.

Each provider must support at least the following roles, although additional roles may be supported for specific uses:
* public (default role: contest data that's available to everyone)
* admin (data or capability only available to contest administrators)

Role-based access may completely hide some objects from the user, may omit certain attributes, or may embargo or omit objects based on the current contest time.
By default, the public user has read-only access (no '''POST''', '''PUT''', '''PATCH''' or '''DELETE''' methods allowed) and does not have access to judgements and runs from submissions made after the contest freeze time.

=== Referential integrity ===

Some attributes in elements are references to IDs of other elements. When such an attribute has a non-<tt>null</tt> value, then the referenced element must exist. That is, the full set of data exposed by the API must at all times be referentially intact.
This implies for example that before creating a [[#Teams|team]] with an <tt>organization_id</tt>, the [[#Organizations|organization]] must already exist. In reverse, that organization can only be deleted after the team is deleted, or alternatively, the team's <tt>organization_id</tt> is set to <tt>null</tt>.

Furthermore, the ID attribute (see below) of elements are not allowed to change. However, note that a particular ID might be reused by first deleting an element and then creating a new element with the same ID.

=== JSON attribute types ===

Attribute types are specified as one of the [https://en.wikipedia.org/wiki/JSON#Data_types.2C_syntax_and_example standard JSON types], or one of the more specific types below.
Implementations must be consistent with respect to the optional parts of each type, e.g. if the optional .uuu is included in any absolute timestamp it must be included when outputting all absolute timestamps.

;Integers
: (type '''<tt>integer</tt>''' in the specification) are JSON numbers that are restricted to be integer. They should be represented in standard integer representation <tt>(-)?[0-9]+</tt>.
;Floating point numbers
: (type '''<tt>float</tt>''' in the specification) are arbitrary JSON numbers that are expected to take non-integer values. It is recommended to use a decimal representation.
;Fixed point numbers
: (type '''<tt>decimal</tt>''' in the specification) are JSON numbers that are expected to take non-integer values. They must be in decimal (non-scientific) representation and have at most 3 decimals. That is, they must be a integer multiple of <tt>0.001</tt>.
;Absolute timestamps
: (type '''<tt>TIME</tt>''' in the specification) are strings containing human-readable timestamps, given in [https://en.wikipedia.org/wiki/ISO_8601 ISO 8601] extended combined date/time format with timezone: <tt>yyyy-mm-ddThh:mm:ss(.uuu)?[+-]zz(:mm)?</tt> (or timezone <tt>Z</tt> for UTC).
;Relative times
: (type '''<tt>RELTIME</tt>''' in the specification) are strings containing human-readable time durations, given in a slight modification of the [https://en.wikipedia.org/wiki/ISO_8601 ISO 8601] extended time format: <tt>(-)?(h)*h:mm:ss(.uuu)?</tt>
;Identifiers
: (type '''<tt>ID</tt>''' in the specification) are given as string consisting of characters <tt>[a-zA-Z0-9_-]</tt> of length at most 36 and not starting with a <tt>-</tt> (dash). IDs are unique within each endpoint.
: IDs are assigned by the person or system that is the source of the object, and must be maintained by downstream systems. For example, the person configuring a contest on disk will typically define the ID for each team, and any CCS or CDS that exposes the team must use the same ID.
: Some IDs are also used as identifiable labels and are marked below along with the recommended format. These IDs should be meaningful for human communication (e.g. team "43", problem "A") and are as short as reasonable but not more than 10 characters. IDs not marked as labels may be random characters and cannot be assumed to be suitable for display purposes.
;Ordinals
: (type '''<tt>ORDINAL</tt>''' in the specification) are used to give an explicit order to a list of objects. Ordinal attributes are integers and must be non-negative and unique in a list of objects, and they should typically be low numbers starting from zero. However, clients must not assume that the ordinals start at zero nor that they are sequential. Instead the ordinal values should be used to sort the list of objects.
;File references
: (types '''<tt>IMAGE</tt>''', '''<tt>VIDEO</tt>''', '''<tt>ARCHIVE</tt>''' and '''<tt>STREAM</tt>''' in the specification) are represented as a JSON object with elements as defined below.

Element for file reference objects:
{| class="wikitable"
|-
! Name
! Type
! Nullable?
! Description
|-
| href
| string
| no
| URL where the resource can be found. Relative URLs are relative to the '''baseurl'''. Must point to a file of intended mime-type. Resource must be accessible using the exact same (possibly none) authentication as the call that returned this data.
|-
| mime
| string
| no
| Mime type of resource.
|-
| width
| integer
| no for '''<tt>IMAGE</tt>'''
| Width of the image, video or stream in pixels. Should not be used for '''<tt>ARCHIVE</tt>'''.
|-
| height
| integer
| no for '''<tt>IMAGE</tt>'''
| Height of the image, video or stream in pixels. Should not be used for '''<tt>ARCHIVE</tt>'''.
|}

The '''href''' attributes may be [https://tools.ietf.org/html/rfc3986 absolute or relative URLs]; relative URLs must be interpreted relative to the '''baseurl''' of the API. For example, if '''baseurl''' is <tt>https://example.com/api</tt>, then the following are equivalent JSON response snippets pointing to the same location:

  "href":"https://example.com/api/contests/wf14/submissions/187/files"
  "href":"contests/wf14/submissions/187/files"

If implementing support for uploading files pointed to by resource links, substitute the href element with a data element with a base64 encoded string of the associated file contents as the value.

For example
    POST https://example.com/api/contests/wf14/organizations
with JSON data
    { "id":"inst105",
      "name":"Carnegie Mellon University",
      ...
      "logo": [{"data": "&lt;base64 string&gt;", "width": 160, "height": 160}]
    }

=== Extensibility ===

This specification is meant to cover the basic data of contests, with the idea that server/client implementations can extend this with more data and/or roles.
In particular, this specification already lists some endpoints or specific attributes as optional.
The following guidelines are meant to ease extensibility.

* Clients should accept extra attributes in endpoints, that are not specified here.
* Servers should not expect clients to recognize more than the basic, required specification.
* In this specification and extensions, an attribute with value <tt>null</tt> may be left out by the server (i.e. not be present). A client must treat an attribute with value <tt>null</tt> equivalently as that attribute not being present.

== Interface specification ==

The following list of API endpoints should be supported. Note that
<code>state</code>, <code>scoreboard</code> and <code>event-feed</code> are singular nouns
and indeed contain only a single element.

All endpoints should support '''GET'''; specific details on other
methods are mentioned below.

==== Types of endpoints ====

The endpoints can be categorised into 3 groups as follows:

;Configuration: contests, judgement-types, languages, problems, groups, organizations, teams, team-members
;Live data: state, submissions, judgements, runs, clarifications, awards
;Aggregate data: scoreboard, event-feed

Configuration is normally set before contest start. Is not expected to, but could occasionally be updated during a contest. It does not have associated
timestamp/contest time attributes. Updates are notified via the event feed.

Live data is generated during the contest and new elements are expected. Data is immutable though, only inserts, no updates or deletes of elements. It does have associated
timestamp/contest time attributes. Inserts and deletes are notified via the event feed. '''Note''': judgements are the exception to immutability in a weak sense: they get updated once with the final verdict.

Aggregate data: Only '''GET''' makes sense. These are not included in the event feed, also note that these should not be considered proper REST endpoints, and that the <tt>event-feed</tt> endpoint is a streaming feed in NDJSON format.

==== Table column description ====

In the tables below, the columns are:
;Name: Attribute name; object sub-attributes are indicated as <tt>object.attribute</tt>.
;Type: Data type of the attribute; either a [https://en.wikipedia.org/wiki/JSON#Data_types.2C_syntax_and_example JSON type] or [[#Recurring_details|a type defined above]].
;Required?: Whether this is a required attribute that '''must''' be implemented to conform to this specification.
;Nullable?: Whether the attribute might be <tt>null</tt> (and thus implicitly can also not be present in that case).
;Source @WF: Specifies whether this attribute is implemented at the ICPC World Finals and by whom.
;Description: Description of the meaning of the attribute and any special considerations.

Note that attributes with <tt>null</tt> value may be left out by the server. Furthermore, optional attributes must still be consistently implemented (or not) *within* each contest. This implies the following for attributes that are:
;Required and not nullable: The attribute must always be present with a value.
;Required and nullable:     The attribute may be <tt>null</tt>, and only in that case it may be left out.
;Optional and not nullable: The attribute may not be implemented, but that implies that no element of the endpoint has the attribute set. If one element has this attribute present, then it must be not <tt>null</tt> and the same must be true for all same type elements within the contest.
;Optional and nullable:     The attribute may be <tt>null</tt> or not present. In the latter case that can either be because it was a left out <tt>null</tt> value or because it was not implemented.

=== Contests ===

Provides information on the current contest.

The following endpoint is associated with contest:
{| class="wikitable"
|-
! Endpoint
! Mime-type
! Required?
! Source @WF
! Description
|-
| /contests
| application/json
| yes
| CDS
| JSON array of all contests with elements as defined in the table below
|-
| /contests/<id>
| application/json
| yes
| CCS
| JSON object of a single contest with elements as defined in the table below
|}

Returns a JSON object with the elements below. If there is no current (this may include about to start or just finished) contest, a 404 error is returned.
{| class="wikitable"
|-
! Name
! Type
! Required?
! Nullable?
! Source @WF
! Description
|-
| id
| ID
| yes
| no
| CCS
| identifier of the current contest
|-
| name
| string
| yes
| no
| CCS
| short display name of the contest
|-
| formal_name
| string
| no
| no
| CCS
| full name of the contest
|-
| start_time
| TIME
| yes
| yes
| CCS
| the scheduled start time of the contest, may be <tt>null</tt> if the start time is unknown or the countdown is paused
|-
| countdown_pause_time
| RELTIME
| no
| yes
| CDS
| The amount of seconds left when countdown to contest start is paused. At no time may both <tt>start_time</tt> and <tt>countdown_pause_time</tt> be non-<tt>null</tt>.
|-
| duration
| RELTIME
| yes
| no
| CCS
| length of the contest
|-
| scoreboard_freeze_duration
| RELTIME
| no
| yes
| CCS
| how long the scoreboard is frozen before the end of the contest
|-
| penalty_time
| integer
| no
| no
| CCS
| penalty time for a wrong submission, in minutes
|-
| banner
| array of IMAGE
| no
| yes
| CDS
| banner for this contest, intended to be an image with a large aspect ratio around 8:1. Only allowed mime type is image/png.
|-
| logo
| array of IMAGE
| no
| yes
| CDS
| logo for this contest, intended to be an image with aspect ratio near 1:1. Only allowed mime type is image/png.
|}

The expected/typical use of <tt>countdown_pause_time</tt> is that once a <tt>start_time</tt> is defined and close, the countdown may be paused due to unforeseen delays. In this case, <tt>start_time</tt> should be set to <tt>null</tt> and <tt>countdown_pause_time</tt> to the number of seconds left to count down. The <tt>countdown_pause_time</tt> may change to indicate approximate delay. Countdown is resumed by setting a new <tt>start_time</tt> and resetting <tt>countdown_pause_time</tt> to <tt>null</tt>.

==== Access restrictions at WF ====

No access restrictions apply to a GET on this endpoint.

==== PATCH start_time ====

To replace the [[Contest Start Interface]], at the ICPC World Finals,
an API provided by a CCS or CDS implementing this specification must have a role
that has the ability to clear or set the contest start time via a PATCH method.

The PATCH must include a valid JSON element with only two or three attributes
allowed: the contest id (used for verification), a start_time (a
<code>&lt;TIME&gt;</code> value or <code>null</code>), and an optional countdown_pause_time (<code>&lt;RELTIME&gt;</code>). As above, countdown_pause_time can only be non-null when start time is null.

The request should fail with a 401 if the user does not have
sufficient access rights, or a 403 if the contest is started or within
30s of starting, or if the new start time is in the past or within 30s.

==== Example ====

Request:
  GET https://example.com/api/contests/wf2014
Returned data:
 {
 	"id": "wf2014",
 	"name": "2014 ICPC World Finals",
 	"formal_name": "38th Annual World Finals of the ACM International Collegiate Programming Contest",
 	"start_time": "2014-06-25T10:00:00+01",
 	"duration": "5:00:00",
 	"scoreboard_freeze_duration": "1:00:00",
 	"penalty_time": 20,
 	"banner": [{
 		"href": "https://example.com/api/contests/wf2014/banner",
 		"width": 1920,
 		"height": 240
 	}]
 }
Request:
  GET https://example.com/api/contests/dress2016
Returned data:
 {
 	"id": "dress2016",
 	"name": "2016 ICPC World Finals Dress Rehearsal",
 	"start_time": null,
 	"countdown_pause_time": "0:03:38.749",
 	"duration": "2:30:00"
 }
Request:
  PATCH https://example.com/api/contests/wf2014
Request data:
 {
 	"id": "wf2014",
 	"start_time": "2014-06-25T10:00:00+01"
 }
Request:
  PATCH https://example.com/api/contests/wf2016
Request data:
 {
 	"id": "wf2016",
 	"start_time": null
 }

=== Judgement Types ===

Judgement types are the possible responses from the system when judging a submission.

The following endpoints are associated with judgement types:
{| class="wikitable"
|-
! Endpoint
! Mime-type
! Required?
! Source @WF
! Description
|-
| /contests/<id>/judgement-types
| application/json
| yes
| CCS
| JSON array of all judgement types with elements as defined in the table below
|-
| /contests/<id>/judgement-types/<id>
| application/json
| yes
| CCS
| JSON object of a single judgement type with elements as defined in the table below
|}

JSON elements of judgement type objects:
{| class="wikitable"
|-
! Name
! Type
! Required?
! Nullable?
! Source @WF
! Description
|-
| id
| ID
| yes
| no
| CCS
| identifier of the judgement type, a 2-3 letter capitalized shorthand, see table below
|-
| name
| string
| yes
| no
| CCS
| name of the judgement. (might not match table below, e.g. if localised)
|-
| penalty
| boolean
| depends
| no
| CCS
| whether this judgement causes penalty time; must be present if and only if contest:penalty_time is present
|-
| solved
| boolean
| yes
| no
| CCS
| whether this judgement is considered correct
|}

==== Access restrictions at WF ====

No access restrictions apply to a GET on this endpoint.

==== Known judgement types ====

The list below contains standardized identifiers for known judgement types. These identifiers should be used by a server. Please send an email to [mailto:cliccs@ecs.csus.edu cliccs@ecs.csus.edu] when there are judgement types missing.

The column '''Big 5''' lists the "big 5" equivalents, if any. A '''*''' in the column means that the judgement is one of the "big 5".

The '''Translation''' column lists other judgements the judgement can safely be translated to, if a system does not support it.

{| class="wikitable"
! ID
! Name
! A.k.a.
! Big 5
! Translation
! Description
|-
| AC
| Accepted
| Correct, Yes (YES)
| *
| -
| Solves the problem
|-
| RE
| Rejected
| Incorrect, No (NO)
| WA?
| -
| Does not solve the problem
|-
| WA
| Wrong Answer
|
| *
| RE
| Output is not correct
|-
| TLE
| Time Limit Exceeded
|
| *
| RE
| Too slow
|-
| RTE
| Run-Time Error
|
| *
| RE
| Crashes
|-
| CE
| Compile Error
|
| *
| RE
| Does not compile
|-
| APE
| Accepted - Presentation Error
| Presentation Error, also see AC, PE, and IOF
| AC
| AC
| Solves the problem, although formatting is wrong
|-
| OLE
| Output Limit Exceeded
|
| WA
| WA, RE
| Output is larger than allowed
|-
| PE
| Presentation Error
| Output Format Error (OFE), Incorrect Output Format (IOF)
| WA
| WA, RE
| Data in output is correct, but formatting is wrong
|-
| EO
| Excessive Output
|
| WA
| WA, RE
| A correct output is produced, but also additional output
|-
| IO
| Incomplete Output
|
| WA
| WA, RE
| Parts, but not all, of a correct output is produced
|-
| NO
| No Output
|
| WA
| IO, WA, RE
| There is no output
|-
| WTL
| Wallclock Time Limit Exceeded
|
| TLE
| TLE, RE
| CPU time limit is not exceeded, but wallclock is
|-
| ILE
| Idleness Limit Exceeded
|
| TLE
| WTL, TLE, RE
| No CPU time used for too long
|-
| TCO
| Time Limit Exceeded - Correct Output
|
| TLE
| TLE, RE
| Too slow but producing correct output
|-
| TWA
| Time Limit Exceeded - Wrong Answer
|
| TLE
| TLE, RE
| Too slow and also incorrect output
|-
| TPE
| Time Limit Exceeded - Presentation Error
|
| TLE
| TWA, TLE, RE
| Too slow and also presentation error
|-
| TEO
| Time Limit Exceeded - Excessive Output
|
| TLE
| TWA, TLE, RE
| Too slow and also excessive output
|-
| TIO
| Time Limit Exceeded - Incomplete Output
|
| TLE
| TWA, TLE, RE
| Too slow and also incomplete output
|-
| TNO
| Time Limit Exceeded - No Output
|
| TLE
| TIO, TWA, TLE, RE
| Too slow and also no output
|-
| MLE
| Memory Limit Exceeded
|
| RTE
| RTE, RE
| Uses too much memory
|-
| SV
| Security Violation
| Illegal Function (IF), Restricted Function
| RTE
| RTE, RE
| Uses some functionality that is not allowed by the system
|-
| IF
| Illegal Function
| Illegal Function (IF), Restricted Function
| RTE
| SV, RTE, RE
| Calls a function that is not allowed by the system
|-
| RCO
| Run-Time Error - Correct Output
|
| RTE
| RTE, RE
| Crashing but producing correct output
|-
| RWA
| Run-Time Error - Wrong Answer
|
| RTE
| RTE, RE
| Crashing and also incorrect output
|-
| RPE
| Run-Time Error - Presentation Error
|
| RTE
| RWA, RTE, RE
| Crashing and also presentation error
|-
| REO
| Run-Time Error - Excessive Output
|
| RTE
| RWA, RTE, RE
| Crashing and also excessive output
|-
| RIO
| Run-Time Error - Incomplete Output
|
| RTE
| RWA, RTE, RE
| Crashing and also incomplete output
|-
| RNO
| Run-Time Error - No Output
|
| RTE
| RIO, RWA, RTE, RE
| Crashing and also no output
|-
| CTL
| Compile Time Limit Exceeded
|
| CE
| CE, RE
| Compilation took too long
|-
| JE
| Judging Error
|
| -
| -
| Something went wrong with the system
|-
| SE
| Submission Error
|
| -
| -
| Something went wrong with the submission
|-
| CS
| Contact Staff
| Other
| -
| -
| Something went wrong
|}

==== Examples ====

Request:
  GET https://example.com/api/contests/wf14/judgement-types
Returned data:
 [{
 	"id": "CE",
 	"name": "Compiler Error",
 	"penalty": false,
 	"solved": false
 }, {
 	"id": "AC",
 	"name": "Accepted",
 	"penalty": false,
 	"solved": true
 }]
Request:
  GET https://example.com/api/contests/wf14/judgement-types/AC
Returned data:
 {
 	"id": "AC",
 	"name": "Accepted",
 	"penalty": false,
 	"solved": true
 }

=== Languages ===

Languages that are available for submission at the contest.

The following endpoints are associated with languages:
{| class="wikitable"
|-
! Endpoint
! Mime-type
! Required?
! Source @WF
! Description
|-
| /contests/<id>/languages
| application/json
| yes
| CCS
| JSON array of all languages with elements as defined in the table below
|-
| /contests/<id>/languages/<id>
| application/json
| yes
| CCS
| JSON object of a single language with elements as defined in the table below
|}

JSON elements of language objects:
{| class="wikitable"
|-
! Name
! Type
! Required?
! Nullable?
! Source @WF
! Description
|-
| id
| ID
| yes
| no
| CCS
| identifier of the language from table below
|-
| name
| string
| yes
| no
| CCS
| name of the language (might not match table below, e.g. if localised)
|}

==== Access restrictions at WF ====

No access restrictions apply to a GET on this endpoint.

==== Known languages ====

Below is a list of standardized identifiers for known languages.
When providing one of these languages, the corresponding identifier should be used.
The language name may be adapted e.g. for localization or to indicate a particular version of the language.
In case multiple versions of a language are provided, those must have separate, unique identifiers. It is recommended to choose new identifiers with a suffix appended to an existing one. For example <tt>cpp17</tt> to specify the ISO 2017 version of C++.


{| class="wikitable"
! ID
! Name
|-
| ada
| Ada
|-
| c
| C
|-
| cpp
| C++
|-
| csharp
| C#
|-
| go
| Go
|-
| haskell
| Haskell
|-
| java
| Java
|-
| javascript
| JavaScript
|-
| kotlin
| Kotlin
|-
| objectivec
| Objective-C
|-
| pascal
| Pascal
|-
| php
| PHP
|-
| prolog
| Prolog
|-
| python2
| Python 2
|-
| python3
| Python 3
|-
| ruby
| Ruby
|-
| rust
| Rust
|-
| scala
| Scala
|}

==== Example ====

Request:
  GET https://example.com/api/contests/wf14/languages
Returned data:
 [{
 	"id": "java",
 	"name": "Java"
 }, {
 	"id": "cpp",
 	"name": "GNU C++"
 }, {
 	"id": "python2",
 	"name": "Python 2"
 }]

=== Problems ===

The problems to be solved in the contest

The following endpoints are associated with problems:
{| class="wikitable"
|-
! Endpoint
! Mime-type
! Required?
! Source @WF
! Description
|-
| /contests/<id>/problems
| application/json
| yes
| CCS
| JSON array of all problems with elements as defined in the table below
|-
| /contests/<id>/problems/<id>
| application/json
| yes
| CCS
| JSON object of a single problem with elements as defined in the table below
|}

JSON elements of problem objects:
{| class="wikitable"
|-
! Name
! Type
! Required?
! Nullable?
! Source @WF
! Description
|-
| id
| ID
| yes
| no
| CCS
| identifier of the problem, at the WFs the directory name of the problem archive
|-
| label
| string
| yes
| no
| CCS
| label of the problem on the scoreboard, typically a single capitalized letter
|-
| name
| string
| yes
| no
| CCS
| name of the problem
|-
| ordinal
| ORDINAL
| yes
| no
| CCS
| ordering of problems on the scoreboard
|-
| rgb
| string
| no
| no
| CCS
| hexadecimal RGB value of problem color as specified in [http://en.wikipedia.org/wiki/Web_colors#Hex_triplet HTML hexadecimal colors], e.g. '#AC00FF' or '#fff'
|-
| color
| string
| no
| no
| CCS
| human readable color description associated to the RGB value
|-
| time_limit
| decimal
| no
| no
| CCS
| time limit in seconds per test data set (i.e. per single run)
|-
| test_data_count
| integer
| yes
| no
| CCS
| number of test data sets
|}

==== Access restrictions at WF ====

The '''public''' role can only access these problems after the contest started.
That is, before contest start this endpoint returns an empty array for clients with the '''public''' role.

==== Examples ====

Request:
  GET https://example.com/api/contests/wf14/problems
Returned data:
  [{"id":"asteroids","label":"A","name":"Asteroid Rangers","ordinal":1,"color":"blue","rgb":"#00f","time_limit":2,"test_data_count":10},
   {"id":"bottles","label":"B","name":"Curvy Little Bottles","ordinal":2,"color":"gray","rgb":"#808080","time_limit":3.5,"test_data_count":15}
  ]

Request:
  GET https://example.com/api/contests/wf14/problems/asteroids
Returned data:
  {"id":"asteroids","label":"A","name":"Asteroid Rangers","ordinal":1,"color":"blue","rgb":"#00f","time_limit":2,"test_data_count":10}

=== Groups ===

Grouping of teams. At the World Finals these are the super regions, at regionals these are often different sites.

The following endpoints are associated with groups:
{| class="wikitable"
|-
! Endpoint
! Mime-type
! Required?
! Source @WF
! Description
|-
| /contests/<id>/groups
| application/json
| no
| CCS
| JSON array of all groups with elements as defined in the table below
|-
| /contests/<id>/groups/<id>
| application/json
| no
| CCS
| JSON object of a single group with elements as defined in the table below
|}

Note that these endpoints must be provided if groups are used. If they are not provided no other endpoint may refer to groups (i.e. return any group_ids).

JSON elements of group objects:
{| class="wikitable"
|-
! Name
! Type
! Required?
! Nullable?
! Source @WF
! Description
|-
| id
| ID
| yes
| no
| CCS
| identifier of the group
|-
| icpc_id
| string
| no
| yes
| CCS
| external identifier from ICPC CMS
|-
| name
| string
| yes
| no
| CCS
| name of the group
|-
| type
| string
| no
| yes
| CCS
| type of this group
|-
| hidden
| boolean
| no
| yes
| CCS
| if group should be hidden from scoreboard. Defaults to false if missing.
|}

==== Access restrictions at WF ====

No access restrictions apply to a GET on this endpoint.

==== Examples ====

Request:
  GET https://example.com/api/contests/wf14/groups
Returned data:
  [{"id":"asia-74324325532","icpc_id":"7593","name":"Asia"}
  ]

Request:
  GET https://example.com/api/contests/wf14/groups
Returned data:
  [{"id":"42425","name":"Division 2","type":"division"}
  ]

=== Organizations ===

Teams can be associated with organizations which will have some associated information, e.g. a logo. Typically organizations will be universities.

The following endpoints are associated with organizations:
{| class="wikitable"
|-
! Endpoint
! Type
! Required?
! Source @WF
! Description
|-
| /contests/<id>/organizations
| application/json
| no
| CCS & CDS
| JSON array of all organizations with elements as defined in the table below
|-
| /contests/<id>/organizations/<id>
| application/json
| no
| CCS & CDS
| JSON object of a single organization with elements as defined in the table below
|}

Note that the first two endpoints must be provided if organizations are used. If they are not provided no other endpoint may refer to organizations (i.e. return any organization_ids).

JSON elements of organization objects:
{| class="wikitable"
|-
! Name
! Type
! Required?
! Nullable?
! Source @WF
! Description
|-
| id
| ID
| yes
| no
| CCS
| identifier of the organization
|-
| icpc_id
| string
| no
| yes
| CCS
| external identifier from ICPC CMS
|-
| name
| string
| yes
| no
| CCS
| short display name of the organization
|-
| formal_name
| string
| no
| yes
| CCS
| full organization name if too long for normal display purposes.
|-
| country
| string
| no
| yes
| not used
| [https://en.wikipedia.org/wiki/ISO_3166-1_alpha-3 ISO 3-letter code] of the organization's country
|-
| url
| string
| no
| yes
| CDS
| URL to organization's website
|-
| twitter_hashtag
| string
| no
| yes
| CDS
| organization hashtag
|-
| location
| object
| no
| yes
| CDS
| JSON object as specified in the rows below
|-
| location.latitude
| float
| depends
| no
| CDS
| Latitude in degrees. Required iff location is present.
|-
| location.longitude
| float
| depends
| no
| CDS
| Longitude in degrees. Required iff location is present.
|-
| logo
| array of IMAGE
| no
| yes
| CDS
| logo of the organization. Only allowed mime type is image/png. A server must provide logos of size 56x56 and 160x160 but may provide other sizes as well.
|}

==== Access restrictions at WF ====

No access restrictions apply to a GET on organizations endpoints.

==== Example ====

Request:
  GET https://example.com/api/contests/<id>/organizations
Returned data:
  [{"id":"inst123","icpc_id":"433","name":"Shanghai Jiao Tong U.","formal_name":"Shanghai Jiao Tong University"},
   {"id":"inst105","name":"Carnegie Mellon University","country":"USA",
    "logo":[{"href":"http://example.com/api/contests/wf14/organizations/inst105/logo/56px","width":56,"height":56},
            {"href":"http://example.com/api/contests/wf14/organizations/inst105/logo/160px","width":160,"height":160}]
   }
  ]

=== Teams ===

Teams competing in the contest.

The following endpoints are associated with teams:
{| class="wikitable"
|-
! Endpoint
! Mime-type
! Required?
! Source @WF
! Description
|-
| /contests/<id>/teams
| application/json
| yes
| CCS & CDS
| JSON array of all teams with elements as defined in the table below
|-
| /contests/<id>/teams/<id>
| application/json
| yes
| CCS & CDS
| JSON object of a single team with elements as defined in the table below
|}

JSON elements of team objects:
{| class="wikitable"
|-
! Name
! Type
! Required?
! Nullable?
! Source @WF
! Description
|-
| id
| ID
| yes
| no
| CCS
| identifier of the team. Usable as a label, at WFs normally the team seat number
|-
| icpc_id
| string
| no
| yes
| CCS
| external identifier from ICPC CMS
|-
| name
| string
| yes
| no
| CCS
| name of the team
|-
| display_name
| string
| no
| yes
| CCS
| display name of the team. If not set, a client should revert to using the name instead.
|-
| organization_id
| ID
| no
| yes
| CCS
| identifier of the [[#Organizations | organization]] (e.g. university or other entity) that this team is affiliated to
|-
| group_ids
| array of ID
| no
| no
| CCS
| identifiers of the [[#Groups | group(s)]] this team is part of (at ICPC WFs these are the super-regions). No meaning must be implied or inferred from the order of IDs. The array may be empty.
|-
| location
| object
| no
| no
| CDS
| JSON object as specified in the rows below
|-
| location.x
| float
| depends
| no
| CDS
| Team's x position in meters. Required iff location is present.
|-
| location.y
| float
| depends
| no
| CDS
| Team's y position in meters. Required iff location is present.
|-
| location.rotation
| float
| depends
| no
| CDS
| Team's rotation in degrees. Required iff location is present.
|-
| photo
| array of IMAGE
| no
| yes
| CDS
| registration photo of the team. Only allowed mime types are image/jpeg and image/png.
|-
| video
| array of VIDEO
| no
| yes
| CDS
| registration video of the team.
|-
| backup
| array of ARCHIVE
| no
| yes
| CDS
| latest file backup of the team machine. Only allowed mime type is application/zip.
|-
| key_log
| array of FILE
| no
| yes
| CDS
| latest key log file from the team machine. Only allowed mime type is text/plain.
|-
| tool_data
| array of FILE
| no
| yes
| CDS
| latest tool data usage file from the team machine. Only allowed mime type is text/plain.
|-
| desktop
| array of STREAM
| no
| yes
| CDS
| streaming video of the team desktop.
|-
| webcam
| array of STREAM
| no
| yes
| CDS
| streaming video of the team webcam.
|-
| audio
| array of STREAM
| no
| yes
| CDS
| streaming team audio.
|}

==== Access restrictions at WF ====

The following access restrictions apply to a GET on this endpoint:
* '''backup''' requires the '''admin''' or '''analyst''' role for access,
* the '''desktop''' and '''webcam''' attributes are available for the '''public''' role only when scoreboard is not frozen.

==== Example ====

Request:
  GET https://example.com/api/contests/wf14/teams
Returned data:
  [{"id":"11","icpc_id":"201433","name":"Shanghai Tigers","organization_id":"inst123","group_ids":["asia-74324325532"]},
   {"id":"123","name":"CMU1","organization_id":"inst105","group_ids":["8","11"]}
  ]

=== Team members ===

Team members of teams in the contest.

The following endpoints are associated with languages:
{| class="wikitable"
|-
! Endpoint
! Mime-type
! Required?
! Source @WF
! Description
|-
| /contests/<id>/team-members
| application/json
| no
| CDS
| JSON array of all team members with elements as defined in the table below
|-
| /contests/<id>/team-members/<id>
| application/json
| no
| CDS
| JSON object of a single team member with elements as defined in the table below
|}

JSON elements of team member objects:
{| class="wikitable"
|-
! Name
! Type
! Required?
! Nullable?
! Source @WF
! Description
|-
| id
| ID
| yes
| no
| CDS
| identifier of the team-member
|-
| icpc_id
| string
| no
| yes
| CDS
| external identifier from ICPC CMS
|-
| team_id
| ID
| yes
| no
| CDS
| [[#Teams | team]] of this team member
|-
| first_name
| string
| yes
| no
| CDS
| first name of team member
|-
| last_name
| string
| yes
| no
| CDS
| last name of team member
|-
| sex
| string
| no
| yes
| CDS
| either '''male''' or '''female''', or possibly <tt>null</tt>
|-
| role
| string
| yes
| no
| CDS
| one of '''contestant''' or '''coach'''
|-
| photo
| array of IMAGE
| no
| yes
| CDS
| registration photo of the team member. Only allowed mime types are image/jpeg and image/png.

|}

==== Access restrictions at WF ====

No access restrictions apply to a GET on this endpoint.

==== Example ====

Request:
  GET https://example.com/api/contests/wf14/team-members
Returned data:
  [{"id":"john-smith","team_id":"43","icpc_id":"32442","first_name":"John","last_name":"Smith","sex":"male","role":"contestant"},
   {"id":"osten-umlautsen","team_id":"43","icpc_id":null,"first_name":"Östen","last_name":"Ümlautsen","sex":null,"role":"coach"}
  ]

=== Contest state ===

Current state of the contest, specifying whether it's running, the scoreboard is frozen or results are final.

The following endpoints are associated with state:

{| class="wikitable"
|-
! Endpoint
! Type
! Required?
! Source @WF
! Description
|-
| /contests/<id>/state
| application/json
| yes
| CCS
| JSON object of the current contest state with elements as defined in the table below
|}

JSON elements of state objects:
{| class="wikitable"
|-
! Name
! Type
! Required?
! Nullable?
! Source @WF
! Description
|-
| started
| TIME
| yes
| yes
| CCS
| Time when the contest actually started, or <tt>null</tt> if the contest has not started yet. When set, this time must be equal to the [[#Contests|contest]] <tt>start_time</tt>.
|-
| frozen
| TIME
| depends
| yes
| CCS
| Time when the scoreboard was frozen, or <tt>null</tt> if the scoreboard has not been frozen. Required iff <tt>scoreboard_freeze_duration</tt> is present in the [[#Contests|contest]] endpoint.
|-
| ended
| TIME
| yes
| yes
| CCS
| Time when the contest ended, or <tt>null</tt> if the contest has not ended. Must not be set if started is <tt>null</tt>.
|-
| thawed
| TIME
| depends
| yes
| CCS
| Time when the scoreboard was thawed (that is, unfrozen again), or <tt>null</tt> if the scoreboard has not been thawed. Required iff <tt>scoreboard_freeze_duration</tt> is present in the [[#Contests|contest]] endpoint. Must not be set if frozen is <tt>null</tt>.
|-
| finalized
| TIME
| yes
| yes
| CCS
| Time when the results were finalized, or <tt>null</tt> if results have not been finalized. Must not be set if ended is <tt>null</tt>.
|-
| end_of_updates
| TIME
| yes
| yes
| CCS
| Time after last update to the contest occurred, or <tt>null</tt> if more updates are still to come. Setting this to non-<tt>null</tt> must be the very last change in the contest.
|}

These state changes must occur in the order listed in the table above, as far as they do occur, except that <tt>thawed</tt> and <tt>finalized</tt> may occur in any order. For example, the contest may never be frozen and hence not thawed either, or, it may be finalized before it is thawed. That, is the following sequence of inequalities must hold:
  started < frozen < ended < thawed    < end_of_updates,
                     ended < finalized < end_of_updates.

A contest that has ended, has been thawed (or was never frozen) and is finalized must not change. Thus, <tt>end_of_updates</tt> can be set once both <tt>finalized</tt> is set and <tt>thawed</tt> is set if the contest was frozen.

==== Access restrictions at WF ====

No access restrictions apply to a GET on this endpoint, but note that when the <tt>frozen</tt> state is set, but <tt>thawed</tt> not yet, then this implies access restrictions for non-privileged users to other endpoints.

==== Example ====

Request:
  GET https://example.com/api/contests/wf14/state
Returned data:
  {
    "started": "2014-06-25T10:00:00+01",
    "ended": null,
    "frozen": "2014-06-25T14:00:00+01",
    "thawed": null,
    "finalized": null,
    "end_of_updates": null
  }

=== Submissions ===

Submissions, a.k.a. attempts to solve problems in the contest.

The following endpoints are associated with submissions:
{| class="wikitable"
|-
! Endpoint
! Type
! Required?
! Source @WF
! Description
|-
| /contests/<id>/submissions
| application/json
| yes
| CCS
| JSON array of all submissions with elements as defined in the table below
|-
| /contests/<id>/submissions/<id>
| application/json
| yes
| CCS
| JSON object of a single submission with elements as defined in the table below
|}

JSON elements of submission objects:
{| class="wikitable"
|-
! Name
! Type
! Required?
! Nullable?
! Source @WF
! Description
|-
| id
| ID
| yes
| no
| CCS
| identifier of the submission. Usable as a label, typically a low incrementing number
|-
| language_id
| ID
| yes
| no
| CCS
| identifier of the [[#Languages | language]] submitted for
|-
| problem_id
| ID
| yes
| no
| CCS
| identifier of the [[#Problems | problem]] submitted for
|-
| team_id
| ID
| yes
| no
| CCS
| identifier of the [[#Teams | team]] that made the submission
|-
| time
| TIME
| yes
| no
| CCS
| timestamp of when the submission was made
|-
| contest_time
| RELTIME
| yes
| no
| CCS
| contest relative time when the submission was made
|-
| entry_point
| string
| yes
| yes
| CCS
| code entry point for specific languages
|-
| files
| array of ARCHIVE
| yes
| no
| CCS
| submission files, contained at the root of the archive. Only allowed mime type is application/zip.
|-
| reaction
| array of VIDEO
| no
| yes
| CDS
| reaction video from team's webcam.
|}

The '''entry_point''' attribute must be included for submissions in languages which do not have a single, unambiguous entry point to run the code. In general the entry point is the string that needs to be specified to point to the code to be executed. Specifically, for Python it is the file name that should be run, and for Java and Kotlin it is the fully qualified class name (that is, with any package name included, e.g. <tt>com.example.myclass</tt> for a class in the package <tt>com.example</tt> in Java). For C and C++ no entry point is required and it must therefore be <tt>null</tt>.

The '''files''' attribute provides the file(s) of a given submission as a zip archive. These must be stored directly from the root of the zip file, i.e. there must not be extra directories (or files) added unless these are explicitly part of the submission content.
For '''POST''', '''PUT''' and '''PATCH''' methods, the '''files''' attribute must contain the base64-encoded string of the zip archive.

==== Access restrictions at WF ====

The '''entry_point''' and '''files''' attribute are accessible only for clients with '''admin''' or '''analyst''' role.
The '''reaction''' attribute is available to clients with '''public''' role only when the contest is not frozen.

==== Example ====

Request:
  GET https://example.com/api/contests/wf14/submissions
Returned data:
  [{"id":"187","team_id":"123","problem_id":"10-asteroids",
    "language_id":"1-java","time":"2014-06-25T11:22:05.034+01","contest_time":"1:22:05.034","entry_point":"Main",
    "files":[{"href":"contests/wf14/submissions/187/files","mime":"application/zip"}]}
  ]

Note that the relative link for <tt>files</tt> points to the location <tt>https://example.com/api/contests/wf14/submissions/187/files</tt> since the base URL for the API is <tt>https://example.com/api</tt>.

=== Judgements ===

Judgements for submissions in the contest.

The following endpoints are associated with judgements:
{| class="wikitable"
|-
! Endpoint
! Mime-type
! Required?
! Source @WF
! Description
|-
| /contests/<id>/judgements
| application/json
| yes
| CCS
| JSON array of all judgements with elements as defined in the table below
|-
| /contests/<id>/judgements/<id>
| application/json
| yes
| CCS
| JSON object of a single judgement with elements as defined in the table below
|}

JSON elements of judgement objects:
{| class="wikitable"
|-
! Name
! Type
! Required?
! Nullable?
! Source @WF
! Description
|-
| id
| ID
| yes
| no
| CCS
| identifier of the judgement
|-
| submission_id
| ID
| yes
| no
| CCS
| identifier of the [[#Submissions | submission]] judged
|-
| judgement_type_id
| ID
| yes
| yes
| CCS
| the [[#Judgement Types | verdict]] of this judgement
|-
| start_time
| TIME
| yes
| no
| CCS
| absolute time when judgement started
|-
| start_contest_time
| RELTIME
| yes
| no
| CCS
| contest relative time when judgement started
|-
| end_time
| TIME
| yes
| yes
| CCS
| absolute time when judgement completed
|-
| end_contest_time
| RELTIME
| yes
| yes
| CCS
| contest relative time when judgement completed
|-
| max_run_time
| decimal
| no
| yes
| CCS
| maximum run time in seconds for any test case
|}

When a judgement is started, each of <tt>judgement_type_id</tt>, <tt>end_time</tt> and <tt>end_contest_time</tt> will be <tt>null</tt> (or missing).
These are set when the judgement is completed.

==== Access restrictions at WF ====

For clients with the '''public''' role, judgements will not be included for submissions received while the scoreboard is frozen. This means that all judgements for submissions received before the scoreboard has been frozen will be sent immediately, and all judgements for submissions received after the scoreboard has been frozen will be sent immediately after the scoreboard has been thawed.

==== Example ====

Request:
  GET https://example.com/api/contests/wf14/judgements
Returned data:
  [{"id":"189549","submission_id":"wf2017-32163123xz3132yy","judgement_type_id":"CE","start_time":"2014-06-25T11:22:48.427+01",
    "start_contest_time":"1:22:48.427","end_time":"2014-06-25T11:23:32.481+01","end_contest_time":"1:23:32.481"},
   {"id":"189550","submission_id":"wf2017-32163123xz3133ub","judgement_type_id":null,"start_time":"2014-06-25T11:24:03.921+01",
    "start_contest_time":"1:24:03.921","end_time":null,"end_contest_time":null}
  ]

=== Runs ===

Runs are judgements of individual test cases of a submission.

The following endpoints are associated with runs:
{| class="wikitable"
|-
! Endpoint
! Mime-type
! Required?
! Source @WF
! Description
|-
| /contests/<id>/runs
| application/json
| yes
| CCS
| JSON array of all runs with elements as defined in the table below
|-
| /contests/<id>/runs/<id>
| application/json
| yes
| CCS
| JSON object of a single run with elements as defined in the table below
|}

JSON elements of run objects:
{| class="wikitable"
|-
! Name
! Type
! Required?
! Nullable?
! Source @WF
! Description
|-
| id
| ID
| yes
| no
| CCS
| identifier of the run
|-
| judgement_id
| ID
| yes
| no
| CCS
| identifier of the [[#Judgements | judgement]] this is part of
|-
| ordinal
| ORDINAL
| yes
| no
| CCS
| ordering of runs in the judgement. Must be different for every run in a judgement. Runs for the same test case must have the same ordinal.  Must be between 1 and <tt>problem:test_data_count</tt>.
|-
| judgement_type_id
| ID
| yes
| no
| CCS
| the [[#Judgement Types | verdict]] of this judgement (i.e. a judgement type)
|-
| time
| TIME
| yes
| no
| CCS
| absolute time when run completed
|-
| contest_time
| RELTIME
| yes
| no
| CCS
| contest relative time when run completed
|-
| run_time
| decimal
| no
| no
| CCS
| run time in seconds
|}

==== Access restrictions at WF ====

For clients with the '''public''' role, runs will not be included for submissions received while the scoreboard is frozen. This means that all runs for submissions received before the scoreboard has been frozen will be sent immediately, and all runs for submissions received after the scoreboard has been frozen will be sent immediately after the scoreboard has been thawed.

==== Example ====

Request:
  GET https://example.com/api/contests/wf14/runs
Returned data:
  [{"id":"1312","judgement_id":"189549","ordinal":28,"judgement_type_id":"TLE",
    "time":"2014-06-25T11:22:42.420+01","contest_time":"1:22:42.420"}
  ]

=== Clarifications ===

Clarification message sent between teams and judges, a.k.a. clarification requests (questions from teams) and clarifications (answers from judges).

The following endpoints are associated with clarification messages:
{| class="wikitable"
|-
! Endpoint
! Mime-type
! Required?
! Source @WF
! Description
|-
| /contests/<id>/clarifications
| application/json
| yes
| CCS
| JSON array of all clarification messages with elements as defined in the table below
|-
| /contests/<id>/clarifications/<id>
| application/json
| yes
| CCS
| JSON object of a single clarification message with elements as defined in the table below
|}

JSON elements of clarification message objects:
{| class="wikitable"
|-
! Name
! Type
! Required?
! Nullable?
! Source @WF
! Description
|-
| id
| ID
| yes
| no
| CCS
| identifier of the clarification
|-
| from_team_id
| ID
| yes
| yes
| CCS
| identifier of [[#Teams | team]] sending this clarification request, <tt>null</tt> if a clarification sent by jury
|-
| to_team_id
| ID
| yes
| yes
| CCS
| identifier of the [[#Teams | team]] receiving this reply, <tt>null</tt> if a reply to all teams or a request sent by a team
|-
| reply_to_id
| ID
| yes
| yes
| CCS
| identifier of clarification this is in response to, otherwise <tt>null</tt>
|-
| problem_id
| ID
| yes
| yes
| CCS
| identifier of associated [[#Problems | problem]], <tt>null</tt> if not associated to a problem
|-
| text
| string
| yes
| no
| CCS
| question or reply text
|-
| time
| TIME
| yes
| no
| CCS
| time of the question/reply
|-
| contest_time
| RELTIME
| yes
| no
| CCS
| contest time of the question/reply
|}

Note that at least one of <tt>from_team_id</tt> and <tt>to_team_id</tt> has to be <tt>null</tt>. That is, teams cannot send messages to other teams.

==== Access restrictions at WF ====

Clients with the '''public''' role can only view clarifications replies from the jury to all teams, that is, messages where both <tt>from_team_id</tt> and <tt>to_team_id</tt> are <tt>null</tt>.

==== Examples ====

Request:
  GET https://example.com/api/contests/wf14/clarifications
Returned data:
  [{"id":"wf2017-1","from_team_id":null,"to_team_id":null,"reply_to_id":null,"problem_id":null,
    "text":"Do not touch anything before the contest starts!","time":"2014-06-25T11:59:27.543+01","contest_time":"-0:15:32.457"}
  ]
Request:
  GET https://example.com/api/contests/wf14/clarifications
Returned data:
  [{"id":"1","from_team_id":"34","to_team_id":null,"reply_to_id":null,"problem_id":null,
    "text":"May I ask a question?","time":"2017-06-25T11:59:27.543+01","contest_time":"1:59:27.543"},
   {"id":"2","from_team_id":null,"to_team_id":"34","reply_to_id":"1","problem_id":null,
    "text":"Yes you may!","time":"2017-06-25T11:59:47.543+01","contest_time":"1:59:47.543"}
  ]
Request:
  GET https://example.com/api/contests/wf14/clarifications
Returned data:
  [{"id":"1","from_team_id":"34","text":"May I ask a question?","time":"2017-06-25T11:59:27.543+01","contest_time":"1:59:27.543"},
   {"id":"2","to_team_id":"34","reply_to_id":"1","text":"Yes you may!","time":"2017-06-25T11:59:47.543+01","contest_time":"1:59:47.543"}
  ]

=== Awards ===

Awards such as medals, first to solve, etc.

The following endpoints are associated with awards:
{| class="wikitable"
|-
! Endpoint
! Mime-type
! Required?
! Source @WF
! Description
|-
| /contests/<id>/awards
| application/json
| no
| CCS
| JSON array of all awards with elements as defined in the table below
|-
| /contests/<id>/awards/<id>
| application/json
| no
| CCS
| JSON object of a single award with elements as defined in the table below
|}

JSON elements of award objects:
{| class="wikitable"
|-
! Name
! Type
! Required?
! Nullable?
! Source @WF
! Description
|-
| id
| ID
| yes
| no
| CCS
| identifier of the award.
|-
| citation
| string
| yes
| no
| CCS
| award citation, e.g. "Gold medal winner"
|-
| team_ids
| array of ID
| yes
| no
| CCS
| JSON array of [[#Teams | team]] ids receiving this award. No meaning must be implied or inferred from the order of IDs. The array may be empty.
|}

==== Access restrictions at WF ====

For clients with the '''public''' role, awards will not include information from judgements of submissions received after the scoreboard freeze until it has been unfrozen.

==== Semantics ====

* Awards are not final until the contest is.
* An award does not have to be present during the contest. However, if it is present, then it must be kept up to date during the contest. E.g. if "winner" will not be updated with the current leader during the contest, it must not be '''create'''d until the award '''is''' awarded.
* If an award is present during the contest this means that if the contest would end immediately and then become final, that award would be final. E.g. the "winner" during the contest should be the current leader. This is of course subject to what data the client can see; the public role's winner may not change during the scoreboard freeze but an admin could see the true current winner.

For some common award cases the following IDs should be used.
{| class="wikitable"
! ID
! Meaning during contest
! Meaning when contest is final
! Comment
|-
| winner
| Current leader(s). Empty if no team has scored.
| Winner(s) of the contest
|
|-
| gold-medal
| Teams currently placed to receive a gold medal. Empty if no team has scored.
| Teams being awarded gold medals
|
|-
| silver-medal
| Teams currently placed to receive a silver medal. Empty if no team has scored.
| Teams being awarded silver medals
|
|-
| bronze-medal
| Teams currently placed to receive a bronze medal, assuming no extra bronze are awarded. Empty if no team has scored.
| Teams being awarded bronze medals
|
|-
| first-to-solve-<id>
| The team(s), if any, that was first to solve problem <id>. This implies that no unjudged submission made earlier remains.
| Same.
| Must never change once set, except if there are rejudgements.
|-
| group-winner-<id>
| Current leader(s) in group <id>. Empty if no team has scored.
| Winner(s) of group <id>
|
|-
| organization-winner-<id>
| Current leader(s) of organization <id>. Empty if no team has scored.
| Winner(s) of organization <id>
| Not useful in contest with only one team per organization (e.g. the WF)
|}

==== Example ====

Request:
  GET https://example.com/api/contests/wf14/awards
Returned data:
  [{"id":"gold-medal","citation":"Gold medal winner","team_ids":["54","23","1","45"]},
   {"id":"first-to-solve-a","citation":"First to solve problem A","team_ids":["45"]},
   {"id":"first-to-solve-b","citation":"First to solve problem B","team_ids":[]}
  ]

=== Scoreboard ===

Scoreboard of the contest.

Since this is generated data, only the '''GET''' method is allowed here, irrespective of role.

The following endpoint is associated with the scoreboard:
{| class="wikitable"
|-
! Endpoint
! Mime-type
! Required?
! Source @WF
! Description
|-
| /contests/<id>/scoreboard
| application/json
| yes
| CCS
| JSON object with scoreboard data as defined in the table below
|}

==== Scoreboard request options ====

The following options can be passed to the scoreboard endpoint.

===== Scoreboard at the time of a given event =====

By passing an [[#Event feed | event]] ID with the "after_event_id" URL argument, the scoreboard can be requested as it was directly after the specified event:

 /scoreboard?after_event_id=xy1234

This makes it possible for a client to get the scoreboard information that is guaranteed to match a certain contest event.
In case no "after_event_id" argument is provided, the current scoreboard will be returned. The request will fail with a 400 error if the id is invalid.

A suggested efficient server-side implementation to provide this, is to store with each event that changes the scoreboard, the new team scoreboard row.

==== Scoreboard format ====

JSON elements of the scoreboard object.
{| class="wikitable"
|-
! Name
! Type
! Required?
! Nullable?
! Source @WF
! Description
|-
| event_id
| ID
| yes
| no
| CCS
| Identifier of the [[#Event feed | event]] after which this scoreboard was generated. This must be identical to the argument <tt>after_event_id</tt>, if specified.
|-
| time
| TIME
| yes
| no
| CCS
| Time contained in the associated event. Implementation defined if the event has no associated time.
|-
| contest_time
| RELTIME
| yes
| no
| CCS
| Contest time contained in the associated event. Implementation defined if the event has no associated contest time.
|-
| state
| object
| yes
| no
| CCS
| Identical data as returned by the [[#Contest state | contest state]] endpoint. This is provided here for ease of use and to guarantee the data is synchronized.
|-
| rows
| JSON array of JSON objects
| yes
| no
| CCS
| A list of rows of team with their associated scores.
|}

The scoreboard <tt>rows</tt> array is sorted according to rank and alphabetical on team name within identically ranked teams.
Here alphabetical ordering means according to the [http://www.unicode.org/reports/tr10/ Unicode Collation Algorithm], by default using the <tt>en-US</tt> locale.

Each JSON object in the rows array consists of:
{| class="wikitable"
|-
! Name
! Type
! Required?
! Nullable?
! Source @WF
! Description
|-
| rank
| integer
| yes
| no
| CCS
| rank of this team, 1-based and duplicate in case of ties
|-
| team_id
| ID
| yes
| no
| CCS
| identifier of the [[#Teams | team]]
|-
| score
| object
| yes
| no
| CCS
| JSON object as specified in the rows below (for possible extension to other scoring methods)
|-
| score.num_solved
| integer
| yes
| no
| CCS
| number of problems solved by the team
|-
| score.total_time
| integer
| yes
| no
| CCS
| total penalty time accrued by the team
|-
| problems
| array of objects
| yes
| no
| CCS
| JSON array of problems with scoring data, see below for the specification of each element
|}

Each problem object within the scoreboard consists of:
{| class="wikitable"
|-
! Name
! Type
! Required?
! Nullable?
! Source @WF
! Description
|-
| problem_id
| ID
| yes
| no
| CCS
| identifier of the [[#Problems | problem]]
|-
| num_judged
| integer
| yes
| no
| CCS
| number of judged submissions (up to and including the first correct one)
|-
| num_pending
| integer
| yes
| no
| CCS
| number of pending submissions (either queued or due to freeze)
|-
| solved
| boolean
| yes
| no
| CCS
| whether the team solved this problem
|-
| time
| integer
| depends
| no
| CCS
| minutes into the contest when this problem was solved by the team. Required iff <tt>solved=true</tt>
|}

==== Access restrictions at WF ====

For clients with the '''public''' role, the scoreboard will not include information from judgements of submissions received after the scoreboard has been frozen until it has been thawed.

==== Example ====

Request:
  GET https://example.com/api/contests/wf14/scoreboard
Returned data:
  {
    "event_id": "xy1234",
    "time": "2014-06-25T14:13:07.832+01",
    "contest_time": "4:13:07.832",
    "state": {
      "started": "2014-06-25T10:00:00+01",
      "ended": null,
      "frozen": "2014-06-25T14:00:00+01",
      "thawed": null,
      "finalized": null,
      "end_of_updates": null
    },
    "rows": [
      {"rank":1,"team_id":"123","score":{"num_solved":3,"total_time":340},"problems":[
        {"problem_id":"1","num_judged":3,"num_pending":1,"solved":false},
        {"problem_id":"2","num_judged":1,"num_pending":0,"solved":true,"time":20},
        {"problem_id":"3","num_judged":2,"num_pending":0,"solved":true,"time":55},
        {"problem_id":"4","num_judged":0,"num_pending":0,"solved":false},
        {"problem_id":"5","num_judged":3,"num_pending":0,"solved":true,"time":205}
      ]}
    ]
  }

=== Event feed - DRAFT ===

Provides the event (notification) feed for the current contest. This is effectively a changelog of create, update, or delete events that have occurred in the REST endpoints. Some endpoints (specifically the [[#Scoreboard | Scoreboard]] and the Event feed itself) are aggregated data, and so these will only ever update due to some other REST endpoint updating. For this reason there is no explicit event for these, since there will always be another event sent. This can also be seen by the fact that there is no scoreboard event in the table of events below.

Every notification provides the current state of a single contest object. There is no guarantee on order of events (except for general requirements below), whether two consecutive changes cause one or two events, duplicate events, or even that different clients will receive the same order or set of events. The only guarantee is that when an object changes one or more times you'll receive an event, and the latest event received for any object is the correct and current state of that object (e.g. if an object was created and deleted you'll always receive a delete event last).

As a concrete example, judgement events are usually fired when judging is started, and fired again when the final judgment is available. If a client connects after the judgement, or a client was disconnected during the judgement, they will typically only receive the final (complete) judgement.

There are two mechanisms that clients can use to receive events: a webhook, or a streaming HTTP feed. Both mechanisms have the same format (payload) and events, but different benefits, drawbacks, and ways to register. Webhooks are better for internet-scale, asynchronous processing, and disconnected systems; the HTTP feed is better for browser-based applications and onsite contests.

==== General requirements ====

The event responses and '''data''' objects contained in it must observe the same restrictions as those of the respective endpoints they represent.
This means that attributes inside the '''data''' element will be present if and only if the client has access to those at the respective endpoint. The client only receives create, update and delete events of elements it has (partial) access to. When time-based access is granted or revoked, create or delete events are dispatched for each affected entity.

Referential integrity must be strictly adhered to for new objects. i.e. if there is a new object that refers to another object (e.g. a submission for a team) then the referenced object must already exist and have been notified. There is no such guarantee for deletion: if an object that is referred to is deleted then by definition all of the child objects will be deleted, but the events may not arrive in strict referential order:

* If an object, A, refers to another object, B, then the event that shows that A has been created or updated to refer to B '''must''' come after the event that shows that B has been created.
* If some data is only available after a specific state change, then the event showing the state change '''must''' come before any update events making that data available. E.g. problems are only available after contest start for the public role, so the state event showing that the contest has started '''must''' come before the problem events creating the problems.
* Since nothing must change after the contest has ended, thawed (or never been frozen), and been finalized, no event may come after the state event showing that.

==== Feed format ====

The feed is served as JSON objects, with every event corresponding to a change in a single object (submission, judgement, language, team, etc.) or full endpoint. The general format for events is:
  {"contest_id": "<id>", "endpoint": "<endpoint>", "id": "<id>", "data": <JSON data for element> }

{| class="wikitable"
|-
! Name
! Type
! Required?
! Nullable?
! Description
|-
| contest_id
| string
| yes
| no
| The contest id.
|-
| endpoint
| string
| yes
| yes
| The API endpoint, i.e. type of contest object above. Can be used for filtering
|-
| id
| string
| yes
| yes
| The id of the object that changed
|-
| data
| object
| yes
| yes
| The data is the object that would be returned if calling the corresponding API endpoint at this time, i.e. an object or null for deletions
|}

===== Filtering =====

TODO - filter by contest id and/or endpoint

===== Examples =====

The following are examples of contest events:

    {"contest_id":"finals","endpoint":"problems","id":null,"data":[
       {"id":"asteroids","label":"A","name":"Asteroid Rangers","ordinal":1,"color":"blue","rgb":"#00f","time_limit":2,"test_data_count":10},
       {"id":"bottles","label":"B","name":"Curvy Little Bottles","ordinal":2,"color":"gray","rgb":"#808080","time_limit":3.5,"test_data_count":15}]}

    {"contest_id":"finals","endpoint":"state","id":null,"data":{
       "started": "2014-06-25T10:00:00+01",
       "ended": null,
       "frozen": "2014-06-25T14:00:00+01",
       "thawed": null,
       "finalized": null,
       "end_of_updates": null}}

    {"contest_id":"finals","endpoint":"teams","id":"11","data":{"id":"11","icpc_id":"201433","name":"Shanghai Tigers","organization_id":"inst123","group_id":"asia"}}

    {"contest_id":"finals","endpoint":"teams","id":"11","data":{"id":"11","icpc_id":"201433","name":"The Shanghai Tigers","organization_id":"inst123","group_id":"asia"}}

    {"contest_id":"finals","endpoint":"teams","id":"11","data":null}

TODO: data is object or array - is that too ugly?


==== Webhook ====

A webhook allows you to receive HTTP callbacks whenever there is a change to the contest. Clients are only notified of future changes; they are expected to use other mechanisms if they need to determine the current state of the contest. Every callback will contain one JSON object as specified above.

Responding to each event with a 2xx response code indicates successful receipt and ensures that the events in the payload are never sent again. If the client responds with anything other than 2xx, the server will continue to periodically try again, potentially with different payloads (e.g. as new events accumulate). Callbacks to each client are always sent synchronously and in order; clients do not need to worry about getting callbacks out of order and should always process each callback fully before processing the next one.

If the client fails to respond to multiple requests over a period of time (configured for each contest), it will be assumed deactivated and automatically removed from future callbacks.

The following endpoint is associated with the webhook:
{| class="wikitable"
|-
! Endpoint
! Mime-type
! Required?
! Source @WF
! Description
|-
| /webhook
| application/json
| yes
| CCS
| List or register for webhook callbacks
|}

JSON elements of webhook objects:

{| class="wikitable"
|-
! Name
! Type
! Required?
! Nullable?
! Description
|-
| url
| string
| yes
| no
| The url for callbacks
|}

TODO: include filter details?


===== Adding a webhook =====

To register a webhook, you need to post your server's callback url. The general format to register a webhook is:
  {"url": "<callback url>", "auth": ... }

{| class="wikitable"
|-
! Name
! Type
! Required?
! Nullable?
! Description
|-
| url
| string
| yes
| no
| The url for callbacks
|-
| auth
| string
| yes
| no
| TODO
|}

===== Example =====

Request:
  POST https://example.com/api/webhook
Payload:
  {"url": "https://myurl", "auth": ... }

Request:
  GET https://example.com/api/webhook
Returned data:
  [{"url":"https://myurl"},{"url":"https://myotherurl"}]

Future payload posted to url:
  POST https://myurl
Payload:
  {"contest_id":"finals","endpoint":"teams","id":"11","data":{"id":"11","icpc_id":"201433","name":"The Shanghai Tigers","organization_id":"inst123","group_id":"asia"}}


==== HTTP Feed ====

The HTTP event feed is a streaming HTTP endpoint that allows connected clients to receive contest events. The feed is a complete log of contest objects that starts 'at the beginning of time' so all existing objects will be sent upon initial connection, but apart from referential integrity requirements they may appear in any order (e.g. teams or problems first).

Each line is an NDJSON formatted contest event as specified above. The feed does not terminate under normal circumstances, so to ensure keep alive a newline must be sent if there has been no event within 120 seconds.

Since this is generated data, only the '''GET''' method is allowed here, irrespective of role.

The following endpoint is associated with the event feed:
{| class="wikitable"
|-
! Endpoint
! Mime-type
! Required?
! Source @WF
! Description
|-
| /contests/<id>/event-feed
| application/x-ndjson
| yes
| CCS
| NDJSON feed of events as defined below
|}

===== Reconnection =====

If a client loses connection or needs to reconnect after a brief disconnect (e.g. client restart), it can use the 'time' argument to specify the last event it received:

 /event-feed?time=xx

If specified, the server will attempt to start sending events around the given time to reduce the volume of events and required reconciliation. If the time passed is too large or the server does not support this attribute, all objects will be sent. There is no guarantee that all updates (e.g. a team name correction, which is not time-based) that occurred during the time the client was disconnected will be reflected.