Common rules for inclusion and querying

optimade.rst

@@ -156,7 +156,7 @@ The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SH
 **Queryable property**
     An entry property that can be referred to in the filtering of results.
     See section `API Filtering Format Specification`_ for more information on formulating filters on properties.
-    The definitions of specific properties in section `Entry List`_ states which ones MUST be queryable and which are RECOMMENDED.
+    The definitions of specific properties in section `Entry List`_ states which ones MUST be queryable, the rest being OPTIONAL.
 
 **ID**
     The ID entry property is a unique string referencing a specific entry in the database.
@@ -589,6 +589,9 @@ There MAY be multiple entry listing endpoints, depending on how many types of en
 Specific standard entry types are specified in section `Entry list`_.
 The API implementation MAY provide other entry types than the ones standardized in this specification, but such entry types MUST be prefixed by a database-provider-specific prefix.
 
+By default, only those properties that are marked as REQUIRED in section `Entry list`_ MUST be present in the response.
+Query parameter *response_fields* MAY be used to explicitly include or exclude properties.
+
 Entry Listing URL Query Parameters
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -731,6 +734,8 @@ Examples:
 - :query-url:`http://example.com/optimade/v0.9/structures/exmpl%3Astruct_3232823`
 - :query-url:`http://example.com/optimade/v0.9/calculations/232132`
 
+The requirements for property inclusion are the same as defined in section `Entry Listing Endpoints`_.
+
 Single Entry URL Query Parameters
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -1076,6 +1081,9 @@ An OPTiMaDe filter expression is passed in the parameter :query-param:`filter` a
 API <https://jsonapi.org/format/1.0/#fetching-filtering>`__.
 The filter expression allows desired properties to be compared against search values; several such comparisons can be combined using the logical conjunctions AND, OR, NOT, and parentheses, with their usual semantics.
 
+Only those properties marked as REQUIRED in section `Entry list`_ MUST be queryable.
+Query support for other properties is OPTIONAL.
+
 When provided as an URL query parameter, the contents of the :query-param:`filter` parameter is URL-encoded by the client in the HTTP GET request, and then URL-decoded by the API implementation before any further parsing takes place.
 In particular, this means the client MUST escape special characters in string values as described below for `String values`_ before the URL encoding, and the API implementation MUST first URL-decode the :query-param:`filter` parameter before reversing the escaping of string tokens.
 
@@ -1382,7 +1390,7 @@ id
 - **Type**: string.
 - **Requirements/Conventions**:
 
-  - **Response**: REQUIRED in the response unless explicitly excluded.
+  - **Response**: REQUIRED in the response.
   - **Query**: MUST be a queryable property with support for all mandatory filter operators.
   - See section `Definition of Terms`_.
 
@@ -1401,7 +1409,7 @@ type
 - **Type**: string.
 - **Requirements/Conventions**:
 
-  - **Response**: REQUIRED in the response unless explicitly excluded.
+  - **Response**: REQUIRED in the response.
   - **Query**: Support for queries on this property is OPTIONAL.
     If supported, only a subset of string comparison operators MAY be supported.
 
@@ -1415,7 +1423,6 @@ immutable\_id
 - **Type**: string.
 - **Requirements/Conventions**:
 
-  - **Response**: OPTIONAL in the response.
   - **Query**: If present, MUST be a queryable property with support for all mandatory filter operators.
 
 - **Examples**:
@@ -1430,7 +1437,7 @@ last\_modified
 - **Type**: timestamp.
 - **Requirements/Conventions**:
 
-  - **Response**: REQUIRED in the response unless explicitly excluded.
+  - **Response**: REQUIRED in the response.
   - **Query**: MUST be a queryable property with support for all mandatory filter operators.
 
 - **Example**:
@@ -1444,9 +1451,6 @@ database-provider-specific properties
 - **Type**: Decided by the API implementation.
 - **Requirements/Conventions**:
 
-  - **Response**: OPTIONAL in the response.
-  - **Query**: Support for queries on these properties are OPTIONAL.
-    If supported, only a subset of filter operators MAY be supported.
   - These MUST be prefixed by a database-provider-specific prefix as defined in appendix `Database-Provider-Specific Namespace Prefixes`_.
 
 - **Examples**: A few examples of valid database-provided-specific property names follows:
@@ -1469,7 +1473,7 @@ elements
 - **Type**: list of strings.
 - **Requirements/Conventions**:
 
-  - **Response**: REQUIRED in the response unless explicitly excluded.
+  - **Response**: REQUIRED in the response.
   - **Query**: MUST be a queryable property with support for all mandatory filter operators.
   - The strings are the chemical symbols, written as uppercase letter plus optional lowercase letters.
   - The order MUST be alphabetical.
@@ -1490,7 +1494,7 @@ nelements
 - **Type**: integer
 - **Requirements/Conventions**:
 
-  - **Response**: REQUIRED in the response unless explicitly excluded.
+  - **Response**: REQUIRED in the response.
   - **Query**: MUST be a queryable property with support for all mandatory filter operators.
 
 - **Example**: :val:`3`
@@ -1507,7 +1511,7 @@ elements\_ratios
 - **Type**: list of floats
 - **Requirements/Conventions**:
 
-  - **Response**: REQUIRED in the response unless explicitly excluded.
+  - **Response**: REQUIRED in the response.
   - **Query**: MUST be a queryable property with support for all mandatory filter operators.
   - Composed by the proportions of elements in the structure as a list of floating point numbers.
   - The sum of the numbers MUST be 1.0 (within floating point accuracy)
@@ -1529,7 +1533,7 @@ chemical\_formula\_descriptive
 - **Type**: string
 - **Requirements/Conventions**:
 
-  - **Response**: REQUIRED in the response unless explicitly excluded.
+  - **Response**: REQUIRED in the response.
   - **Query**: MUST be a queryable property with support for all mandatory filter operators.
   - The chemical formula is given as a string consisting of properly capitalized element symbols followed by integers or decimal numbers, balanced parentheses, square, and curly brackets ``(``,\ ``)``, ``[``,\ ``]``, ``{``, ``}``, commas, the ``+``, ``-``, ``:`` and ``=`` symbols.
     The parentheses are allowed to be followed by a number.
@@ -1560,8 +1564,7 @@ chemical\_formula\_reduced
 - **Type**: string
 - **Requirements/Conventions**:
 
-  - **Response**: REQUIRED in the response unless explicitly excluded.
-
+  - **Response**: REQUIRED in the response.
   - **Query**: MUST be a queryable property.
     However, support for filters using partial string matching with this property is OPTIONAL (i.e., BEGINS WITH, ENDS WITH, and CONTAINS).
     Intricate querying on formula components are instead recommended to be formulated using set-type filter operators on the multi valued :property:`elements` and :property:`elements_proportions` properties.
@@ -1589,8 +1592,6 @@ chemical\_formula\_hill
 - **Type**: string
 - **Requirements/Conventions**:
 
-  - **Response**: OPTIONAL in the response.
-  - **Query**: Support for queries on these properties are OPTIONAL. If supported, only a subset of filter operators MAY be supported.
   - The overall scale factor of the chemical proportions is chosen such that the resulting values are integers that indicate the most chemically relevant unit of which the system is composed.
     For example, if the structure is a repeating unit cell with four hydrogens and four oxygens that represents two hydroperoxide molecules, :property:`chemical_formula_hill` is :val:`"H2O2"` (i.e., not :val:`"HO"`, nor :val:`"H4O4"`).
   - If the chemical insight needed to ascribe a Hill formula to the system is not present, the property MUST be handled as unset.
@@ -1616,7 +1617,7 @@ chemical\_formula\_anonymous
 - **Type**: string
 - **Requirements/Conventions**:
 
-  - **Response**: REQUIRED in the response unless explicitly excluded.
+  - **Response**: REQUIRED in the response.
   - **Query**: MUST be a queryable property. However, support for filters using partial string matching with this property is OPTIONAL (i.e., BEGINS WITH, ENDS WITH, and CONTAINS).
 
 - **Examples**:
@@ -1637,7 +1638,7 @@ dimension\_types
 - **Type**: list of integers.
 - **Requirements/Conventions**:
 
-  - **Response**: REQUIRED in the response unless explicitly excluded.
+  - **Response**: REQUIRED in the response.
   - **Query**: MUST be a queryable property. Support for equality comparison is REQUIRED, support for other comparison operators are OPTIONAL.    
   - MUST be a list of length 3.
   - Each integer element MUST assume only the value 0 or 1.
@@ -1656,8 +1657,7 @@ lattice\_vectors
 - **Type**: list of list of floats.
 - **Requirements/Conventions**:
 
-  - **Response**: REQUIRED in the response unless explicitly excluded, except when property `dimension_types`_ is equal to :val:`[0, 0, 0]` (in this case it is OPTIONAL).
-  - **Query**: Support for queries on this property is OPTIONAL. If supported, filters MAY support only a subset of comparison operators.
+  - **Response**: REQUIRED in the response.
   - MUST be a list of three vectors *a*, *b*, and *c*, where each of the vectors MUST BE a list of the vector's coordinates along the x, y, and z Cartesian coordinates.
     (Therefore, the first index runs over the three lattice vectors and the second index runs over the x, y, z Cartesian coordinates).
   - For databases that do not define an absolute Cartesian system (e.g., only defining the length and angles between vectors), the first lattice vector SHOULD be set along *x* and the second on the *xy*-plane.
@@ -1674,8 +1674,6 @@ cartesian\_site\_positions
 - **Type**: list of list of floats and/or unknown values
 - **Requirements/Conventions**:
 
-  - **Response**: REQUIRED in the response unless explicitly excluded.
-  - **Query**: Support for queries on this property is OPTIONAL. If supported, filters MAY support only a subset of comparison operators.
   - It MUST be a list of length N times 3, where N is the number of sites in the structure.
   - An entry MAY have multiple sites at the same Cartesian position (for a relevant use of this, see e.g., the property `assemblies`_).
   - If a component of the position is unknown, the :val:`null` value should be provided instead (see section `Properties with unknown value`_).
@@ -1693,11 +1691,6 @@ nsites
 
 - **Description**: An integer specifying the length of the :property:`cartesian_site_positions` property.
 - **Type**: integer  
-- **Requirements/Conventions**:
-
-  - **Response**: REQUIRED in the response unless explicitly excluded.
-  - **Query**: MUST be a queryable property with support for all mandatory filter operators.
-
 - **Examples**:
 
   - :val:`42`
@@ -1715,8 +1708,6 @@ species\_at\_sites
 - **Type**: list of strings.
 - **Requirements/Conventions**:
 
-  - **Response**: REQUIRED in the response unless explicitly excluded.
-  - **Query**: Support for queries on this property is OPTIONAL. If supported, filters MAY support only a subset of comparison operators.
   - MUST have length equal to the number of sites in the structure (first dimension of the list property `cartesian_site_positions`_).
   - Each species name mentioned in the :property:`species_at_sites` list MUST be described in the list property `species`_ (i.e. for each value in the :property:`species_at_sites` list there MUST exist exactly one dictionary in the :property:`species` list with the :property:`name` attribute equal to the corresponding :property:`species_at_sites` value).
   - Each site MUST be associated only to a single species.
@@ -1742,8 +1733,6 @@ species
 
 - **Requirements/Conventions**:
 
-  - **Response**: REQUIRED in the response unless explicitly excluded.
-  - **Query**: Support for queries on this property is OPTIONAL. If supported, filters MAY support only a subset of comparison operators.
   - Each list member MUST be a dictionary with the following keys:
 
     - **name**: REQUIRED; gives the name of the species; the **name** value MUST be unique in the :property:`species` list;
@@ -1977,7 +1966,6 @@ Database-provider-specific entry types MUST have all properties described above
 
 - **Requirements/Conventions for properties in database-provider-specific entry types**:
 
-  - **Response**: OPTIONAL in the response.
   - **Query**: Support for queries on these properties are OPTIONAL.
     If supported, only a subset of filter operators MAY be supported.
 
@@ -2062,7 +2050,6 @@ Relationships with calculations MAY be used to indicate provenance where a struc
 
     - **Requirements/Conventions for database-provider-specific properties of calculations entries**:
 
-      - **Response**: OPTIONAL in the response.
       - **Query**: Support for queries on these properties are OPTIONAL.
         If supported, only a subset of filter operators MAY be supported.