From 7498ba451116d457a4b12cad589b8c6149c16b6e Mon Sep 17 00:00:00 2001 From: Yury-Fridlyand Date: Fri, 7 Jul 2023 18:35:21 -0700 Subject: [PATCH 1/5] Update docs. Signed-off-by: Yury-Fridlyand --- docs/user/dql/expressions.rst | 2 +- docs/user/general/datatypes.rst | 107 +++++++++++++++++++++++++++----- 2 files changed, 92 insertions(+), 17 deletions(-) diff --git a/docs/user/dql/expressions.rst b/docs/user/dql/expressions.rst index 39d381f59c..275795707c 100644 --- a/docs/user/dql/expressions.rst +++ b/docs/user/dql/expressions.rst @@ -160,7 +160,7 @@ Here is an example for different type of comparison operators:: +---------+----------+---------+----------+----------+---------+ It is possible to compare datetimes. When comparing different datetime types, for example `DATE` and `TIME`, both converted to `DATETIME`. -The following rule is applied on coversion: a `TIME` applied to today's date; `DATE` is interpreted at midnight. +The following rule is applied on coversion: a `TIME` applied to today's date; `DATE` is interpreted at midnight. See example below:: os> SELECT current_time() > current_date() AS `now.time > today`, typeof(current_time()) AS `now.time.type`, typeof(current_date()) AS `now.date.type`; fetched rows / total rows = 1/1 diff --git a/docs/user/general/datatypes.rst b/docs/user/general/datatypes.rst index 4f1f3100c2..c8bd5cc6dd 100644 --- a/docs/user/general/datatypes.rst +++ b/docs/user/general/datatypes.rst @@ -155,15 +155,15 @@ The following matrix illustrates the conversions allowed by our query engine for +--------------+------+-------+---------+------+-------+--------+---------+--------------+------+--------+-----------+------+------+----------+----------+-----------+-----+--------+-----------+---------+ | STRING | E | E | E | E | E | E | IE | X | X | N/A | IE | IE | IE | IE | X | X | X | X | X | X | +--------------+------+-------+---------+------+-------+--------+---------+--------------+------+--------+-----------+------+------+----------+----------+-----------+-----+--------+-----------+---------+ -| TIMESTAMP | X | X | X | X | X | X | X | X | X | E | N/A | | | X | X | X | X | X | X | X | +| TIMESTAMP | X | X | X | X | X | X | X | X | X | E | N/A | IE | IE | IE | X | X | X | X | X | X | +--------------+------+-------+---------+------+-------+--------+---------+--------------+------+--------+-----------+------+------+----------+----------+-----------+-----+--------+-----------+---------+ -| DATE | X | X | X | X | X | X | X | X | X | E | | N/A | | X | X | X | X | X | X | X | +| DATE | X | X | X | X | X | X | X | X | X | E | E | N/A | IE | E | X | X | X | X | X | X | +--------------+------+-------+---------+------+-------+--------+---------+--------------+------+--------+-----------+------+------+----------+----------+-----------+-----+--------+-----------+---------+ -| TIME | X | X | X | X | X | X | X | X | X | E | | | N/A | X | X | X | X | X | X | X | +| TIME | X | X | X | X | X | X | X | X | X | E | E | E | N/A | E | X | X | X | X | X | X | +--------------+------+-------+---------+------+-------+--------+---------+--------------+------+--------+-----------+------+------+----------+----------+-----------+-----+--------+-----------+---------+ -| DATETIME | X | X | X | X | X | X | X | X | X | E | | | | N/A | X | X | X | X | X | X | +| DATETIME | X | X | X | X | X | X | X | X | X | E | E | E | E | N/A | X | X | X | X | X | X | +--------------+------+-------+---------+------+-------+--------+---------+--------------+------+--------+-----------+------+------+----------+----------+-----------+-----+--------+-----------+---------+ -| INTERVAL | X | X | X | X | X | X | X | X | X | E | | | | X | N/A | X | X | X | X | X | +| INTERVAL | X | X | X | X | X | X | X | X | X | E | X | X | X | X | N/A | X | X | X | X | X | +--------------+------+-------+---------+------+-------+--------+---------+--------------+------+--------+-----------+------+------+----------+----------+-----------+-----+--------+-----------+---------+ | GEO_POINT | X | X | X | X | X | X | X | X | X | | X | X | X | X | X | N/A | X | X | X | X | +--------------+------+-------+---------+------+-------+--------+---------+--------------+------+--------+-----------+------+------+----------+----------+-----------+-----+--------+-----------+---------+ @@ -236,8 +236,7 @@ Numeric values ranged from -2147483648 to +2147483647 are recognized as integer Date and Time Data Types ======================== -The date and time data types are the types that represent temporal values and SQL plugin supports types including DATE, TIME, DATETIME, TIMESTAMP and INTERVAL. By default, the OpenSearch DSL uses date type as the only date and time related type, which has contained all information about an absolute time point. To integrate with SQL language, each of the types other than timestamp is holding part of temporal or timezone information, and the usage to explicitly clarify the date and time types is reflected in the datetime functions (see `Functions `_ for details), where some functions might have restrictions in the input argument type. - +The date and time data types are the types that represent temporal values and SQL plugin supports types including ``DATE``, ``TIME``, ``DATETIME``, ``TIMESTAMP`` and ``INTERVAL``. By default, the OpenSearch DSL uses ``date`` type as the only date and time related type, which has contained all information about an absolute time point. To integrate with SQL language, each of the types other than timestamp is holding part of temporal or timezone information, and the usage to explicitly clarify the date and time types is reflected in the datetime functions (see `Functions `_ for details), where some functions might have restrictions in the input argument type. Date ---- @@ -299,7 +298,7 @@ Interval data type represents a temporal duration or a period. The syntax is as | Interval | INTERVAL expr unit | +----------+--------------------+ -The expr is any expression that can be iterated to a quantity value eventually, see `Expressions `_ for details. The unit represents the unit for interpreting the quantity, including MICROSECOND, SECOND, MINUTE, HOUR, DAY, WEEK, MONTH, QUARTER and YEAR.The INTERVAL keyword and the unit specifier are not case sensitive. Note that there are two classes of intervals. Year-week intervals can store years, quarters, months and weeks. Day-time intervals can store days, hours, minutes, seconds and microseconds. Year-week intervals are comparable only with another year-week intervals. These two types of intervals can only comparable with the same type of themselves. +The expr is any expression that can be iterated to a quantity value eventually, see `Expressions `_ for details. The unit represents the unit for interpreting the quantity, including ``MICROSECOND``, ``SECOND``, ``MINUTE``, ``HOUR``, ``DAY``, ``WEEK``, ``MONTH``, ``QUARTER`` and ``YEAR``. The ``INTERVAL`` keyword and the unit specifier are not case sensitive. Note that there are two classes of intervals. Year-week intervals can store years, quarters, months and weeks. Day-time intervals can store days, hours, minutes, seconds and microseconds. Year-week intervals are comparable only with another year-week intervals. These two types of intervals can only comparable with the same type of themselves. Conversion between date and time types @@ -320,7 +319,7 @@ Conversion from DATE Conversion from TIME >>>>>>>>>>>>>>>>>>>> -- Time value cannot be converted to any other date and time types since it does not contain any date information, so it is not meaningful to give no date info to a date/datetime/timestamp instance. +- When time value converted to any other datetime types, the date part of the new value is filled up with today's date. For example, a time value converted to date would produce today's date (like `CURDATE` function); a time value X converted to a timestamp would produce today's date at time X. Conversion from DATETIME @@ -354,18 +353,94 @@ A string can also represent and be converted to date and time types (except to i | True | False | True | +------------------------------------------------------------+-------------------------------------+----------------------------------+ +Please, see `more examples here <../dql/expressions.rst#toc-entry-15>`_. + +Date formats +------------ + +SQL plugin supports all named formats for OpenSearch ``date`` data type, custom formats and their combination. Please, refer to `OpenSearch docs `_ for format description. +Plugin detects which type of data is stored in ``date`` field according to formats given and returns results in the corresponding SQL types. +Given an index with the following mapping. + +.. code-block:: json + + { + "mappings" : { + "properties" : { + "date1" : { + "type" : "date", + "format": "yyyy-MM-dd" + }, + "date2" : { + "type" : "date", + "format": "date_time_no_millis" + }, + "date3" : { + "type" : "date", + "format": "hour_minute_second" + }, + "date4" : { + "type" : "date" + }, + "date5" : { + "type" : "date", + "format": "yyyy-MM-dd || time" + } + } + } + } + +Querying such index will provide a response with ``schema`` block as shown below. + +.. code-block:: json + + { + "query" : "SELECT * from date_formats LIMIT 0;" + } + +.. code-block:: json + + { + "schema": [ + { + "name": "date5", + "type": "timestamp" + }, + { + "name": "date4", + "type": "timestamp" + }, + { + "name": "date3", + "type": "time" + }, + { + "name": "date2", + "type": "timestamp" + }, + { + "name": "date1", + "type": "date" + }, + ], + "datarows": [], + "total": 0, + "size": 0, + "status": 200 + } + String Data Types ================= -A string is a sequence of characters enclosed in either single or double quotes. For example, both 'text' and "text" will be treated as string literal. To use quote characters in a string literal, you can use two quotes of the same type as the enclosing quotes:: +A string is a sequence of characters enclosed in either single or double quotes. For example, both 'text' and "text" will be treated as string literal. To use quote characters in a string literal, you can use two quotes of the same type as the enclosing quotes or a backslash symbol (``\``):: - os> SELECT 'hello', "world", '"hello"', "'world'", '''hello''', """world""" + os> SELECT 'hello', "world", '"hello"', "'world'", '''hello''', """world""", 'I\'m', "I\"m" fetched rows / total rows = 1/1 - +-----------+-----------+-------------+-------------+---------------+---------------+ - | 'hello' | "world" | '"hello"' | "'world'" | '''hello''' | """world""" | - |-----------+-----------+-------------+-------------+---------------+---------------| - | hello | world | "hello" | 'world' | 'hello' | "world" | - +-----------+-----------+-------------+-------------+---------------+---------------+ + +-----------+-----------+-------------+-------------+---------------+---------------+----------+----------+ + | 'hello' | "world" | '"hello"' | "'world'" | '''hello''' | """world""" | 'I\'m' | "I\"m" | + |-----------+-----------+-------------+-------------+---------------+---------------+----------+----------| + | hello | world | "hello" | 'world' | 'hello' | "world" | I'm | I"m | + +-----------+-----------+-------------+-------------+---------------+---------------+----------+----------+ Boolean Data Types ================== From 0e4dacd163f9e4126e9e63a42573028631d78713 Mon Sep 17 00:00:00 2001 From: Yury-Fridlyand Date: Mon, 10 Jul 2023 10:23:31 -0700 Subject: [PATCH 2/5] Apply suggestions from code review Signed-off-by: Yury-Fridlyand Co-authored-by: Forest Vey --- docs/user/general/datatypes.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/user/general/datatypes.rst b/docs/user/general/datatypes.rst index c8bd5cc6dd..79d1c2d494 100644 --- a/docs/user/general/datatypes.rst +++ b/docs/user/general/datatypes.rst @@ -236,7 +236,7 @@ Numeric values ranged from -2147483648 to +2147483647 are recognized as integer Date and Time Data Types ======================== -The date and time data types are the types that represent temporal values and SQL plugin supports types including ``DATE``, ``TIME``, ``DATETIME``, ``TIMESTAMP`` and ``INTERVAL``. By default, the OpenSearch DSL uses ``date`` type as the only date and time related type, which has contained all information about an absolute time point. To integrate with SQL language, each of the types other than timestamp is holding part of temporal or timezone information, and the usage to explicitly clarify the date and time types is reflected in the datetime functions (see `Functions `_ for details), where some functions might have restrictions in the input argument type. +The date and time data types represent temporal values and the SQL plugin supports types including ``DATE``, ``TIME``, ``DATETIME``, ``TIMESTAMP`` and ``INTERVAL``. By default, the OpenSearch DSL uses ``date`` type as the only date and time related type as it contains all information about an absolute time point. To integrate with SQL language each of the types other than timestamp hold part of the temporal or timezone information. This information can be used to explicitly clarify the date and time types reflected in the datetime functions (see `Functions `_ for details), where some functions might have restrictions in the input argument type. Date ---- @@ -319,7 +319,7 @@ Conversion from DATE Conversion from TIME >>>>>>>>>>>>>>>>>>>> -- When time value converted to any other datetime types, the date part of the new value is filled up with today's date. For example, a time value converted to date would produce today's date (like `CURDATE` function); a time value X converted to a timestamp would produce today's date at time X. +- When time value is converted to any other datetime types, the date part of the new value is filled up with today's date. For example, (like `CURDATE` function); a time value X converted to a timestamp would produce today's date at time X. Conversion from DATETIME From 9752afb59987cbe874fa6e847b9baa132cd06650 Mon Sep 17 00:00:00 2001 From: Yury-Fridlyand Date: Mon, 10 Jul 2023 11:36:04 -0700 Subject: [PATCH 3/5] Add example of quote escaping. Signed-off-by: Yury-Fridlyand --- docs/user/general/datatypes.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/user/general/datatypes.rst b/docs/user/general/datatypes.rst index 79d1c2d494..f14edd6005 100644 --- a/docs/user/general/datatypes.rst +++ b/docs/user/general/datatypes.rst @@ -434,13 +434,13 @@ String Data Types A string is a sequence of characters enclosed in either single or double quotes. For example, both 'text' and "text" will be treated as string literal. To use quote characters in a string literal, you can use two quotes of the same type as the enclosing quotes or a backslash symbol (``\``):: - os> SELECT 'hello', "world", '"hello"', "'world'", '''hello''', """world""", 'I\'m', "I\"m" + os> SELECT 'hello', "world", '"hello"', "'world'", '''hello''', """world""", 'I\'m', 'I''m', "I\"m" fetched rows / total rows = 1/1 - +-----------+-----------+-------------+-------------+---------------+---------------+----------+----------+ - | 'hello' | "world" | '"hello"' | "'world'" | '''hello''' | """world""" | 'I\'m' | "I\"m" | - |-----------+-----------+-------------+-------------+---------------+---------------+----------+----------| - | hello | world | "hello" | 'world' | 'hello' | "world" | I'm | I"m | - +-----------+-----------+-------------+-------------+---------------+---------------+----------+----------+ + +-----------+-----------+-------------+-------------+---------------+---------------+----------+----------+----------+ + | 'hello' | "world" | '"hello"' | "'world'" | '''hello''' | """world""" | 'I\'m' | 'I''m' | "I\"m" | + |-----------+-----------+-------------+-------------+---------------+---------------+----------+----------+----------| + | hello | world | "hello" | 'world' | 'hello' | "world" | I'm | I'm | I"m | + +-----------+-----------+-------------+-------------+---------------+---------------+----------+----------+----------+ Boolean Data Types ================== From c7b0e9933f8a1e4c2a51e1ad664cf4879f3338cc Mon Sep 17 00:00:00 2001 From: Yury-Fridlyand Date: Mon, 10 Jul 2023 11:40:42 -0700 Subject: [PATCH 4/5] Update docs/user/general/datatypes.rst Signed-off-by: Yury-Fridlyand Co-authored-by: Matthew Wells --- docs/user/general/datatypes.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user/general/datatypes.rst b/docs/user/general/datatypes.rst index f14edd6005..fd3f3f526b 100644 --- a/docs/user/general/datatypes.rst +++ b/docs/user/general/datatypes.rst @@ -319,7 +319,7 @@ Conversion from DATE Conversion from TIME >>>>>>>>>>>>>>>>>>>> -- When time value is converted to any other datetime types, the date part of the new value is filled up with today's date. For example, (like `CURDATE` function); a time value X converted to a timestamp would produce today's date at time X. +- When time value is converted to any other datetime types, the date part of the new value is filled up with today's date, like with the `CURDATE` function. For example, a time value X converted to a timestamp would produce today's date at time X. Conversion from DATETIME From 7c3018bff97dfe4db6428de8acb676195d46fb82 Mon Sep 17 00:00:00 2001 From: Yury-Fridlyand Date: Mon, 10 Jul 2023 13:04:43 -0700 Subject: [PATCH 5/5] Update docs/user/general/datatypes.rst Signed-off-by: Yury-Fridlyand Co-authored-by: Matthew Wells --- docs/user/general/datatypes.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user/general/datatypes.rst b/docs/user/general/datatypes.rst index fd3f3f526b..a265ffd4c9 100644 --- a/docs/user/general/datatypes.rst +++ b/docs/user/general/datatypes.rst @@ -236,7 +236,7 @@ Numeric values ranged from -2147483648 to +2147483647 are recognized as integer Date and Time Data Types ======================== -The date and time data types represent temporal values and the SQL plugin supports types including ``DATE``, ``TIME``, ``DATETIME``, ``TIMESTAMP`` and ``INTERVAL``. By default, the OpenSearch DSL uses ``date`` type as the only date and time related type as it contains all information about an absolute time point. To integrate with SQL language each of the types other than timestamp hold part of the temporal or timezone information. This information can be used to explicitly clarify the date and time types reflected in the datetime functions (see `Functions `_ for details), where some functions might have restrictions in the input argument type. +The datetime types supported by the SQL plugin are ``DATE``, ``TIME``, ``DATETIME``, ``TIMESTAMP``, and ``INTERVAL``, with date and time being used to represent temporal values. By default, the OpenSearch DSL uses ``date`` type as the only date and time related type as it contains all information about an absolute time point. To integrate with SQL language each of the types other than timestamp hold part of the temporal or timezone information. This information can be used to explicitly clarify the date and time types reflected in the datetime functions (see `Functions `_ for details), where some functions might have restrictions in the input argument type. Date ----