Feature: Add `@dateFormat` decorator #1796

timotheeguerin · 2023-04-05T22:45:43Z

github-actions · 2023-04-05T22:45:57Z

Changes in this PR will be published to the following url to try(check status of TypeSpec Pull Request Try It pipeline for publish status):
Playground: https://cadlplayground.z22.web.core.windows.net/prs/1796/

Website: https://cadlwebsite.z1.web.core.windows.net/prs/1796/

timotheeguerin · 2023-04-05T22:48:13Z

packages/compiler/lib/decorators.tsp

@@ -337,6 +337,9 @@ extern dec projectedName(target: unknown, targetName: string, projectedName: str
 */
 extern dec discriminator(target: object | Union, propertyName: string);

+alias KnownDateFormat = "rfc1123" | "rfc7231" | "unixTimeStamp";
+extern dec dateFormat(target: zonedDateTime | ModelProperty, format: KnownDateFormat | string);


Does @dateFormat make the most sense as the name and not @dateSerializationFormat. It is different from @format on a string which actually dictate how the string is formatted vs here we are telling what serialization pattern to use

I like shorter names, but dateSerializationFormat is more accurate

markcowl · 2023-04-06T00:20:53Z

packages/compiler/lib/decorators.tsp

@@ -337,6 +337,9 @@ extern dec projectedName(target: unknown, targetName: string, projectedName: str
 */
 extern dec discriminator(target: object | Union, propertyName: string);

+alias KnownDateFormat = "rfc1123" | "rfc7231" | "unixTimeStamp";
+extern dec dateFormat(target: zonedDateTime | ModelProperty, format: KnownDateFormat | string);


I like shorter names, but dateSerializationFormat is more accurate

markcowl · 2023-04-06T00:21:45Z

packages/openapi3/test/primitive-types.test.ts

+      deepStrictEqual(res.schemas.MyDate, { type: "string", format: "date-time" });
+    });
+
+    it("set format to 'timestamp' and type to 'ingeger' for 'unixtimestamp' format", async () => {


Suggested change

it("set format to 'timestamp' and type to 'ingeger' for 'unixtimestamp' format", async () => {

it("set format to 'timestamp' and type to 'integer' for 'unixtimestamp' format", async () => {

markcowl · 2023-04-06T00:31:49Z

docs/standard-library/built-in-decorators.md

+
+
+```typespec
+dec dateFormat(target: zonedDateTime | ModelProperty, format: rfc1123 | rfc7231 | unixTimeStamp | string)


The values should be rfc3339 and rfc7231. rfc7231 is the replacement for rfc1123

packages/compiler/lib/decorators.ts

markcowl · 2023-04-06T00:33:51Z

packages/openapi3/src/openapi.ts

@@ -1307,6 +1308,16 @@ function createOAPIEmitter(program: Program, options: ResolvedOpenAPI3EmitterOpt
      newTarget.format = "password";
    }

+    const dateFormat = getDateFormat(program, typespecType);


We should also set the default dateFormat, which requires knowing whether the date is used in a header (rfc7231) or elsewhere (rfc 3339)

We’ll in openapi date-time already assume that from what I understand.

AFAIK, no, see: https://github.com/OAI/OpenAPI-Specification/blob/main/versions/3.0.3.md?plain=1#L170

date-time should be the default for non-header types, but for headers, we need to be explicit

Oh I see, I’ll update

markcowl · 2023-04-06T00:34:49Z

packages/openapi3/test/primitive-types.test.ts

@@ -189,4 +189,56 @@ describe("openapi3: primitives", () => {
      });
    });
  });
+
+  describe("date format", () => {
+    it("set format to 'date-time' by default", async () => {


Should have an alternate where this is ued in a header, and the default format is rfc-7231

bterlson · 2023-04-06T00:39:58Z

Sorry forgot the context, remind me why this isn't just the existing format decorator?

Also how does this combine with utcDateTime and offsetDateTime? What are the semantic differences between say a utcDateTime without a format and an offsetDateTime with a unixTimestamp format?

Co-authored-by: Mark Cowlishaw <markcowl@microsoft.com>

timotheeguerin · 2023-04-06T00:47:03Z

@bterlson Think that design discussion was done when I was one leave so can’t say about what was discussed but from my comment above I think it is different from format as it means serialization format vs format means a know pattern for the string.

for the utc and offset date time I am not fully sure either what was the rationale behind that distinction

but I guess that would mean you shouldn’t do Unix time stamp for offsetdattime

bterlson · 2023-04-06T01:12:27Z

I was probably present for the discussion but I've forgotten it. Forgive me! (Also I've spent a bunch of time in Json Schema land and paging back in arcane date knowledge)

I worry that this decorator's design isn't fully rationalized with existing/planned features so it might be worth circling back a little? Some thoughts:

The existing format decorator isn't just for string types (despite what the docs say, int16 is an example integer format). Json Schema leaves format unbounded and allows it to be used for any type. We should probably too? (Or the json schema emitter will need yet another format decorator?)

The Json Schema and OAS include serialization format in its format registry - format: "date-time" requires RFC3339. Seems reasonable to say that we're extending this list with some additional known formats. A somewhat orthogonal axis might make it hard for OAS and JsonSchema emitters to know what combinations are valid/when to emit a registered format vs. fall back to pattern or something?

The idea behind utcDateTime and offsetDateTime is to describe the contract the service is exposing with regard to whether it expects/persists/round trips time zones and offsets (zonedDateTime), offsets (offsetDateTime) or just ticks since epoch in UTC (utcDateTime).

Format is mostly orthogonal to this notion, except that some formats might imply information loss depending on type it's applied to. I think this should be an error probably? unixTimestamp should not be applied to either offsetDateTime or utcDateTime - if you want a unixTimestamp, you should use utcDateTime because you're saying clients will not send and your service will not accept or persist either offsets or time zones.

timotheeguerin · 2023-04-06T01:20:05Z

My argument for not using format still applies. In json schema we are saying the value is a string with a specific format (date time). Or an int being something more specific maybe.

but here we have already that distinction of a type which is where it feels weird to me

format(in jsonschema and openapi), @Format -> precise the type
Date format(or what we want to do here) -> specify how we can serialize this type of the wire

maybe this is not the right solution but format to me sounds like the wrong one as well

bterlson · 2023-04-06T02:37:35Z

Doesn't format: date-time in OpenAPI/Json Schema describe the serialization format moreso than the data type? date-time requires RFC3339, meanwhile format is type-agnostic, the type is described by the type field (though date-time only makes sense on a string).

timotheeguerin · 2023-04-06T15:48:49Z

@bterlson I guess it does technically but to me it still sounds like @format is mean to represent a certain pattern for the data, which in the case of jsonschema and openapi is a Serialization format.

Like if we look at those format in jsonschema:

uuid: string is representing a uuid
uri: string is representing a uri
date-time: string is representing a rfc3339 date time
int8: that integer can only be from -127 to 128

Using format on the date times would to me imply similar style of pattern restriction

Like, not that it make sense but if you said @format("date") on utcDateTime) to specify that even though this is using a full date time representation, the time will always be 0.

bterlson · 2023-04-07T01:04:42Z

In a world where pattern can apply to any type, isn't serialization format just a pattern restriction? I see your point if we say format is basically stringFormat as effectively a well-known list of regexps for pattern, but in a world where format can apply to any type, I don't know that the distinction is meaningful. At the least, it seems hard to describe to people when one is used over the other.

timotheeguerin · 2023-04-07T01:56:50Z

I guess an argument against that decorator is, let’s say we have this new geolocation type, I guess that also needs to specify how to serialize it. Seems overkill to have to have a new decorator per month primitive type that might serialize in many different ways

lmazuel · 2023-04-18T18:15:30Z

docs/standard-library/built-in-decorators.md

+
+#### Target
+
+`union zonedDateTime | ModelProperty`


Should this work for duration as well?

markcowl · 2023-04-26T18:07:17Z

@format -> type format in openapi
@serializationFormat -> general formatting hints that might encompass openapi format and other formatting mechanisms

Note that, if we have a core decorator that serves the purpose of @format, then we likely want to call it something else
-or -

@format

-or -

type-specific format decorators

// we decided against type-specific decorators

Also note that there is need for a logical 'duration' that has need for a 'units' parameter, which might also be accommodated by the @serializationFormat decorator.

timotheeguerin · 2023-04-26T19:26:33Z

`@format` vs `@encode` decorators

Merging with original @encode proposal which is basically the same thing https://github.com/Azure/typespec-azure/issues/1790

Description:

@format: Existing decorator meant to provide an known pattern for the type
@encode (mentied previously as @serializationFormat) Is meant to provide the serialization protocol for that type over the wire

/**
 * @param encodeFormat protocol name for the encoding
 * @param encodeToType optional type that this should be encoded to. Default to `string`
 */
extern dec encode(target: unknown, encodeFormat: valueof string, encodeToType?: string | numeric)

Examples

Specify uuid, email, ipv4, ipv6

model Bar {
  @format("uuid") id: string;
  @format("email") userEmail: string;
  @format("ipv4") ipv4: string;
  @format("ipv6") ipv6: string;
}

Specify rfc3339

model Bar {
  @encode("rfc3339")
  createdAt: utcDateTime;

  @encode("rfc3339")
  modifiedAt: offsetDateTime;

  @encode("rfc3339")
  runtime: duration;

  // We could also technically represent the datetime as a string and then we'd use the `@format` decorator to specify the pattern of that string
  @format("date-time")
  createdAtButString: string;
}

Specify rfc7231

model Bar {
  @encode("rfc7231")
  createdAt: utcDateTime;

  @encode("rfc7231")
  modifiedAt: offsetDateTime;

  @encode("rfc7231")
  runtime: duration;

  @format("date-time-rfc7231")
  createdAtButString: string;
}

Specify unixTimestamp

model Bar {
  @encode("unixTimestamp", int32)
  createdAt: utcDateTime;

  @encode("unixTimestamp", int32)  // invalid
  modifiedAt: offsetDateTime;

  @encode("unixTimestamp", int32) // invalid
  runtime: duration;


}

Specify Duration serialized as seconds

model Bar {
  @encode("seconds", int32)
  runtimeAsDouble: duration;
  @encode("seconds", float64)
  runtimeAsInt: duration;
}

timotheeguerin · 2023-05-03T23:14:14Z

Replaced by updated @encode decorator #1899

timotheeguerin added 2 commits April 5, 2023 15:34

Add date format decorator to compiler

d176f35

Add openapi3 logic

4c9dcb3

timotheeguerin requested review from bterlson, daviwil, nguerrera, markcowl and allenjzhang as code owners April 5, 2023 22:45

timotheeguerin commented Apr 5, 2023

View reviewed changes

timotheeguerin added 3 commits April 5, 2023 15:49

Chanmgelog and regendocs

61cb5af

Add timestamp special handling

7be46b9

Merge branch 'main' into feature/date-format-decorator

0e8b73e

markcowl requested changes Apr 6, 2023

View reviewed changes

Update packages/compiler/lib/decorators.ts

aff8a16

Co-authored-by: Mark Cowlishaw <markcowl@microsoft.com>

daviwil approved these changes Apr 6, 2023

View reviewed changes

lmazuel reviewed Apr 18, 2023

View reviewed changes

docs/standard-library/built-in-decorators.md

#### Target

`union zonedDateTime | ModelProperty`

Copy link

Member

lmazuel Apr 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this work for duration as well?

tadelesh mentioned this pull request Apr 26, 2023

Add client format on duration tests Azure/cadl-ranch#273

Closed

4 tasks

timotheeguerin closed this May 3, 2023

timotheeguerin deleted the feature/date-format-decorator branch July 26, 2023 22:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature: Add `@dateFormat` decorator #1796

Feature: Add `@dateFormat` decorator #1796

timotheeguerin commented Apr 5, 2023

github-actions bot commented Apr 5, 2023

timotheeguerin Apr 5, 2023 •

edited

Loading

markcowl Apr 6, 2023

markcowl Apr 6, 2023

markcowl Apr 6, 2023

markcowl Apr 6, 2023

markcowl Apr 6, 2023

timotheeguerin Apr 6, 2023

markcowl Apr 6, 2023

timotheeguerin Apr 6, 2023

markcowl Apr 6, 2023

bterlson commented Apr 6, 2023

timotheeguerin commented Apr 6, 2023 •

edited

Loading

bterlson commented Apr 6, 2023

timotheeguerin commented Apr 6, 2023

bterlson commented Apr 6, 2023

timotheeguerin commented Apr 6, 2023

bterlson commented Apr 7, 2023

timotheeguerin commented Apr 7, 2023

lmazuel Apr 18, 2023

markcowl commented Apr 26, 2023 •

edited

Loading

timotheeguerin commented Apr 26, 2023 •

edited

Loading

timotheeguerin commented May 3, 2023

	it("set format to 'timestamp' and type to 'ingeger' for 'unixtimestamp' format", async () => {
	it("set format to 'timestamp' and type to 'integer' for 'unixtimestamp' format", async () => {



		```typespec
		dec dateFormat(target: zonedDateTime \| ModelProperty, format: rfc1123 \| rfc7231 \| unixTimeStamp \| string)

Feature: Add @dateFormat decorator #1796

Feature: Add @dateFormat decorator #1796

Conversation

timotheeguerin commented Apr 5, 2023

github-actions bot commented Apr 5, 2023

timotheeguerin Apr 5, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bterlson commented Apr 6, 2023

timotheeguerin commented Apr 6, 2023 • edited Loading

bterlson commented Apr 6, 2023

timotheeguerin commented Apr 6, 2023

bterlson commented Apr 6, 2023

timotheeguerin commented Apr 6, 2023

bterlson commented Apr 7, 2023

timotheeguerin commented Apr 7, 2023

Choose a reason for hiding this comment

markcowl commented Apr 26, 2023 • edited Loading

timotheeguerin commented Apr 26, 2023 • edited Loading

@format vs @encode decorators

Examples

Specify uuid, email, ipv4, ipv6

Specify rfc3339

Specify rfc7231

Specify unixTimestamp

Specify Duration serialized as seconds

timotheeguerin commented May 3, 2023

Feature: Add `@dateFormat` decorator #1796

Feature: Add `@dateFormat` decorator #1796

timotheeguerin Apr 5, 2023 •

edited

Loading

timotheeguerin commented Apr 6, 2023 •

edited

Loading

markcowl commented Apr 26, 2023 •

edited

Loading

timotheeguerin commented Apr 26, 2023 •

edited

Loading

`@format` vs `@encode` decorators