Tag types

[zpdDG4gta8XKpMCd]

Problem

• There is no straight way to encode a predicate (some checking) about a value in its type.

Details

There are situations when a value has to pass some sort of check/validation prior to being used. For example: a min/max functions can only operate on a non-empty array so there must be a check if a given array has any elements. If we pass a plain array that might as well be empty, then we need to account for such case inside the min/max functions, by doing one of the following:

• crashing
• returning undefined
• returning a given default value

This way the calling side has to deal with the consequences of min/max being called yet not being able to deliver.

function min<a>(values: a[], isGreater: (one: a, another: a) => boolean) : a {
   if (values.length < 1) { throw Error('Array is empty.'); }
   // ...
}
var found;
try {
   found = min([]);
} catch (e) {
   found = undefined;
}

Solution

A better idea is to leverage the type system to rule out a possibility of the min function being called with an empty array. In order to do so we might consider so called tag types.

A tag type is a qualifier type that indicates that some predicate about a value it is associated with holds true.

const enum AsNonEmpty {} // a tag type that encodes that a check for being non-empty was passed

function min<a>(values: a[] & AsNonEmpty) : a {
   // only values that are tagged with AsNonEmpty can be passed to this function
   // the type system is responsible for enforcing this constraint
   // a guarantee by the type system that only non-empty array will be passed makes
   // implementation of this function simpler because only one main case needs be considered
   // leaving empty-case situations outside of this function
}

min([]); // <-- compile error, tag is missing, the argument is not known to be non-empty
min([1, 2]); // <-- compile error, tag is missing again, the argument is not know to be non-empty

it's up to the developer in what circumstances an array gets its AsNonEmpty tag, which can be something like:

// guarntee is given by the server, so we always trust the data that comes from it
interface GuaranteedByServer {
    values: number[] & AsNonEmpty;
}

Also tags can be assigned at runtime:

function asNonEmpty(values: a[]) : (a[] & AsNonEmpty) | void {
    return values.length > 0 ? <a[] & AsNonEmpty> : undefined;
}

function isVoid<a>(value: a | void) : value is void {
  return value == null;
}

var values = asNonEmpty(Math.random() > 0.5 ? [1, 2, 3] : []);
if (isVoid(values)) {
   // there are no  elements in the array, so we can't call min
   var niceTry = min(values); // <-- compile error; 
} else {
    var found = min(values); // <-- ok, tag is there, safe to call
}

As was shown in the current version (1.6) an empty const enum type can be used as a marker type (AsNonEmpty in the above example), because

• enums might not have any members and yet be different from the empty type
• enums are branded (not assignable to one another)

However enums have their limitations:

• enum is assignable by numbers
• enum cannot hold a type parameter
• enum cannot have members

A few more examples of what tag type can encode:

• string & AsTrimmed & AsLowerCased & AsAtLeast3CharLong
• number & AsNonNegative & AsEven
• date & AsInWinter & AsFirstDayOfMonth

Custom types can also be augmented with tags. This is especially useful when the types are defined outside of the project and developers can't alter them.

• User & AsHavingClearance

ALSO NOTE: In a way tag types are similar to boolean properties (flags), BUT they get type-erased and carry no rutime overhead whatsoever being a good example of a zero-cost abstraction.

UPDATED:

Also tag types can be used as units of measure in a way:

• string & AsEmail, string & AsFirstName:

var email = <string & AsEmail> 'aleksey.bykov@yahoo.com';
var firstName = <string & AsFirstName> 'Aleksey';
firstName = email; // <-- compile error

• number & In<Mhz>, number & In<Px>:

var freq = <number & In<Mhz>> 12000;
var width =   <number & In<Px>> 768;
freq = width; // <-- compile error

function divide<a, b>(left: number & In<a>, right: number & In<b> & AsNonZero) : number & In<Ratio<a, b>> {
     return <number & In<Ratio<a, b>>> left  / right;
}

var ratio = divide(freq, width); // <-- number & In<Ratio<Mhz, Px>>

[mhegazy]

A few thoughts, in the typescript compiler we have used brands to achieve a similar behavior, see: https://github.com/Microsoft/TypeScript/blob/master/src/compiler/types.ts#L485; #1003 could make creating tags a little bit cleaner. A nominal type would be far cleaner solution (#202).

[zpdDG4gta8XKpMCd]

one more use case just came across:

var scheduled = setTimeout(function() { }, 1000);
clearInterval(scheduled); // <-- runtime bug

with type tag this situation could have been avoided

declare function setTimeout(act: () => void, delay: number) : number & AsForTimeout;
declare function clearInterval(scheduled: number & AsForInterval);
var scheduled = setTimeout(function() { }, 1000);
clearInterval(scheduled); // <-- compile error

[jovdb]

With Typescript 2 you can now simulate the behavior you want:

declare class MinValue<T extends number> {
    private __minValue: T;
}

// MinValue type guard
function hasMinValue<T extends number>(value: number, minValue: T): value is number & MinValue<T> {
    return value >= minValue;
}

// Use it
function delay(fn: Function, milliSeconds: number & MinValue<0>) { }

const delayInMs = 200;
delay(() => { }, delayInMs); // error: number not assignable to MinValue<0>

if (hasMinValue(delayInMs, 0)) {
    delay(() => { }, delayInMs); // OK
}

if (hasMinValue(delayInMs, 100)) {
    delay(() => { }, delayInMs); // error: MinValue<100> not assignable to MinValue<0>
}

with this concept, you can also create Ranges:

// MaxValue 
declare class MaxValue<T extends number> {
    private __maxValue: T;
}

// MaxValue type guard
function hasMaxValue<T extends number>(value: number, maxValue: T): value is number & MaxValue<T> {
    return value <= maxValue;
}

// RangeValue
type RangeValue<TMin extends number, TMax extends number> = MinValue<TMin> & MaxValue<TMax>; 

// RangeValue type guard
function inRange<TMin extends number, TMax extends number>(value: number, minValue: TMin, maxValue: TMax): value is number & RangeValue<TMin, TMax> {
    return value >= minValue && value >= maxValue;
}

// Range example
//-----------------
type Opacity = RangeValue<0, 1>;
function setTransparency(opacity: number & Opacity) {
   // ...
}


const opacity = 0.3;
setTransparency(opacity); // error: 'number' not assignable to MinValue<0>

if (inRange(opacity, 0, 1)) { 
    setTransparency(opacity); // OK
}

if (inRange(opacity, 0, 3)) { 
    setTransparency(opacity); // error: MinValue<0> & MaxValue<3> not assignable to MinValue<0> & MaxValue<1>
}

[mindplay-dk]

IMO this feature is long over due - mainly because there are so many types of strings: UUID, e-mail address, hex color-code and not least entity-specific IDs, which would be incredibly useful when coupling entities to repositories, etc.

I currently use this work-around:

export type UUID = string & { __UUID: undefined }

This works "okay" in terms of type-checking, but leads to confusing error-messages:

Type 'string' is not assignable to type 'UUID'.
  Type 'string' is not assignable to type '{ __UUID: void; }'.

The much bigger problem is, these types aren't permitted in maps, because an index signature parameter type cannot be a union type - so this isn't valid:

interface ContentCache {
    [content_uuid: UUID]: string;
}

Is there a better work-around I don't know about? I've tried something like declare type UUID extends string, which isn't permitted, and declare class UUID extends String {}, which isn't permitted as keys in a map either.

Is there a proposal or another feature in the works that will improve on this situation?

[mindplay-dk]

I am struggling hard with this in a model I've been building this past week.

The model is a graph, and nodes have input and output connectors - these connectors are identical in shape, and therefore inevitably can be assigned to each other, which is hugely problematic, since the distinction between inputs and outputs (for example when connecting them) is crucial, despite their identical shapes.

I have tried work-arounds, including a discriminator type: "input" | "output" or distinct type "tags" like __Input: void and __Output: void respectively.

Both approaches leave an unnecessary run-time footprint in the form of an extra property, which will never be used at run-time - it exists solely to satisfy the compiler.

I've also attempt to simply "lie" about the existence of a discriminator or "tag" property to satisfy the compiler, since I'll never look for these at run-time anyway - that works, but it's pretty misleading to someone who doesn't know this codebase, who might think that they can use these properties to implement a run-time type-check, since that's literally what the declaration seem to say.

In addition, I have a ton of different UUID key types in this model, which, presently, I only model as strings, for the same reason - which means there's absolutely no guarantee that I won't accidentally use the wrong kind of key with the wrong collection/map etc.

I really hope there's a plan to address this in the future.

[ethanfrey]

Great issue. Added two thoughts here:

The original usecase from @Aleksey-Bykov sounds a lot like dependent-types, implemented most famously in Idris, where you can define a type "array of n positive integers", and a function that appends two arrays and returns a third array of (n+m) positive integers. Non-empty is a simple case of that. If you like this power, take a look at https://www.idris-lang.org/example/

The simpler usecase from @mindplay-dk is actually what I am struggling with right now. In my example, I have Uint8Array, which may be a PrivateKey or a PublicKey for a cryptographic hash function and I don't want to mix them up. Just like the input and output nodes. This was my solution...

interface IPrivateKey extends Uint8Array {
  readonly assertPrivateKey: undefined;
}

function asPrivate(bin: Uint8Array): IPrivateKey {
  // tslint:disable-next-line:prefer-object-spread
  return Object.assign(bin, { assertPrivateKey: undefined });
}

function signMessage(msg: string, secret: IPrivateKey): Uint8Array {
 // ...
}

Same for public key. When I just did type IPublicKey = Uint8Array, tsc would treat them interchangeably.

I would like to see a bit longer example of how you use the UUID type, but it sounds quite similar to my approach in principle. Maybe there is a better way to deal with this. The Object.assign ugliness bothers me and adds runtime overhead.

[dgreensp]

@ethanfrey How about:

function asPrivate(bin: Uint8Array): IPrivateKey {
  return bin as IPrivateKey
}

[mindplay-dk]

@ethanfrey @dgreensp see comments by @SimonMeskens here - the unique symbol solution is very close to what I was looking for.

[SimonMeskens]

btw @ethanfrey, TS also does dependent types already. I use them a lot, you might want to check out https://github.com/tycho01/typical to see how.

[ethanfrey]

@dgreensp I tried what you said, but tsc complained that Uint8Array didn't have the property assertPrivateKey. The problem with using ghost properties to define types.

@mindplay-dk That solution looks perfect, that I can cleanly cast with as IPrivateKey or as IPublicKey, and the information is carried around throughout tsc (cannot convert one to the other), but has no runtime footprint. Thank you for that link.

I'm still learning typescript, and want to thank you all for being a welcoming community.

[ethanfrey]

@SimonMeskens great link. not just the repo, but its references (rambda and lens) also. I have played with dependent types with LiquidHaskell, and studied a bit of Idris, but it seems to need a symbolic algebra solved many times... Two big examples are tracking the length of an array and tracking the set of items in a container.

I was looking at typical to see how they tracked that and found: https://github.com/tycho01/typical/blob/master/src/array/IncIndexNumbObj.ts commented out.... Am I missing something?

This example seems quite nice. https://github.com/gcanti/typelevel-ts#naturals But it seems they had to predefine all possible numbers to do math: https://github.com/gcanti/typelevel-ts/blob/master/src/index.ts#L66-L77

This also looks interesting https://ranjitjhala.github.io/static/refinement_types_for_typescript.pdf but seems to be a demo project and currently inactive: https://github.com/UCSD-PL/refscript

[dgreensp]

@ethanfrey Interesting, I wonder why my IDE didn't seem to complain. Well, the bottom line is you just need to cast through any:

function asPrivate(bin: Uint8Array): IPrivateKey {
  return bin as any
}

The other example you mention also uses an any-cast, on an entire function signature no less:

const createInputConnector:
    (node: GraphNode) => InputConnector =
    <any>createConnector;

The general principle at work, in both cases, is that the compile-time type need not bear any particular relationship to the runtime type. Given this fact, there is very little constraining what you can do. The fact that you need a "dirty" any-cast to mark something as public/private or input/output is a feature, not a bug, because it means that only your special marker functions can do it.

[ethanfrey]

@dgreensp Ahh... the any cast did the trick.

I was doing

const key : IPrivateKey = Buffer.from("top-secret") as IPrivateKey;

which was complaining. but using any fixed that.

const key : IPrivateKey = Buffer.from("top-secret") as any as IPrivateKey;

[ForbesLindesay]

You can just do:

const key : IPrivateKey = Buffer.from("top-secret") as any;

[jmagaram]

I've been trying to find an elegant solution to the "primitive obsession" problem. I want my interfaces and method arguments to use data types like UniqueID, EmailAddress, and Priority rather than string, string, and number. This would lead to more type safety and better documentation. Tagging seems overly complicated to me. Type aliases ALMOST do what I want. I think I could get 99% of the way there with syntax like the following below. This is just like a type alias except you have to explicitly cast to assign a value to the new type, but implicit casting from the new type to the primitive is just fine.

type EmailAddress < string; // rather than '='

const emailAddressRegExp = new RegExp("[a-z]+@.com");

function newEmailAddress(e: string): EmailAddress {
    if (!emailAddressRegExp.test(e)) {
        throw new TypeError("That isn't a valid email address");
    }
    return new EmailAddress(e.trim()); // cast operator might be fine instead
}

let email1: EmailAddress = newEmailAddress("bob@yahoo.com"); // ok
let email2: EmailAddress = "fail"; // compile error, unsafe cast
let email3 = "banana" as EmailAddress; // compiles ok, forcing the cast
let s: string = email1; // ok implicit cast to wider type always works

== UPDATE ==

I've been using syntax like below and I'm getting more comfortable with it. Pretty simple once you get the hang of intersection types. I still think it would be more obvious if I could just type something like type EmailAddress restricts string.

export type UniqueId = string & { _type: "uniqueId" };
export type Priority = number & { _type: "priority" };

== UPDATE ==

Functionally using intersection types gives me most of what I want with regard to type safety. The biggest hassle with it is that when I mouse-over certain symbols, like function names, to see what arguments they accept I see lots of string & { _type: "personId" } and this is quite ugly and hard to decipher. I really just want to see that a function accepts a PersonId or EmailAddress not some hacked solution to this nominal typing issue.

[ForbesLindesay]

I built https://github.com/ForbesLindesay/opaque-types which uses a transpiler to support opaque and nominal types, with optional runtime validation for the cast operation. It may be of interest to people here.

[qm3ster]

@ForbesLindesay way to go!
What do you think about https://github.com/gcanti/newtype-ts?

[ForbesLindesay]

@qm3ster that looks pretty cool. The iso and prism functions are a neat approach to avoiding runtime cost for multiple opaque types in a single project. The main issue I see with it is that you've used functional terminology, instead of plain english words. You've also tightly coupled your solution to functional programming concepts like Options, which are not commonly used in JavaScript. The other small points are that you have _URI and _A show up as properties on your opaque values, when in fact those properties do not exist at runtime.

The main advantage I see to using my transpiled approach, is that it's very easy to completely change, in the event that new versions of typescript break the current approach. The secondary advantage is that I get to use clean syntax, and ensure that the type name is the same as the name of the object that provides utility functions for cast/extract.

I mainly wanted to prototype how I thought they might behave if added to the language.

[qm3ster]

@ForbesLindesay It's not my project :v I just used it a few times.
I see how the notation might be an impedance to wider adoption.
It's positioning itself as part of a larger gcanti/fp-ts ecosystem, hence the Option type.

[qwerty2501]

@ProdigySim @SimonMeskens Nice solution but it seems to has some cons as below.

• The notation declare const symbolName:unique symbol is long. And therefore, the user should write multi line when define opaque type alias.
• The symbolName in declare const symbolName:unique symbol is necessary not. However the user should define symbolName in user's namespace.

Then I tried improvement. And it seems working.
Please let me know if there are problems.

The definition of Opaque:

interface SourceTag{
    readonly tag:symbol;
}

declare const OpaqueTagSymbol: unique symbol;

declare class OpaqueTag<S extends SourceTag>{
    private [OpaqueTagSymbol]:S;
}

export type Opaque<T,S extends SourceTag> = T & OpaqueTag<S> | OpaqueTag<S>;

usage:

type UserId = Opaque<string,{ readonly tag:unique symbol}>;

type UserId2 = Opaque<string,{ readonly tag:unique symbol}>;

const userId:UserId = 'test' as UserId ;

const userId2:UserId2 = userId; // compile error

The notation Opaque<string,{ readonly tag:unique symbol}> can be written in one line.

[ProdigySim]

I tried out that approach and it's definitely a shorter syntax, but I think most of the time I would not be too worried about one extra line since I will create relatively few opaque types, and I will probably add other boilerplate/helpers in the type's module.

One difference between the two approaches is the error message we get from typescript:

From @SimonMeskens 's setup:

[ts]
Type 'Opaque<string, typeof EmailSymbol>' is not assignable to type 'Opaque<string, typeof UserIdSymbol>'.
  Type 'Opaque<string, unique symbol>' is not assignable to type 'OpaqueTag<unique symbol>'.
    Types of property '[OpaqueTagSymbol]' are incompatible.
      Type 'typeof EmailSymbol' is not assignable to type 'typeof UserIdSymbol'.

From @qwerty2501 's setup:

Type 'Opaque<string, { readonly tag: typeof tag; }>' is not assignable to type 'Opaque<string, { readonly tag: typeof tag; }>'. Two different types with this name exist, but they are unrelated.

I stripped the namespace from both errors to make them more equivalent. The latter is shorter, but the former explicitly calls out Email vs UserId.

[qwerty2501]

@ProdigySim
True. I think it is trade off between "easy to understand error" and "the syntax is shorter".

[ProdigySim]

I checked out how Flow handles opaque types in comparison to our solutions. They have some interesting behavior.

Notably:

1. Opaque types are treated differently in the file they're created in. Implicit conversions from underlying-type to Opaque Type are allowed in the same file the type is created in.
2. The Opaque Types behave similarly to @SimonMeskens 's original solution (string & { [TagSym]: typeof UserIdSymbol }). That is, implicit conversions TO the underlying type are allowed.
3. Opaque Types can be used as index types. e.g. { [idx: UserId]: any } is a valid type. This is not currently possible in typescript afaict.

I put together a typescript playground link demonstrating different constructions of Opaque<T> and their capabilities.

1. "Super Opaque": OpaqueTag<S>-- no reference to underlying type, no valid casts to an underlying type.
2. "Weak Opaque": T & OpaqueTag<S> -- matches flow behavior closely, automatic downcasts to T
3. "Strong Opaque": T & OpqaueTag<S> | OpaqueTag<S> -- keeps some reference to underlying type for explicit conversions, but doesn't allow implicit casting.

I think each could have uses; but a first-party Typescript solution could definitely allow the best of all worlds here.

[MicahZoltu]

For others coming across this, a fairly succinct solution that results in short but readable error messages can be found over here. Comes with caveats, since it is setup to ignore a compiler warning, but so far I like the UX of it the most out of all of the options I have seen so far. Need to test it cross-module still though.

[StephanSchmidt]

Wrote a small tag type library based on some ideas here. Will change when TS gets nominal types.

https://github.com/StephanSchmidt/taghiro

Happy for feedback.

[qwerty2501]

I guess, it is necessary for typescript to support official opaque type alias.
Because both(@ProdigySim and me) solutions has the number of writing characters bit too much.

[mtaran-google]

FYI: An issue with the solutions in #4895 (comment) and #4895 (comment) is that if your build process involves interfacing between different modules using .d.ts files from tsc -d, they'll need to be adjusted to make the private members public. Otherwise, the type of the private member gets stripped, making the helpers lose their intended effect.

[thehappycoder]

@StephanSchmidt I am using taghiro. It's very easy to use and 9t works well so far!

[mindplay-dk]

We have talked about this one a few times recently. @weswigham volunteered to write up a proposal.

@mhegazy @weswigham did this ever happen?

[weswigham]

There are two proposals in the form of experimental implementations in PRs.

[beeplin]

@weswigham sorry to raise this issue again. Is there is roadmap for this?

[shicks]

I had a recent need for a tag type that none of the existing workarounds can solve. I rebased @weswigham's #33290 (at shicks/TypeScript:structural-tag) and found that it works more or less perfectly.

To summarize this solution, it introduces a new type operator, tag, that creates a new type tag T from any existing type T, such that tag T <: tag U only if T <: U, but otherwise tag T behaves essentially like unknown in terms of inference and other capabilities. I believe this is the ideal solution to this issue for a handful of reasons:

Branding without the lies

Type branding is a very common practice, seen in libraries, numerous blogs, and even in the TypeScript codebase itself. But to this day, the standard approach (workaround, rather) is to lie to the type checker by adding fake properties:

type Brand = {brand: void};
type BrandedString = string&Brand;

This may be "free" at runtime, but you can get into some trouble while type checking, since this brand property doesn't actually exist. Additionally, if a primitive type (such as string) isn't intersected with the brand, the checker assumes it's an object, which can lead to wildly inaccurate narrowing. The tag T type makes no such promises.

Reimagining some code from the TS codebase:

type Branded<T, B> = T & tag {[K in B]: void};
export interface ClassElement extends Branded<NamedDeclaration, 'ClassElement'> {
  readonly name?: PropertyName;
}

Inherently structural, but nominal-friendly

These tag types are structural, and behave consistently to all the existing structural types (unlike the unique type proposal in #33038): you can even reflect into them via conditional types (T extends tag infer U ? ...). This makes them a very natural addition to the language.

That said, tag presents a relatively clean emergent solution to nominal typing (#202) as well, by simply combining with unique symbol types: tag {readonly _: unique symbol} generates a guaranteed-unique type everywhere it shows up in source. Nominal typing falls out more or less automatically when you either intersect with such a type, or add a brand-typed property to an interface.

Introducing a type operator/primitive solely for the purpose of nominal typing would be going against the grain (hence the repeated complaint on #202 and elsewhere that TypeScript is structural, not nominal), but this would provide the necessary tools for those who want nominal typing to achieve it, while still remaining essentially structural. For those who want to ensure that multiple different versions of their API are compatible, they can use structural tags with fixed strings. For those who want to guarantee that nothing outside their module is assignable to their nominal type, they can use unique symbols to get that guarantee.

True opacity

In addition to allowing authors to avoid lies and enabling brands and nominal types, this solution introduces a new type with important capabilities that's currently impossible to express. As mentioned above, the current status quo for brands is to use an object literal type (e.g. {brand: void}). If this type is intersected with a primitive, the type checker will assume its typeof is that primitive, but if not, the checker will (potentially erroneously) assume it's object. This makes it impossible to accurately use these faux brands to refer to types whose actual runtime representation (e.g. primitive vs object) is truly opaque. In fact, the only type that currently prevents narrowing by typeof is unknown, but unknown will clobber any other type in a union (since unknown|T === unknown), and provides no actual safety.

The beauty of tag T is that it prevents typeof narrowing (like unknown), but (unlike unknown) it's still preserved in union types. This gives it a unique capability that no type currently expressible in TypeScript can do. To wit, adapting the "impossible" example from above:

type SafeBigint = tag {readonly __safeBigint: unique symbol};
declare const x: string|SafeBigint;
if (typeof x === 'string') {
  use(x);
  //  ^? const x: string | (string & SafeBigint)
} else {
  use(x);
  //  ^? const x: SafeBigint
}

Because the tag type is completely opaque, the narrowing recognizes that it could show up in either branch, and thus avoids potentially-incorrect usage of the SafeBigint wrapper type. And because it's not a supertype of all other types, it doesn't clobber unions. It's also impossible to cast directly to/from (without casting through unknown or any), which can provide good type safety for API design.

---

Final words

I'm asking that the TypeScript team reconsider the tag T approach as a solution to both tagged/branded types (this issue) and nominal types (#202). The closed PR #33290 or my rebase of it demonstrates clear utility. There are no backwards-compatibility risks. And I'd argue that it's the best solution to nominal typing that stays true to TypeScript's structural roots. It allows using it in flexible, emergent, ways that lead to better, sounder, types.

Thank you for your consideration.

[Pyrolistical]

Another library example is zod .brand