Possible bug with parsing Token Studio file

[mikestecker]

I have loaded up the example Token Studio file that is included in the plugin (also on their repo here), however I'm getting some errors, which I'm guessing is likely due to the way the object is nested? The error is scale: missing $type

Is this a bug?

[drwpow]

Ah hm. The auto-detection of Tokens Studio format kicks in whenever there’s a $theme and $metadata top-level entry in the JSON file. Because that was in the exported files I tested, and assumed it would always be there. But that may not be the case!

There might need to be some other way to detect a Tokens Studio format file from DTFM, because it’s still important that DTFM errors are validated and thrown (which is the error you’re seeing now—it’s trying to say it’s invalid DTFM).

So ideally we have one of the following solutions:

1. Find some other way to auto-detect a Tokens Studio file (which, given your example, nothing else jumps out to me as a thing that only a Tokens Studio file would have and not, say, Style Dictionary)
2. Add a tokens.config.js setting, e.g. format: 'tokensStudio'
3. Add a CLI flag, e.g. --format=tokensStudio (or perhaps in tandem with Update core to Rust #2)

[mikestecker]

I checked my file for $metadata but was unable to find it, so I opened Token Studio and did a full export from their example file which added it. However, now I'm getting other errors saying that it's looking for other values that don't exist in the tokens.json file such as core.spacing.multi-valueRight: can't find {dimension.xl}. I put the file up in a gist if you want to test.

[mikestecker]

But I still think you should update with one of the proposed solutions to force reading the json as a token studio file. I noticed that if you don't check the "All token sets" option on export, it doesn't export the $metadata entry.

[drwpow]

That’s great insight, thank you! Admittedly, I don’t have access to a really robust Tokens Studio project; I’ve just been testing it on more “examples.” And sorry you had to run into a rough edge, but that’s helpful info not only for docs, but parsing as well.

After thinking about it a bit, I’m still not sure how I want to solve the syntax warning here. Perhaps the error message should change to something along the lines of:

[error message]

Parsing as DTCG format. If this is in a different format, configure the `format` option in `tokens.config.mjs` [link]

(obviously assuming a format option is added to the config, which doesn’t exist yet)

There’s another entire possibility I’m considering where Cobalt has more configurable/pluggable parsers, but that would be a different direction for this library, and is beside the point of this issue. Bare-minimum, I am interested in supporting Tokens Studio well in the core library. And short-term am committed to the DTCG format. But will keep an open mind when it comes to how people are using/consuming design tokens as more and more people jump on this bandwagon.

[drwpow]

Improved parsing of Tokens Studio files, but I’m still a bit hesitant to add the format option to the CLI, especially after thinking about upcoming changes for 2.0 (#201). Ideally, I would like to just have Tokens Studio standardize on a consistent output (codify their specification) and have Cobalt just parse it correctly, automatically. I think opening up the can of worms to just have Cobalt parse anything is getting outside the bounds of what I would like to set for this project (Tokens Studio basically just gets “special” treatment because I’m a fan of their work, they are a widely-used tool, and are active contributors to the DTCG spec even if their output format doesn’t adhere strictly to it).