Topic/fix var diff string

[LFSaw]

interface to Steno requires prevTokens to be of type [Char].

[telephon]

on the long run, it would probably better to keep them in the same format instead of converting them fore and back. Would this be too much of a change?

[telephon]

The thing is that in Steno, the tokens can also be words, not only chars. So the assumption that this is a string is problematic in the VarDiffString class.

[LFSaw]

I think you're right but I currently do not have the headspace to dive into this fairly complex code... In the long run, I'd prefer to not use the CLI diff at all but use Ron's versions (probably diff2 as a default...)

[LFSaw]

The thing is that in Steno, the tokens can also be words, not only chars. So the assumption that this is a string is problematic in the VarDiffString class.

Curious: How does one operate on non-char tokens with the Steno interface (t.("hello"))?

[telephon]

You define a different preProcessor

t.preProcessor = nil; // remove default shortcuts
t.diff.tokenizer = { |str| str.split($ ) };

// then you can write

t.value("hello word word");

There is an example under preProcessor in the Steno helpfile.