ProcessBatch "token recognition error"

[cosminioan]

hi Michael,
wonderful script/platform -- awesome job!!!

I ran into a possible issue, it seems trying to execute the command line for SSAS on prem and I get the following errors, even though in the end, the command finishes fine, after some time, with a confirmation:

Tabular Editor 2.15.3 (build 2.15.7767.23827)

Loading custom actions from: C:\Users\xxx\CustomActions.json
line 316:73 token recognition error at: '?'
line 329:78 token recognition error at: '?'
line 353:87 token recognition error at: '?'
line 358:67 token recognition error at: '?'
line 439:106 token recognition error at: '?'
line 506:95 token recognition error at: '?'
line 511:99 token recognition error at: '?'
line 516:71 token recognition error at: '?'
line 1334:71 token recognition error at: '?'
line 1344:85 token recognition error at: '?'
line 1349:76 token recognition error at: '?'
line 1370:65 token recognition error at: '?'
line 1408:93 token recognition error at: '?'
line 1413:95 token recognition error at: '?'
line 1418:106 token recognition error at: '?'
line 1666:67 token recognition error at: '?'
line 1778:69 token recognition error at: '?'
line 1806:70 token recognition error at: '?'
line 1858:89 token recognition error at: '?'
line 1886:69 token recognition error at: '?'
line 1895:67 token recognition error at: '?'
line 1928:85 token recognition error at: '?'
line 316:73 token recognition error at: '?'
line 329:78 token recognition error at: '?'
line 353:87 token recognition error at: '?'
line 358:67 token recognition error at: '?'
line 439:106 token recognition error at: '?'
line 506:95 token recognition error at: '?'
line 511:99 token recognition error at: '?'
line 516:71 token recognition error at: '?'
line 1334:71 token recognition error at: '?'
line 1344:85 token recognition error at: '?'
line 1349:76 token recognition error at: '?'
line 1370:65 token recognition error at: '?'
line 1408:93 token recognition error at: '?'
line 1413:95 token recognition error at: '?'
line 1418:106 token recognition error at: '?'
line 1666:67 token recognition error at: '?'
line 1778:69 token recognition error at: '?'
line 1806:70 token recognition error at: '?'
line 1858:89 token recognition error at: '?'
line 1886:69 token recognition error at: '?'
line 1895:67 token recognition error at: '?'
line 1928:85 token recognition error at: '?'
Loading model...
Loaded script: C:\YYY\ProcessBatches.cs
Executing script 0...
Processing type 'calculate' of the 'CubeName' model has finished in 8 seconds.

any thoughts on what might be going on?
thank you --

[m-kovalsky]

I believe the issue is with one of your saved custom actions. The issues show before the model is even loaded. Plus, the Processing Manager scripts do not have 1900 lines of code. See if you have a very long custom action (1900+ lines of code) and remove it (for now). The custom action may need to be adjusted slightly.

[cosminioan]

hi Michael, thanks for the reply. I thought so at first, so I removed all of the custom actions. Then I just added the ProcessingManager by using the "open" dialog, instead of copy/paste, thinking maybe that the windows clipboard copy/paste may create formatting problems. But still similar issues. So, with a brand new (and only custom script), I am getting the following:

Tabular Editor 2.15.3 (build 2.15.7767.23827)

Loading custom actions from: C:\Users\XXX\CustomActions.json
line 532:71 token recognition error at: '?'
line 542:85 token recognition error at: '?'
line 547:76 token recognition error at: '?'
line 568:65 token recognition error at: '?'
line 606:93 token recognition error at: '?'
line 611:95 token recognition error at: '?'
line 616:106 token recognition error at: '?'
line 864:67 token recognition error at: '?'
line 976:69 token recognition error at: '?'
line 1004:70 token recognition error at: '?'
line 1056:89 token recognition error at: '?'
line 1084:69 token recognition error at: '?'
line 1093:67 token recognition error at: '?'
line 1126:85 token recognition error at: '?'
line 532:71 token recognition error at: '?'
line 542:85 token recognition error at: '?'
line 547:76 token recognition error at: '?'
line 568:65 token recognition error at: '?'
line 606:93 token recognition error at: '?'
line 611:95 token recognition error at: '?'
line 616:106 token recognition error at: '?'
line 864:67 token recognition error at: '?'
line 976:69 token recognition error at: '?'
line 1004:70 token recognition error at: '?'
line 1056:89 token recognition error at: '?'
line 1084:69 token recognition error at: '?'
line 1093:67 token recognition error at: '?'
line 1126:85 token recognition error at: '?'
Loading model...
Loaded script: C:\Custom path with spaces\ProcessBatches.cs
Executing script 0...
Processing type 'calculate' of the 'CubeName' model has finished in 8 seconds.

It turns out, the execution does not like the path with spaces, for ProcessBatches.cs, even though it is enclosed with double-quotes in the command line invocation, as originally envisioned. Once I put ProcessBatch in a folder without spaces in the name, all worked out like a charm. Problem solved so I thought I'd chime in with my findings.

many thanks again for your wonderful work... and support :-)
Cos

[m-kovalsky]

I found the issue. It has to do with how Tabular Editor records certain syntax when it saves a Custom Action. If you go to the CustomActions.json file in the AppData\Local\TabularEditor folder on your computer, you can see that at a certain point the 'Execute' line shows as red (in Notepad ++). It's due to the syntax in the processing manager script that has "raw.githubuser...". Need to open a bug in Tabular Editor for this one.

[cosminioan]

awesome -- thanks for the investigation Michael, and forwarding to Daniel. you guys rock!!!

[m-kovalsky]

Thanks to Daniel's advice, this issue has been resolved in Release Version 1.1.1

https://github.com/m-kovalsky/ProcessingManager/releases/tag/1.1.1

[cosminioan]

awesome!!! great work and even greater/faster feedback!!! kudos to all involved!

[cosminioan]

removed the ProcessManager.cs, installed the new version and still get errors, although this time a bit less -- interesting why showing twice, after line 810:

line 610:73 token recognition error at: '?'
line 623:78 token recognition error at: '?'
line 647:87 token recognition error at: '?'
line 652:67 token recognition error at: '?'
line 733:106 token recognition error at: '?'
line 800:95 token recognition error at: '?'
line 805:99 token recognition error at: '?'
line 810:71 token recognition error at: '?'
line 610:73 token recognition error at: '?'
line 623:78 token recognition error at: '?'
line 647:87 token recognition error at: '?'
line 652:67 token recognition error at: '?'
line 733:106 token recognition error at: '?'
line 800:95 token recognition error at: '?'
line 805:99 token recognition error at: '?'
line 810:71 token recognition error at: '?'

[m-kovalsky]

Please try deleting your custom actions file and then adding the new script as a custom action. All the zero-width spaces were removed from the file (nothing left to remove). I tried this on my machine and did not get any token recognition errors after loading the latest script as a custom action and processing a batch.

[cosminioan]

yep, that did it (removing the entire file). I guess removing all custom actions individually and re-adding ProcessManager did not help -- but removing the file and re-adding ProcessingManager did -- so probably some zero-length width spaces were still lurking around...
many thanks, Michael -- issue closed.