JS: Backport³ and more additions & fixes

[Willy-JL]

What's new

• backport (here) of backported missing things (there) of great backport & rework (over yonder) xD
• i separated into individual commits that make sense on their own, might be easier to look there instead of whole diff
• original authors/contributors credited with co-author where relevant
• badusb:
  • numpad keys support
  • keyboard layout support
  • altPrint() and altPrintln() support
  • quit() function to unlock usb profile (original intent was to alternate badusb profile and mass storage profile, we have this in momentum and other cfw as usbdisk module, but outside scope of this pr, can backport this module later on if you're interested)
• serial
  • readAny() to avoid starving read loops with many small reads or hinder usability with few large reads
  • end() to release serial handle allowing new initialization in same script, maybe for gpio usage or different serial port or settings
  • expansion service is automatically disabled to avoid conflict
• storage
  • added simple example script we had previously, ported to new remade storage module
• gui:
  • text_input:
    • fixed null ptr crash when user did not pass max len prop. logic of props is from what i can see, always configure view for minimum usable state, props should configure things on top as optional
    • defaultText and defaultTextClear props
  • byte_input
  • file_picker
    • there is no file picker based on View object, so this is only gui/* module that is not based on view_factory
    • instead it uses usual dialogs app file picker in sync manner
  • viewDispatcher.currentView to allow users to have extra logic in navigation callbacks for example
  • updated gui example script with showcase of all additions above
• globals:
  • toString() works correctly on negative numbers, toString(-42) used to give "0"
  • parseInt()
  • toUpperCase() and toLowerCase()
  • example for string functions
  • __filepath and __dirpath to allow relative path contextualization, for example for load() or storage.open()
• example interactive.js repl script, allows running js in limited capacity without editing files
• many additions to typedefs that were missing or incomplete, still not everything is documented i believe but getting closer

Needs feedback:

• toString(), toUpperCase(), toLowerCase() could all be moved to respective classes i think, for strings there is getprop_builtin_string() in mjs source, can add there, for number there is no such function but may be possible to add another if branch in getprop_builtin() for number type
• __dirpath and __filepath are set when launching script in global scope, but means that when executing scripts with load(), these have values of original script, since scope lookup is recursive (see mjs_find_scope() and its usages). i think better behavior would be load() gives child script its own correct __dirpath and __filepath but there are some issues:
  • load() accepts a custom scope parameter. with custom scope, assignment in child script is confined to custom scope, lookup is recursive to parent scopes. with no custom scope, code is executed in current scope, so everything is shared like code was inlined
  • for custom scope, we can get custom scope object passed by user and inject __filepath and __dirpath into it, easy
  • for no custom scope, i dont see any good solution: one way is saving current __dirpath and __filepath values from scope, changing them, running load(), then resetting to previous values. might work fine in most cases but if child script runs load() of its own and uses callbacks or things like this, child2 might run a function defined in child1 and then its screwed up. this is a rare edge case of course, if no function callback is passed from child1 to child2 then no code from child1 could run while child2, and second instance of mjs_load() would restore child1's __dirpath and __filepath correctly

Verification

• unit tests
• example scripts

Checklist (For Reviewer)

• PR has description of feature/bug or link to Confluence/Jira task
• Description contains actions to verify feature/bugfix
• I've built this code, uploaded it to the device and verified feature/bugfix

[portasynthinca3]

Additional comments, in order of decreasing importance:

• I don't like how with the max length thing the fields have to be arranged in a specific way for the thing to work. Field ordering is absolutely not something that the ECMAScript specification guarantees. This is especially important with the new build pipeline underway.
• Not an issue with this particular PR, just a thing to note in regards to [FL-3918] Full-fledged JS SDK + npm packages #3963: .d.ts files would have to be moved, and the new declarations would have to get @version doc comments.
• I don't like that the file browser is not a view. But that's an issue for later, I don't think that this PR needs to change this.

LGTM otherwise

[Willy-JL]

• I don't like how with the max length thing the fields have to be arranged in a specific way for the thing to work. Field ordering is absolutely not something that the ECMAScript specification guarantees. This is especially important with the new build pipeline underway.

mJS does, properties are parsed from code top to bottom and stored in a linked list inserting at the head, so at the end of object parsing the properties are in exact inverse order from what one would expect. inversing the processing order of props in makeWith() also doesn't seem smart, would be so time inefficient due to it being a singly linked list from what i can see. but yes, not ideal to rely on this behavior, and also it is counterintuitive to the user.

i see 2 possible solutions:

• changing the examples to show view.set("defaultText", "text"); after the first makeWith() call
• changing behavior of defaultText and defaultData assign to resize the buffer to atleast the size specified by the default provided. for example, default is 4 bytes for byteinput, default data is 10 long so enlarge to 10, then we also have length prop provided, so enlarge or shrink as requested. this brings up another problem however: what if the user provides a defaultdata longer than the length they specify? if length is assigned first, then it will be resized later by default data. if defaultdata is assigned first, then it will be shrunk later by length. and we cannot solve this by enforcing the size specified by length, because thats the original issue: if defaultdata is assigned before than user provided length, then length is still default of 4 and default data is cutoff despire the user specifying otherwise.

so really there is no solution. this is a chicken and egg problem. best compromise to me soudns liek resizing to atelast size specified by default data, and if user also provides a length that is incompatible with this, it is their issue to deal with order of processing

• Not an issue with this particular PR, just a thing to note in regards to [FL-3918 Full-fledged JS SDK + npm packages [FL-3918] Full-fledged JS SDK + npm packages #3963]([FL-3918] Full-fledged JS SDK + npm packages #3963): .d.ts files would have to be moved, and the new declarations would have to get @version doc comments.

if the intention is to merge that pr before this one then ill make those changes as required

[portasynthinca3]

@Willy-JL

Field ordering is absolutely not something that the ECMAScript specification guarantees

mJS does

We prefer to rely on specifications and not implementation details :) Like I said, gonna become very important with our new tools that are going to be processing JS code before mJS even sees it.

With regards to the solution, I think we can just keep the buffer the maximum size out of all sizes that we know of (user-requested max length, user-provided default text, etc.) and explicitly throw an error if the requested max length is smaller than the length of the provided default text.

if the intention is to merge that pr before this one then ill make those changes as required

We'll see how it goes :) I don't know which one will get merged first, that depends on Aku.

[Willy-JL]

ah well, that sounds good enough. ill change it again to track default data length separately and error if length is specified less than that

[Willy-JL]

one thing im not sure of: string returned by mjs to c is not guaranteed to be null terminated. props pass just a const char* with no length value. this sounds like a problem waiting to happen to me. like will my strlen() for default text overflow? idk probably? lol

[Willy-JL]

@portasynthinca3 i dont have my flipper with me right now for another few hours, logic should be ok but i cannot test right now.

only thing that remains is my concern with strings passed as props possibly not being nul terminated... will this be an issue?

[portasynthinca3]

one thing im not sure of: string returned by mjs to c is not guaranteed to be null terminated. props pass just a const char* with no length value. this sounds like a problem waiting to happen to me. like will my strlen() for default text overflow? idk probably? lol

@Willy-JL This is a valid concern based on the doc comment. However, after reviewing the actual implementation of mjs_get_string and some parts of the mJS' memory manager, I can't see a single code path where a string wouldn't get null terminated:

• MJS_TAG_STRING_I and MJS_TAG_STRING_5: embedded in the lower 5 bytes of term. The tag is stored in the upper 2 bytes, leaving a byte in the middle which will be 0 b/c terms are zero-initialized, at least in mjs_mk_string.
• MJS_TAG_STRING_O: GC'ed owned strings, which are always stored with the terminator at runtime. The parser does store them without the terminator in the same giant buffer with all the other strings, but we never use raw internally referenced strings from the parser. The bytecode interpreter does, but it converts them into nice null-terminated strings with mjs_mk_string for us.
• MJS_TAG_STRING_F: foreign strings, not managed by mJS at all. The main use case is to avoid wasting RAM for strings that you already have in .rodata. One could sneak a non-null-terminated string into mjs_mk_string(/*...*/, false), but I'd just call that irresponsible, honestly. Either way, nothing in our codebase does that; not us, not mJS itself.
• MJS_TAG_STRING_C: seems to be used internally during garbage collection, never seen by us.
• MJS_TAG_STRING_D: all uses are commented out, the symbols in the commented out code are no longer there, I can't tell.

Perhaps we're dealing with outdated documentation here. I personally have never gotten a string from mJS that isn't null terminated, and I'm convinced that my analysis shows that this can never happen. We might want to ask the mJS team about this, but their repo seems dead.

P.S.: Holy cow, is mJS code ever hard to read ;_;

[Willy-JL]

Sounds great, thanks for the very detailed break down!
Then we should be good on this pr? I can test the changes I added in about an hour when I have my flipper nearby.

[portasynthinca3]

LGTM, assuming the functionality works as intended.

[Willy-JL]

just tested on device and seems to work great!

[Willy-JL]

@skotopes fixed, works fine now (and i learned my lesson to actually flash what i code, not that but ported to another fork xD)

also i mirrored the JS SDK changes porta made but for the additions of this PR, hope it looks fine

[skotopes]

;-)
I have more questions to @portasynthinca3, since she promised to review this PR

[skotopes]

Good job everyone!
Really awesome to see JS becoming more and more usable