Skip to content

Releases: Moddable-OpenSource/moddable

May 4, 2021

04 May 22:45
Compare
Choose a tag to compare

This release contains major updates to the XS JavaScript engine. After updating, please be sure to do a full, clean build of both the Moddable SDK tools and all your projects. Details are in the Update sections of the Getting Started guide for macOS, Windows, and Linux.

There is a new branch for on-going work to support ESP-IDF v4.3. The port is already stable and usable, with all major features confirmed to be working. This branch will updated as needed while ESP-IDF v4.3 is still pre-release. When it is final, we expect to switch over from ESP-ESP v4.2.

  • XS
    • JavaScript global variables defined during preload are now stored in ROM (Flash) until modified
      • Saves at least 1300 bytes per virtual machine, up to 2100 bytes for projects using Piu
      • Reduces memory required for a new Secure ECMAScript compartment to 624 bytes
    • Changes to XS IDs
      • Default 16-bit ID values now use all 16-bits (previously limited to 15-bits) allowing for up to 65535 unique property names
      • Option to use 32-bit ID values (#define mx32bitID 1) for non-embedded builds
      • xsGet now only works for property ID values, not indices. To get by index use xsGetIndex. Similar changes to xsSet and xsHas.
    • Fixes
      • Fix memory allocator bug that unintentionally caused buffers to be allocated at minimal size, leading to thrashing. Introduced recently with changes to detect overflow in buffer size calculations. #637 (reported by @warner at Agoric)
      • Symbol.for('bar').toString() did not include the symbol name. #621 (reported by @dckc at Agoric)
      • Fix injection attack in Function constructor #623 (reported by @dckc at Agoric)
      • Fix garbage collector interaction when invoking hasProperty in the run loop #627 (reported by @dckc at Agoric)
  • ECMAScript® Embedded Systems API Specification
    • Updates to io.md to synchronize with final draft of ECMAScript® Embedded Systems API Specification (Ecma TC53) (requested by @stc1988)
    • Fix build conflicts with IO modules and Pins modules
  • Graphics: Commodetto and Piu
    • New Piu GIF Image object to directly embed animated GIF in Piu screen
    • Animated GIF decoding directly to black and white (1-bit), gray scale (4-bit), and 256 index color to reduce memory required for back-buffer when possible
    • Add indexed 256 color bitmap format to Commodetto to support animated GIF
    • In Piu Port, fix obscure potential problems with garbage collection timing
    • Change function name from convert to process in Commodetto pixel format conversion document. #637 (reported by @stc1988)
  • PWM on ESP32
    • Supports manifest defines for the off value and frequency
    • Checks error on pwm.write and explicitly sets additional configuration struct members
  • Crypto and TLS
    • Optimize Montgomery and Jacobian functions.
    • Reduce memory allocations in cryptographic functions
    • Optimize fxBigInt_uadd_prim using GCC built-ins
    • TLS client implementation now always replies to CERTIFICATE request, even if the response is empty. Fixes compatibility with servers build with Java 10.
  • Optimizations to BLE implementation focused on reducing how often garbage collector runs

April 6, 2021

06 Apr 20:21
Compare
Choose a tag to compare
  • XS
    • Error Stack now captured at time of creation rather than when used with throw to be more consistent with browser engines.
    • Marshalling objects between virtual machines now supports most built-ins including instances of TypedArray, RegExp, Map, Set, Proxy, DataView and Date. Objects with cyclic references, references to objects in ROM, and private fields may also be marshalled between virtual machines cloned from the same image. Details in new XS Marshalling document.
    • WeakMap no longer corrupted during garbage collection. #592 #608 (report by @dckc at Agoric)
    • Fix native stack overflows in parser. #586 and #587 (reported by @rain6851)
    • Fix invalid access on exception when reading uninitialized variable. #585 (reported by @rain6851)
    • Fix string buffer overflow in fxIDToString. #583 (reported by @rain6851)
    • Detect chunk math size overflow. #580 #581 #582 #587 (reported by @dckc and @rain6851)
    • Gathering of a function's rest arguments to an Array no longer invalidates the byte-code pointer (if garbage collector runs at exactly the wrong moment). #604 (reported and isolated by @dckc at Agoric)
    • Add mxNoChunks debug build option to use malloc to allocate chunks instead of the XS internal chunk allocator. This gives more visibility to Address Sanitizer into memory accesses by XS.
    • Issues with RegExp flags and invalid JSON characters detected by Address Sanitizer fixed
    • xsl (linker) only strips runProgramEnvironment when eval is stripped. Fixes REPL example.
  • Implementation of ECMAScript® Embedded Systems API Specification (Ecma TC53)
    • Implemented SPI controller on ESP8266 and ESP32. Includes ILI9341 display and XPT2046 touch examples.
    • GPIO 16 now works with Digital and DigitalBank on ESP8266
  • Graphics and UI
    • Animated GIF decoding is now supported in Commodetto. The implementation builds on the GIF Animator core by Larry Bank. See the example and documentation for details. Usable now with more improvements planned.
    • Piu Texture constructor accepts color and alpha bitmap arguments to allow bitmaps created outside Piu as textures
    • Color cell encoded images no longer crash when clipped on the left edge
  • Devices
    • Backlight on Moddable Two can be configured to be initially off to eliminate flicker at start-up
    • TTGO LilyGo ST7789 1.14" LED Screen display support. #576, #601 (requested by @louisvangeldrop)
    • Microphone support for M5StackCore2 and M5StickC (contributed by @stc1988)
    • Espressif hosts use c_read* macros in place of direct espRead* function calls allowing direct reads (faster) on ESP32.
    • Some large chunk allocations on embedded targets now succeed because chunk allocator includes free space in current chunk heap when determining if there is enough free space
  • Examples
    • Remove stray character from CLI. #597 (reported by @dbhaig)
    • Instrumentation examples now explicitly include the JavaScript instrumentation module (which is not always built-in). #598 (reported by @dbhaig)
    • MQTTConnection module now supports changing the client ID after instantiation because MAC address may not be available at instantiation. (reported by @Frida854)
  • Tools
    • When building tools on Linux, explicitly include pthread library
    • mcconfig merges release SDKCONFIG fragment on ESP32 build rather than replacing
  • Modules
    • Add Modules module to synchronously check for presence of modules, synchronously load modules, and enumerate installed modules in host and mods. Replaces LoadMod used in IoT Development for ESP32 and ESP8266 with JavaScript examples.
    • Add manifest for microseconds module
    • GCM mode in TLS reduces peak memory by re-using buffers like CDC mode
  • Documentation
    • Getting Started documentation updates based on user feedback, including how to create shell start-up file

March 2, 2021

02 Mar 23:56
Compare
Choose a tag to compare
  • New modules
  • XS
    • BigInt values supported as keys for Maps and Sets #577 (reported by @dckc at Agoric)
    • First attempt at Error Stacks (subset of the Stage 1 proposal)
    • xsbug now reports exception when a Promise rejection is unhandled
    • Module specifiers may now have a namespace prefix (e.g. "embedded:io/serial")
    • Fix keywords as property names in object binding variable statements #565 (reported by @dckc at Agoric)
    • Fix Object.prototype.hasOwnProperty when called with no arguments (also fixed similar unreported issue with several other methods of Object - __defineGetter__, __defineSetter__, __lookupGetter__, __lookupSetter__, propertyIsEnumerable) #556 (reported by @YaoHouyou)
    • Unsafe optimizations enabled with mxBoundsCheck set to 0 (OFF) handled consistently. Note that mxBoundsCheck defaults to OFF in release builds. Projects running untrusted code should set turn mxBoundsCheck on.
  • Implementation of ECMAScript® Embedded Systems API Specification (Ecma TC53)
    • Implement banks to support more than 32 pins
    • Implement IO for ESP32: DigitalBank, Analog, PWM, I2C, Serial, UDP, TCP, Listener. (works with J5e!)
    • Switch to "embedded:" module namespace from original "builtin:" placeholder
    • Update examples to match latest specification draft and to use Host Provider Instance
    • Add I2C example driver for AMG8833 Grid-EYE infrared camera
  • Fixes and improvements
    • Fix Piu sound on devices without a Host global. #558 (reported by @stc1988)
    • Fix Poco color cell crash when rendering clipped narrow widths using drawFrame
    • Piu Timeline object can again be built independently of Piu #559 (reported by @zombeej)
    • Analog audio output now works on M5Stack with ESP-IDF 4.2. ESP-IDF changes broke this by reversing the channels. #560 (reported by @stc1988)
    • Fix race condition when closing audio output on ESP-IDF 4.2 Caused static to be placed on some M5 devices #560
    • Fix exception in auto-rotate for M5Stack Fire and Core2
    • Fix ESP32 build issue on Windows when building projects on a different drive than the drive that holds the ESP-IDF
    • Remove unneeded dependency on display driver in pngdisplay and image-frames examples #571 (reported by @dbhaig)
    • Eliminate warning when building audio input on ESP32 #561
    • Implement display brightness control for M5Stack Core2 (contributed by @stc1988)

February 3, 2021

04 Feb 03:08
Compare
Choose a tag to compare

This focus of this release is updating the Moddable SDK to use ESP-IDF version 4.2. Developers using the Moddable SDK with ESP32 will need to update. Links to instructions are provided below.

  • ESP-IDF version 4.2
    • ESP32 builds now require ESP-IDF version 4.2, upgrading from version 3.3.2.
    • Instructions for updating the ESP-IDF are available for macOS, Windows, and Linux.
    • Additional information about the update is available in our blog post.
    • Higher speed SPI output on all ESP32 MCUs
    • Support for ESP32-S2 chips
    • Support for Saola development boards from Espressif
    • Support for the Kaluga development board from Espressif. This includes support for high-speed rendering to the display, audio playback, touch panel button, physical buttons, and the color NeoPixel LED.
    • Support for external PSRAM on ESP32-S2 which significantly increases memory available to JavaScript
  • ESP32 runtime
    • Fix misinterpretation of servo parameters on ESP32. #541 (reported by @MKGaru)
    • ESP32 PWM implementation now allows PWM to be completely disabled and automatically stops when closed
    • ESP32 preferences now implements Preference.keys. #368 (contributed by @wilberforce)
  • Build
    • Windows build works with lowercase drive letters
    • Added clean option to Windows build #546
    • mcrun on Windows now respects UPLOAD_PORT environment variable
    • mcrun now automatically uses correct baud-rate for target device
    • ESP32 build now puts all generated sdkconfig files into build results
  • XS
    • Language conformance document updated with latest results from test262
    • Show contents of function closures and promise status in xsbug as shown in this screen capture. #127 (Suggested by @bmeck)
    • Improve debugging with tail call optimization by checking for valid function early (reported by @Agoric)
    • Fixed bug where variables declared with let behaved as if declared with const under obscure circumstances. #540 (reported by @MKGaru)
    • Removed obsolete fxMarkHost and fxSweepHost from documentation
    • Strip feature of the XS linker now works correctly on Windows
  • TypeScript
    • Update declaration files to export previously internal-only types
    • Set forceConsistentCasingFileNames to avoid case-sensitive issues in the future
    • Fix property name in Monitor constructor. (contributed by @meganetaaan)
  • Examples
  • Documentation

December 30, 2020

31 Dec 00:05
Compare
Choose a tag to compare

Welcome to the final Moddable SDK release of 2020.

  • XS JavaScript engine
    • Debugging support for source text parsed by eval! Watch the demo. It is still not recommended to use eval in embedded projects. (requested by @Agoric)
    • Remove unused historic features thanks to a very through review of runtime properties by @erights (see #523 and #524 for details)
    • Fix dead-strip bug introduced by removing historic features #530, #531 (independently reported by @cmidgley and @menway)
    • Closing braces } in the body of a script or a module were parsed as EOF. The bug was probably there for a decade or so! #526 (reported by @YaoHouyou)
    • Fixes for several obscure new issues reported by test262:
      • Sloppy direct eval that defines arguments in the default value of a parameter of an arrow function: (x, y = eval("var arguments = x")) => { print(arguments) }
      • Most Atomics methods can be used on TypedArray based on ArrayBuffer now (instead of only TypedArray based on SharedArrayBuffer).
      • The prototype of GeneratorFunction and AsyncGeneratorFunction is not a function.
      • Proxies can be created with revoked proxy as target or handler.
      • If the length of a function is Infinite, the length of its bound functions is also Infinite.
      • Function.prototype.toString behaves correctly if the function name is no identifier.
      • Setting or defining the length of an array coerce the length before checking if the length property is read-only.
    • Add documentation on static size allocations in XS (requested by @chances)
  • New HX711 driver. Supports digital scales. (contributed by @meganetaaan)
  • New module allows preferences to work on the simulator on Linux. Pure JavaScript implementation. (contributed by @cmidgley)
  • Add native API to get and set preference values for ESP32 (matches ESP8266)
  • BLE
    • Removed bonded property passed to client onConnected callback. Code should check bonded state from the onAuthenticated callback.
  • Graphics
    • Poco renderer's clip function return value indicates when fully clipped out to optimize drawing code
    • Commodetto's ReadPNG module now properly decodes PNG images that use a filter on the first scan line
  • Audio
    • AudioOut instances no longer crash when closed from a callback
    • M5Stack host waits for start-up sound to finishing playing before running app. (Same behavior as all other hosts with a start-up sound)
  • FT2606 touch driver
    • Implement calibration for FT2606 touch driver
    • Add calibration app for FT2606 running on Moddable One and Moddable Two
    • Add default touch calibration settings for Moddable One and Moddable Two
  • TypeScript support
    • Fixed case of piu/MC.d.ts and Resource.d.ts files for case sensitive file systems
    • Include Piu declarations in manifest_typings.json #527 (reported by @stc1988)
    • Add global declarations for Piu's Skin, Texture, Style, Behavior, Content, Container, Application, Scroller, Row, Column, Layout, Die, Port, Label, and Transition
    • Add declarations for the interpolators Piu adds to Math
    • Add declarations for piu/CombTransition and piu/WipeTransition
    • Add ArrayBuffer.prototype.concat, BigInt.fromArrayBuffer, and BigInt.bitLength to XS typings (found as result of @erights review in #523)
    • Remove DOM typings from build (the Moddable SDK does not provide a browser runtime!)

Note: Several of these changes are to the Moddable SDK tools. Therefore, after updating it is necessary to rebuild Moddable SDK tools and perform a clean build of your project. Details on how to do that are in the Update sections of the Getting Started guide for macOS, Windows, and Linux.

December 16, 2020

16 Dec 23:28
Compare
Choose a tag to compare
  • XS JavaScript engine

    • XS hosts for computers, ESP8266, and ESP32 now ignore .js and .mjs extensions when resolving modules. This capability is provided to ease use of JavaScript source code originally written for, or shared with, the web platform. It is not recommended for code intended only for use with the Moddable SDK. (requested by @dckc, @wmhilton, and a cast of thousands)
    • Fixes to optional chain call. While the implementation successfully passed test262, in some cases it left the stack in an incorrect state. Now you can safely write this.close?.()
    • Much larger stack in XS linker when preloading
    • Note: To use these XS changes, your need to update, rebuild Moddable SDK tools, and perform a clean build of your project. Details on how to do that are in the Update sections of the Getting Started guide for macOS, Windows, and Linux.
  • TypeScript support

    • Improvements to built-in type declarations, including the handful of XS extensions to the built-ins
    • Fix bug when building TypeScript source files stored in sub-directories (reported by @ariznaf)
  • BLE

    • Add SM.deleteBonding() API and associated onBondingDeleted() callback to delete a single bonded peer
    • Pass address and address type to client/server onDisconnected() callbacks
    • Pass bonded status to server onAuthenticated() callback
    • BLE keyboard example enhancements and fixes: generate callbacks for key down / up, fix caps lock handling, freeze instances for preload, and report arrow keys by name.
  • Documentation

  • ECMAScript Modules for Embedded Systems

  • Simulators

    • Simulated Wi-Fi connection for simulator (thanks to @wilberforce for some help)
    • ZIP module now builds on Windows simulator
    • Fix build of simulator on Linux for Raspberry Pi (reported by @natevw)
  • Enhancements

    • Add message constants to Socket
    • Add get battery voltage and current to AXP192 driver (by @stc1988)
    • Add Deflate implementation using miniz. Works in simulator but requires too much memory for most MCUs. Renamed Inflate directory to zlib.
    • Commodetto and Piu now use the Unicode horizontal ellipsis character when truncating text if available in the font. If not, they fall back to three consecutive periods. Good typography matters.
    • Moddable office window display animation app added to contributed directory. Details in blog post by Lizzie Prader.
    • bles2gatt tool now reports details of JSON parse errors in service descriptions and absence of some required elements
    • Updated tools documentation to expand on using command line arguments to mcconfig to add to the config #509 (suggested by @dckc)
  • Fixes

    • Fix remote socket disconnect failure on lwip socket implementation #503 (reported by @mlnuaa)
    • Don't double count HTTP client received bytes when using a callback to receive fragments #516 (reported by @wilberforce)
    • serial2xsbug now sets the correct timezone when running on Windows #502 (thanks to @wilberforce for help in tracking this down)
    • Instrumentation works again for Web Workers
    • Fix timers to be safe with Web Workers on QCA4020
    • Fix bug in Preference compaction on ESP8266
    • Fix ResourceIterator compilation on ESP8266
    • mcconfig and mcrun manifests now allow multiple path levels in directory targets on Windows too (reported by @ariznaf)

November 27, 2020

28 Nov 07:44
Compare
Choose a tag to compare
  • BLE
    • Add addressType to Client and Connection constructor dictionaries
    • Add getter for appearance on advertising data
    • Implement support for bonding on HID devices (mouse and keyboard)
  • Added Resource Iterator as optional patch to Resource module (original request by @wilberforce in #429). To use it, add $(MODDABLE)/modules/files/resourceiterator/manifest.json to include section of your project manifest, then Resource may be used as an iterator. If a mod (archive) is installed, its resources are also returned by the iterator.
import Resource from "Resource"

for (let name of Resource)
	trace(name + "\n");
  • Poco JavaScript API now has close method to release resources immediately, rather than waiting for garbage collector
  • Piu uses an iterative algorithm when marking fonts during garbage collection to reduce worst-case depth of native stack.
  • Wi-Fi scanner on ESP32 has option for passive scan (set active to false in dictionary passed to constructor)
  • Experimental ESP32 support for networking when both station and access point interfaces are active. Clients and servers work as well as mDNS.
  • Fix time calculations in RTC module #498 (reported by @stc1988, fixed by @wilberforce)
  • Add global require function to REPL to allow synchronous import of modules. (requested by @cmidgley)
  • Examples
    • Fixes to webConfigWifi example to eliminate simultaneous connection attempts
    • Wi-Fi continuous scan module example updated to latest API (target parameter removed from callbacks)
  • Tools and build
    • Set baud rate of xsbug connection for mcrun so it works with M5 products that don't use the default baud rate. (from @wilberforce)
    • mcconfig and mcrun manifests now allow multiple path levels in directory targets (requested by @ariznaf)
    • Moddable SDK builds on Raspberry Pi without any patches (thanks to @jins-tkomoda and @diogoviannaaraujo)
    • Moddable SDK on macOS Big Sur works with ESP32 and ESP8266. Getting Started documentation updated.
    • xsbug and mcsim now set the Retina flag for better rendering quality on high resolution displays (suggested by @tbdr)
    • Improved bezel image for 240x240 screens in simulator (from @sveniv)
  • modClock Project
    • Fix build problems
    • Reconnect after disconnect
    • Rework logic for entering and exiting access point mode
    • Static probe method added to RTC modules to eliminate exceptions on start-up when scanning for RTC
    • Sort Wi-Fi access point list
    • Set character encoding of HTML pages to UTF-8 so access points with non-ASCII characters display correctly

November 16, 2020

16 Nov 22:56
Compare
Choose a tag to compare
  • Crashes in the XS JavaScript engine fixed - #483, #484, and #485. This included a crash calling RegExp.prototype.toString with a proxy that returns numbers for source and flags. (reported by @kvenux)
  • Experimental metering feature in XS to allow a host to restrict the number of byte-codes executed. Not ready for general use.
  • Added low-level hook to Poco renderer to support custom drawing extensions.
  • Work underway to move all desktop simulators from the venerable screen test to mcsim to allow for more complete simulation. The new simulator is only used when the target platform begins with sim/. This work is still experimental and not ready for general use.
  • Turn Wi-Fi off at start-up on ESP8266 because some boards ship with flash preferences that enable it unexpectedly.
  • Improvements to the BLE mouse example - hit test order, mouse up handling, and rapid double clicks
  • BLE HID (Human Interface Device) module optimization to accelerate initial device connection.
  • BLE simplifications for remembering native objects (eliminates a build warning)
  • Piu sound example now includes M5Stack and M5Stack Core2 on the safe list (reported by @stc1988)
  • M5Stack now uses M5Button class for debouncing, like other M5 products. (from @stc1988)
  • serial2xsbug does a simple validation of the archive file to avoid installing corrupt mod. (based on report by @ariznaf)
  • mcrun on Windows now uses release tools like macOS and Linux
  • Fix deprecation warning for openFile in Piu PC on macOS Big Sur
  • Added support for 240x240 OLED displays using ST7789. (from @sveniv)
  • Support all four SPI modes in modSPI for ESP8266, ESP32, SiLabs Gecko, and Qualcomm QCA4020. (from @sveniv)
  • ILI9341 display driver supports configuring the SPI mode. (from @sveniv)
  • Add 240x240 display size to simulator (from @sveniv)

October 30, 2020

31 Oct 00:14
Compare
Choose a tag to compare

October 30, 2020

  • Instances of Digital now have a close method
  • In instances of TypedArray (Uint8Array, Float32Array, etc) an internal slot was visible to scripts. This was generally harmless but caused problems in certain circumstances. Fixes #479 and #480 (reported by @dckc)
  • Faster ESP32 builds on Windows by avoiding unnecessary time stamp change to sdkconfig.h
  • On Windows, detect invalid MODDABLE environment variable. Provides early error for #471 (reported by @thomasqbrady)
  • Fix for screen initialization on certain revisions M5Stack models (from @meganetaaan)
  • M5 device targets that support audio finish play the start-up sound before running the app. Fixes #482 (reported by @stc1988)
  • Add driver for BM8563 real time clock. Use in second generation M5 Stick C device target. (from @wilberforce)
  • AudioOut instances no longer are automatically garbage collected; they must be explicitly closed
  • NFNT font engine now uses the offset (from width offset table) when calculating horizontal position
  • New build targets in the simulator for Moddable One, Moddable Two, and Moddable Three. The devices support rotation.
> mcconfig -d -m -p sim/moddable_one
> mcconfig -d -m -p sim/moddable_two
> mcconfig -d -m -p sim/moddable_three
> mcconfig -d -m -p sim/moddable_three -r 90

Moddable Three Moddable Three rotated

October 20, 2020

20 Oct 21:42
Compare
Choose a tag to compare

This most significant change in this release is a fix to JSON parsing. The XS JavaScript engine has always been lenient in parsing for convenience, accepting trailing commas that are prohibited by the JSON specification. Here's an example, where the comma after "./main", ], and } are all disallowed in JSON (though allowed in JavaScript code):

{
	"include": "$(MODDABLE)/examples/manifest_base.json",
	"modules": {
		"*": [
			"./main",
		],
	},
}

With this release, XS fully conforms to the JSON specification, rejecting trailing commas. If your project manifests included trailing commas, you will now encounter parsing errors when building your project with mcconfig.

Note: This update to XS is backwards compatible with the byte-code from the October 14, 2020 release. Therefore, you do not have to rebuild the Moddable SDK tools, though it is recommended.

  • XS JavaScript engine fully conforms to JSON parsing requirements
  • All JSON files (about 300!) in the Moddable SDK have been updated to eliminate trailing commas #476 (reported by @erights)
  • mcconfig and mcrun now report manifest parsing errors with the file name and line number. Previously this only worked with debug builds of the tools.
  • Add $USERHOME environment variable to mcconfig which maps to $HOME on macOS and Linux, and $USERPROFILE on Windows. This is used to access some device specific build resources.
  • Fix build of servo module on Windows by using $USERPROFILE
  • Replace stray comma with semicolon in piu/bars example. Fixes #475 (reported by @stc1988)
  • Fix for display brightness control on all versions of M5Stick-C hardware #474 (from @wilberforce).
  • 16 color (4-bit palette color) displays work again in simulator and ESP8266. Works for the first time on ESP32. #477 (reported by @stc1988)
  • Added I2S to the Windows ESP8266 build to allow audio examples to build