Skip to content

Commit

Permalink
Added documentation for new display color API and bitmap API
Browse files Browse the repository at this point in the history
  • Loading branch information
siliconwitch committed Jul 19, 2024
1 parent 5cc83f4 commit ad3a8a0
Show file tree
Hide file tree
Showing 3 changed files with 39 additions and 14 deletions.
Binary file added frame/images/frame-sprite-engine.drawio.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added frame/images/frame-ycbcr-colorspace.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
53 changes: 39 additions & 14 deletions frame/lua.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,16 +36,13 @@ The API specification is still undergoing heavy development. Some of them may ch

The display engine of allows drawing of text, sprites and vectors. These elements can be layered atop one another simply in the order they are called, and then be displayed in one go using the `show()` function.

The Frame display is capable of rendering up to 16 colors at one time. These colors are preset by default, however each color can be overridden by any 8bit YCbCr color using the `palette` command.
The Frame display is capable of rendering up to 16 colors at one time. These colors are preset by default, however each color can be overridden.

| API                                                                                                                      | Description |
| API                                                                                                          | Description |
|:---------|:------------|
| `frame.display.palette{}` | *Details coming soon*
| `frame.display.text(string, x, y, {color='WHITE', align='TOP_LEFT'})` | Prints the given `string` to the display at `x` and `y`. A `color` can optionally be provided to print the text in one of the 16 palette colors, and `align` can optionally be provided to jutify the text as either `'TOP_LEFT'`, `'TOP_CENTER'`, `'TOP_RIGHT'`, `'MIDDLE_LEFT'`, `'MIDDLE_CENTER'`, `'MIDDLE_RIGHT'`, `'BOTTOM_LEFT'`, `'BOTTOM_CENTER'`, or `'BOTTOM_RIGHT'`
| `frame.display.bitmap()` | *Details coming soon*
| `frame.display.vector()` | *Details coming soon*
| `frame.display.show()` | Shows the drawn objects on the display

| `frame.display.text(string, x, y, {color='WHITE', spacing=4})` | Prints the given `string` to the display at co-ordinates `x` and `y`. A `color` can optionally be provided to print the text in one of the palette colors. Valid options for `color` are: `WHITE` ,`GREY` ,`RED` ,`PINK` ,`DARKBROWN` ,`BROWN` ,`ORANGE` ,`YELLOW` ,`DARKGREEN` ,`GREEN` ,`LIGHTGREEN` ,`NIGHTBLUE` ,`SEABLUE` ,`SKYBLUE` or `CLOUDBLUE`. `spacing` can optionally be provided to adjust the character spacing of the printed text.
| `frame.display.vector()` | *Details coming soon*
| `frame.display.show()` | Shows the drawn objects on the display

#### Example
{: .no_toc }
Expand All @@ -56,6 +53,34 @@ frame.display.text('Hello world', 50, 100)
frame.display.show()
```

| Low level functions                                                        | Description |
|:---------|:------------|
| `frame.display.assign_color(color, r, g, b)` | Changes the rendered color in slot `color` with a new color given by the components `r`, `g,` and `b`. Valid options for `color` are: `VOID`, `WHITE` ,`GREY` ,`RED` ,`PINK` ,`DARKBROWN` ,`BROWN` ,`ORANGE` ,`YELLOW` ,`DARKGREEN` ,`GREEN` ,`LIGHTGREEN` ,`NIGHTBLUE` ,`SEABLUE` ,`SKYBLUE` or `CLOUDBLUE`. Note that changing the `VOID` color will change the rendered background color of the display. The RGB components are internally converted to a 10bit YCbCr value that represents the true colorspace of the display. There may therefore be rounding errors for certain RGB combinations.
| `frame.display.assign_color_ycbcr(color, y, cb, cr)` | Same as above, however the `y`, `cb`, and `cr` represent the true 10bit colorspace of the display. Each component has a range of 4, 3, and 3 bits respectively.
| `frame.display.bitmap(x, y, width, color_format, palette_offset, data)` | Prints raw bitmap data to the display at co-ordinates `x` and `y`. `width` should be the width of the bitmap. `color_format` represents the total colors users, and should be either `2`, `4`, or `16`. `palette_offset` offsets the colors indexed from the palette. If no offset is desired, set this to `0`. `data` should be a string containing the bitmap data.

#### Color Palette
{: .no_toc }

The Frame display is capable of rendering up to 16 colors at a time. Each color is indexed 0-15, and are named in the following order: `VOID`, `WHITE` ,`GREY` ,`RED` ,`PINK` ,`DARKBROWN` ,`BROWN` ,`ORANGE` ,`YELLOW` ,`DARKGREEN` ,`GREEN` ,`LIGHTGREEN` ,`NIGHTBLUE` ,`SEABLUE` ,`SKYBLUE` or `CLOUDBLUE`. `VOID` represents the background (normally black) color. Each color can be overridden per frame to any 10bit YCbCr color from the colorspace shown below. This space contains a total of 1024 possible colors.

![Frame display YCbCr colorspace](/frame/images/frame-ycbcr-colorspace.png)

The `assign_color()` function simplifies color selection by allowing the user to enter regular 24bit RGB values which are internally converted to the YCbCr colorspace. Note however that the color actually rendered will be rounded to one of the above colors.

#### Sprite Engine
{: .no_toc }

The sprite engine in Frame is capable of quickly rendering bitmap data to anywhere on the screen. Sprites data can be stored in one of three different color formats. 2 color, 4 color, and 16 color. 2 color mode will only use the first two colors in the color palette, but allows storing 8 pixels per byte of sprite data. 4 color will use the first four colors in the palette, but requires twice as much data per pixel. Finally, 16 color mode allows accessing the entire color palette, but requires 4 bits per pixel. Pixel data in Lua is simply represented as a long string of all the pixels. The `width` parameter tells the sprite engine when to move to the next line when rendering out each pixel from the string.

Each pixel represented by the pixel data simply indexes one of the colors from the color palette. For 2 color mode, a bit (pixel) value of `1` will print a pixel of color `WHITE`. The internal font glyphs used by the `text()` function are essentially all 2 color sprites.

To achieve different color text, the `palette_offset` feature is used. This value offsets how the colors are indexed during the render. Using the same 2 color example as above, but combined with a `palette_offset` value of 3, now indexes `PINK` instead of `WHITE` for all pixel values of `1`. The same works for all the other color modes. Note that the `VOID` color is never shifted. A pixel value of `0` will always index `VOID`, no matter the `palette_offset`.

The example below shows how a single sprite can be shown in different colors using the `palette_offset` feature. Note how the color palette has been adjusted to repeat the same colors after index 5, but with red changed to green.

![Frame display sprite engine example](/frame/images/frame-sprite-engine.drawio.png)

---

### Camera
Expand All @@ -66,7 +91,7 @@ The camera capability of Frame allows for capturing and downloading of single JP
|:---------|:------------|
| `frame.camera.capture{quality_factor=50}` | Captures a single image from the camera. The `quality_factor` option can help reduce file sizes by adjusting the JPEG quality. The four possible options are `10`, `25`, `50`, and `100`. Higher values represent higher quality, but also larger file sizes.
| `frame.camera.read(num_bytes)` | Reads out a number of bytes from the camera capture memory as a byte string. Once all bytes have been read, `nil` will be returned
| `frame.camera.auto{metering='AVERAGE', exposure=0, shutter_kp=0.1, gain_kp=1.0, gain_limit=248.0}` | Runs the automatic exposure and gain algorithm. This funtion must be called every 100ms for the best performance. `metering` can be one of three modes, `SPOT`, `CENTER_WEIGHTED`, or `AVERAGE`. `exposure` can be a value between `-2.0` and `2.0` where lower values will return slightly darker images, and higher values will return slightly brighter images. `shutter_kp` and `gain_kp` allow fine tuning of the auto exposure algorithm. Higher values can make reaching the desired exposure faster, but may result in instability and oscilliation of the control loop. These values are generally more sensitive when using the `SPOT` or `CENTER_WEIGHTED` metering modes. `gain_limit` can be used to cap the gain to below the maximum of `248`. This is useful to reduce noise in darker scenes and results in faster exposure when going from darker to brighter scenes.
| `frame.camera.auto{metering='AVERAGE', exposure=0, shutter_kp=0.1, gain_kp=1.0, gain_limit=248.0}` | Runs the automatic exposure and gain algorithm. This function must be called every 100ms for the best performance. `metering` can be one of three modes, `SPOT`, `CENTER_WEIGHTED`, or `AVERAGE`. `exposure` can be a value between `-2.0` and `2.0` where lower values will return slightly darker images, and higher values will return slightly brighter images. `shutter_kp` and `gain_kp` allow fine tuning of the auto exposure algorithm. Higher values can make reaching the desired exposure faster, but may result in instability and oscillation of the control loop. These values are generally more sensitive when using the `SPOT` or `CENTER_WEIGHTED` metering modes. `gain_limit` can be used to cap the gain to below the maximum of `248`. This is useful to reduce noise in darker scenes and results in faster exposure when going from darker to brighter scenes.

#### Example
{: .no_toc }
Expand Down Expand Up @@ -105,7 +130,7 @@ end

### Microphone

The microphone on Frame allows for streaming audio to a host device in real-time. Transfers are limited by the Bluetooth bandwith which is typically around 40kBps under good signal conditions. The audio bitrate for a given `sample_rate` and `bit_depth` is: `sample_rate * bit_depth / 8` bytes per second. An internal 32k buffer automatically compensates for additional tasks that might otherwise briefly block Bluetooth transfers. If this buffer limit is exceeded however, then discontinuities in audio might occour.
The microphone on Frame allows for streaming audio to a host device in real-time. Transfers are limited by the Bluetooth bandwidth which is typically around 40kBps under good signal conditions. The audio bitrate for a given `sample_rate` and `bit_depth` is: `sample_rate * bit_depth / 8` bytes per second. An internal 32k buffer automatically compensates for additional tasks that might otherwise briefly block Bluetooth transfers. If this buffer limit is exceeded however, then discontinuities in audio might occur.

| API                                                                                            | Description |
|:---------|:------------|
Expand Down Expand Up @@ -154,7 +179,7 @@ The tap gesture will always wake up Frame from `frame.sleep()`.
| API | Description |
|:---------|:------------|
| `frame.imu.direction()` | Returns a table containing the `roll`, `pitch` and `heading` angles of the wearer's head position
| `frame.imu.tap_callback(handler)` | Assigns a callback to the tap gesture. `handler` must be a function, or can be `nil` to deactive the callback
| `frame.imu.tap_callback(handler)` | Assigns a callback to the tap gesture. `handler` must be a function, or can be `nil` to deactivate the callback

| Low level functions | Description |
|:---------|:------------|
Expand Down Expand Up @@ -182,7 +207,7 @@ The Bluetooth API allows for sending and receiving raw byte data over Bluetooth.
| API | Description |
|:---------|:------------|
| `frame.bluetooth.address()` | Returns the device MAC address as a 17 character string. E.g. `4E:87:B5:0C:64:0F`
| `frame.bluetooth.receive_callback(handler)` | Assigns a callback to handle received Bluetooth data. `handler` must be a function, or can be `nil` to deactive the callback
| `frame.bluetooth.receive_callback(handler)` | Assigns a callback to handle received Bluetooth data. `handler` must be a function, or can be `nil` to deactivate the callback
| `frame.bluetooth.max_length()` | Returns the maximum length of data that can be sent or received in a single transfer
| `frame.bluetooth.send(data)` | Sends data to the host device. `data` must be a string, but can contain byte values including 0x00 values anywhere in the string. The total length of the string must be less than or equal to `frame.bluetooth.max_length()`

Expand Down Expand Up @@ -214,7 +239,7 @@ The file system API allows for writing and reading files to Frame's non-volatile
| `frame.file.mkdir(pathname)` | Creates a new directory with the given `pathname`
| `f:read(*num_bytes)` | Reads a number of bytes from a file. If no argument is give, the whole line is returned
| `f:write(data)` | Writes data to the file. `data` must be a string and can contain any byte data
| `f:close()` | Closes the file. It is important to close files once done writing, otherwise they may become courrupted
| `f:close()` | Closes the file. It is important to close files once done writing, otherwise they may become corrupted

#### Example
{: .no_toc }
Expand Down Expand Up @@ -290,6 +315,6 @@ The system API provides miscellaneous functions such as `sleep` and `update`. It

| Low level functions                            | Description |
|:---------|:------------|
| `frame.stay_awake(enable)` | Prevents Frame from going to sleep while it's docked onto the charging cradle. This can help during development where continious power is needed, however may degrade the display or cause burn-in if used for extended periods of time
| `frame.stay_awake(enable)` | Prevents Frame from going to sleep while it's docked onto the charging cradle. This can help during development where continuous power is needed, however may degrade the display or cause burn-in if used for extended periods of time
| `frame.fpga_read(address, num_bytes)` | Reads a number of bytes from the FPGA at the given address
| `frame.fpga_write(address, data)` | Writes data to the FPGA at a given address. `data` can be a string containing any byte values

0 comments on commit ad3a8a0

Please sign in to comment.