Manual.txt

 /============================================\
/-Beryllium OS standalone documentation sheet--\
\-----| bill88t |------------------------------/
 \============================================/

Official repository: https://github.com/beryllium-org/OS
This manual is up to date with Release 0.5.1 @ 25/jul/2024.
Anything passed that point is not covered by this manual.

      /------------|
----<| Index table |
      \------------|

0.  Index table          : line 10
1.  Basic commands       : line 27
2.  Scripting            : line 409
3.  Errors               : line 431
4.  Api functions        : line 511
5.  jCurses              : line 617
6.  Led activity         : line 670
7.  Virtual Paths        : line 681
8.  GPIO Usage           : line 702
9.  Network driver usage : line 730
10. Isolated processes   : line 746
11. Packages             : line 802
12. Devices and consoles : line 949
13. Contributions        : line 995

      /---------------|
----<| Basic commands |
      \---------------|

The following commands are the ones that are included with every installation, with the
package "coreutils".
The commands are sorted alphabetically.


advance_build_date

Checks if the system RTC (Real Time Clock) is not initialized, and advances it to build time.


alias [COMMAND=data]

Runs data, whenever command is typed


bcast [DATA]

Broadcast a message to all terminals


bell

Ring the bell on the current terminal


cat [file]

Catenates the contents of file [file]


cd [DIR]

Changes to specified directory
Please refer to the virtual paths section, on more info on how does the board filesystem appear.


clear

Clear the current terminal, clearing scrollback too.


clear_line

Clear the current terminal line.


cp [SOURCE] [DESTINATION]

Copies source to destination.
Please keep in mind the current implementation is INCREDIBLY FRAGILE.


date [set]...

View the time, or set it
Valid format "date set dd mm yyyy hr mm ss"


detect

Fetch the term.detect_size() for context updating


devmode [OPTIONS]

Enables usb access and other development features.
This temporarily prohibits write access to the built in filesystem.
The notice can be skipped by passing -q.
This change by default is applied for one boot.
To make this permenant, which is absolutely NOT RECOMMENDED, pass -p.
To revert the change, set `usb_msc_enabled = false` under
section [BERYLLIUM] in the board's `/settings.toml`.


df [-h]

Report all attached file system's space usage.


dir [??].. ??

A paradox of unknown origin. Unknown effects.


dmesg

Prints out the current dmesg log.


dropcache [FILE]...

Drops the specified file, or if none is specified, all files, from compiled code cache.
On boards without 150kb ram free, code is not cached, and this command is useless.


echo [DATA]...

Print some variables or text.


exec [FILE]

false

Returns false.


exit [CODE]

Attempts to disconnect the shell.
If the shell cannot be disconnected, ask the user to disconnect instead.


false

Returns a false exit code.


flush

Flushes terminal forcibly.


fpexec [file]

Runs a python file in a new process.


free [-b/-k/-m/-g]

Display memory usage.


halt

Sync all filesystems, remount everything read-only and halt the system.


head [-n int] [FILE]

Output the first part of files.
By default displays the first 10 lines of the FILE.


help [COMMAND]

Displays all currently discovered commands.


hold

Hold terminal output.


history [save/load/clear]

Displays the command history.


hostname [VALUE]

Displays the hostname if no VALUE is passed, else sets it
to that VALUE.


jpkg [install/uninstall/list] [PACKAGE(S)]

The built-in Beryllium os package manager.
For more information run `jpkg --help`, `man jpkg` or
view section JPKG.


jz [-q/-d/--decompress] [JZ ARCHIVE] [TARGET DIRECTORY]

Decompress a .jz archive to the TARGET DIRECTORY.
Extracts to '.' if not specified.


ledset [int / int int int]

Set the led to the given value(s).


less [FILE]

Opposite of more, a simple file viewer.


ls [OPTIONS]... [DIR]...

Outputs the directory listing.


man [COMMAND]

Display the manual for the given [COMMAND] with `less`.


mkdir [DIR]...

Make a new directory


mknod [NAME]

Create an empty device node.
Resulting name returned.


modprobe [MODULE as MODULE_TYPE]

Load a kernel MODULE with it's type specified by MODULE_TYPE.


mv [SRC] [DST]

Moves files or folders from source to destination.


pinout

Display current board pinout.


pexec [-n] [python commands]

Runs python commands in the current process.


pinout

Displays the current board's pinout map.


preload [FILE]

Compile and load a python file into ram.


pwd

Print the current working directory.


python

Opens a virtual python REPL.
To pass commands instead of actually opening the REPL, use pexec instead.
To pass files to run instead of actually opening the REPL, use fpexec instead.


quit

Quit Beryllium OS and exit to CircuitPython REPL.


random

Returns a random number. Displays it if based is not silenced.


reboot [MODE]

Reboots the board.
Optionally, you can pass a reboot MODE from: [safemode, uf2, bootloader]


reload

Reloads the os fully.


rm [FILE(S)]...

Delete FILE(S).
Recursive deletes not yet supported.


rmdir [DIR]...

Delete an empty directory.


rmnod [NAME]

Remove the specified device.


runparts [FOLDER]

Run all the .py files in a folder by alphabetic order.
Used by init.


runtasks

Enters task execution mode, running all background tasks.


sleep [TIME]

Sleep TIME, in seconds.


sync

Sync all currently attached filesystems.


sysinfo

Displays system info.


terminal [get/list/activate]...

A command to retrieve / set the current console.
Allows retrieving the list of consoles too.


title [TEXT]

Sets the terminal title to the TEXT.


top

Display the current process stack, along with useful info.


touch [FILE]

Create a new FILE.


true

Returns true.


unalias [ALIAS]

Removes ALIAS.


uname [-a]

Print system information.


unhold

Releases a terminal, allowing data output.


unset [Variable]

Deletes a Variable.
Can also be used on GPIO variables.
The GPIO will be deinitialized.


var [DATA]...

Create a new variable. syntax: var a = "ok"
Numbers do not need brackets.
The command itself is optional, `a = "is also valid"`
On how to use GPIO, refer to the relevant section.


watch [-s] [-n SECONDS] [COMMAND]

Run a COMMAND every n SECONDS (default 2 seconds).
Hide the titlebar text with `-s`.


      /----------|
----<| Scripting |
      \----------|

Based script files need to have the .lja file extension. All internal or external commands
can be used in scripts, along with kernel api functions.
To have a script appear as a command, put it in /{pv[0]["root"]}/bin/
(where pv[0]["root"] is the absolute root path name).

To execute python code, you are advised to use pexec & fpexec.
The scope is the same as the one of the kernel and you can call & alter kernel data freely.
This however is very unsafe (as in, you may touch foreign variables), and you are instead advised
to use the given variable container storage.

When fpexec is called, the code is given an empty container storage under `pv[get_pid()]`.
You can manipulate it either directly or with the `vr` functions. More info in the Container Storage
section of the manual.

To access shell input you can use `be.api.xarg()` or `be.based.user_vars["argj"].split()`.
You can find more info regarding xarg under it's api documentation.


      /-------|
----<| Errors |
      \-------|

The different errors used by the based shell.
CODE:
      be.based.error([number])
      where [number] is one of the error below

1 : Syntax Error
      The command cannot work with the given arguments
      Read #basic-commands

2 : Input Error
      The input given is not understandable from the command

3 : Error
      This is a general error

4 : [file]: No such file or directory
      The file cannot be found

5 : Network unavailable
      The network interface is not functional

6 : Display not attached
      The display is not connected / registered.

7 : Filesystem unwritable, board in developer mode
      This error shows up when a write to the internal storage occures while developer mode
      is enabled.
      By CircuitPython design, when the internal storage is being made available from usb,
      it is unwritable to the beryllium os.
      By default, developer mode lasts for one reboot, however when used with "-p" it's enabled
      permanently.
      To revert that, set `usb_msc_enabled = false` under section [BERYLLIUM] in the board's
      `/settings.toml`.

8 : Missing files
      This error occurs when files needed for the command to run are missing.

9 : Missing arguments
      This error occur when the command is used without the necessary parameters.

10 : File exists
      A file that shouldn't exist, does exist.

11 : Not enough system memory
      POV: you are using a microcontroller.

12 : Error, variable already used:
      The variable name specified is already in use, and can't be modified in this way.

13 : Terminal too small, minimum size: size:
      The current terminal size is too small, resize the terminal to a size bigger than
      the one specified.

14 : Is a file
      What was passed, is a file, when it shouldn't.

15 : Is a directory
      What was passed, is a directory, when it shouldn't.

16 : Directory not empty
      The directory should have been empty, but wasn't.

17 : Not a directory
      What was passed, is not a directory, when it should.

18 : Not a file
      What was passed, is not a file, when it should.

19 : Not a device
      What was passed, is not a device, when it should.

20 : Unhandled exception
      Under fpexec execution, an exception was raised that was not caught by any code from within
      the fpexec code, and it was instead caught by the kernel.


      /--------------|
----<| Api Functions |
      \--------------|

Beryllium OS api functions.
Available from:
    be.api.[function](parameters)

    tasks
        run() -> None
            Run background tasks if needed.

        add(name: str, priority: int 0-100, check_func: function, run_func: function) -> int
            Add a task, returns a pid.

        rm(pid: int)
            Deletes a task.

    security
        auth <- class
            __init__(value: any)
            key(value: any) -> bool
            id <- property
        A class that can be used for securely store a private variable.
        It will never permit access to that variable.
        You can only check if a value matches.
        Causes a blocking delay when authentication fails.

    getvar(var)
        Variable getter.
        Returns variable var, from be.based.user_vars or be.based.system_vars.
        The var name has to be an str.
        In case it doesn't exist, returns None.

    setvar(var, data=None, system=False)
        Variable setter / deleter.
        Sets variable var, in be.based.user_vars.
        The var name has to be an str.
        When the data is None, deletes the variable.
        The system option, when True, uses be.based.system_vars instead.

    xarg(rinpt=None, fn=False)
        Argument parser.
        When no input stream specified, rely on be.based.user_vars["argj"].
        Returns a dict with the following stuff:
          "w" for the words that don't belong to a specific option.
               Example: "ls /bin", "/bin" is gonna be returned in "w"
          "hw" for the words, that were hidden due to an option. Example "ls -a /bin", "/bin" is
               not gonna be in "w" as it is a part of "o" but will be in "hw".
          "o" for all the options, with their respective values.
               Example: "ls -a /bin", {"a": "/bin"} is gonna be in "o"
          "n" if False is passed to fn, contains the filename

    fs
        resolve(back: str = None) -> str
            Path translation from Beryllium OS virtual paths to real path.
            For example: &/settings.toml -> /settings.toml
                         /bin/ls.py -> /Beryllium/bin/ls.py

        base(path=".") -> str
            Given a path, finds what is the physical path.
            ../../../../../ -> /
            /bin/../bin -> bin

        isdir(dirr: str, rdir: str = None) -> int
            Tests if input is a folder, file or doesn't exist.
            Returns:
                - 0 for files
                - 1 for folders
                - 2 if it doesn't exist

        open <- class
            Equivelant to open(), but works with all virtual paths.

        listdir(path=".") -> list
            An advanced directory lister, that supports all virtual paths.
            Returns a list that contains lists.
            Each sublist is structured as follows:
                0: name
                1: type, can be "f", "d", "c" for file, folder, device respectively.
                2: permissions, a 3 item list following unix user/group/others format.
                3: size in bytes
                4: modification time in struct_time
                5: owner
                6: group

    code_load(filename: str) -> code
        Loads a python code file and returns a code object.
        May raise an OSError if the file does not exist, or other errors,
        during code compilation.

    adv_input(whatever: any, _type=str):
        Universal variable requester.
        Fetches variables from all contexts and returns it _type casted.

    subscript(filen: str) -> None
        A helper function for calling for scripts to run under other scripts,
        under the same scope, pid and variable container storage.

    console_connected() -> bool
        Tests if any console is connected, and automatically loads it.

    bcast(msg: str) -> None
        Broadcasts a message to all consoles.


      /--------|
----<| jCurses |
      \--------|

jCurses are the beryllium version of nCurses.
The library is preinstalled in "/lib".

Each program should (but it's not necessary) use it's own jcurses object.
Based, by default, creates the "term" jcurses object.

Useful functions:

    - .write(strr=str, end="\n")
     The default write function.
     Always prints to the active terminal only.

    - .nwrite(strr=str)
     Same as `.write`, but does not print "\n" at the end.
     It saves byte size when there are 25 septilion `end=""`'s.
     Always prints to the active terminal only.

    - .detect_size()
     Detects the size of the terminal & returns it as [rows, collumns]

    - .detect_pos()
    Detects the current cursor position & returns it as [row, collumn]

    - .ctx_reg(name)
    Registers the current cursor position as a bookmark with the name specified.

    - .move(ctx=None, x=None, y=None)
    Moves the cursor to a specified x (row), y (collumn), or a bookmark.
    You can specify both a bookmark & and x, y coordinate.
    In that case, the x, y will be used as an offset.

    - .clear()
    Clears the screen & goes to top.

    - .clear_line()
    Clears the current line.

    - .line(char)
    Draws a line with the given char.

    - .map()
    Testing function, prints chars inputted.

    - .rem_gib()
    Empty stdin for ansi code usage. The data is not discarded.

jCurses does not depend onto Beryllium OS.


      /-------------|
----<| Led activity |
      \-------------|

Beryllium OS supports both generic led's as well as neopixels and rgb leds for it's activity led.

Generic leds stay on when idle, and off on command activity.
Neopixels stay green when idle, glow more when special characters are being inputted,
green normally, red when an error occurs, yellow on activity and blue when waiting.


      /--------------|
----<| Virtual paths |
      \--------------|

The beryllium kernel is in the board's root, while the os root is under by default under /Beryllium.
The root path is set by the kernel in pv[0]["root"].
Root cannot be "/".
For simulating the root being under "/" virtual paths are used with kernel integrations.
The be.api.fs.resolve function stands as a middleman between userspace all os path requests.
However it doesn't prohibit the access to the board's real root.
It work by mapping:

    Real path | Betterpath
       "/"    |  "&/"
 "/Beryllium" |  "/"

And treating all subsequent paths by that rule.
It additionally maps "~" to the user's home directory, and "-" to the previous directory.
All kernel functions use resolved virtual paths automatically.


      /-----------|
----<| GPIO Usage |
      \-----------|

GPIO pins as variables, both digital and analog io is supported.
Example:
```
    a = gp#GP9
```
This would temporarily allocate board.GP9 and read it's state digitally, storing the result
to variable 'a'. The pin would then be free'ed.

doing the opposite:
```
    gp#GP9 = 1
```
would set board.GP9 as an output pin and set it's value to 1.
From there reading back the pin will switch it back to read mode, but not deinit it.

Temporarily, only analog reads are supported. This is achieved by:
```
    a = adc#GP25
```

For using I2C, UART and ADC there are program provided from the adctools,
i2ctools and uartutils packages.


      /---------------------|
----<| Network driver usage |
      \---------------------|

To use the built-in circuitpython module "wifi" as your network connection:
  - Run "modprobe driver_wifi as network" to load the beryllium os wifi driver.
     You can optionally add it to /boot/Init.lja, to load it on boot.
  - Use iwctl to configure the wifi connection.
     For info on how to use iwctl, please refer to [https://wiki.archlinux.org/title/iwd]
  - You connections will automatically be stored into &/settings.toml.
     You can automatically reconnect on reboot by running `iwctl station wifi auto`.

Wifi boards are automatically set to use iwctl in automatic mode during boot.
This behaviour can be edited through /boot/boot.d/


      /-------------------|
----<| Isolated processes |
      \-------------------|

During beryllium execution every variable created persists across processes.
And every process can access any other process's stuff. Very secure, yea.
Having a parser that effectively blocks touching other processes would be
too much of an overhead and impractical.

Still, it's possible there are variable nameshakes and forgotten variables.
(If you use variable "i", your parent process could also be using it, and you could
break some loop, causing a `while True`)
If you do not delete your variables before the program exit, they WILL REMAIN!

For this I made a container storage for processes.
Whenever fpexec is run, the code inside gets a pid.
You can read that value with `get_pid()`.
Every process also gets a dict in `pv`.
This dict will be automatically deleted after the process is exited.

You can directly access from python it as `pv[get_pid()]`.

To set a (new) variable directly from python,
run `pv[get_pid()]["new_variable_name"] = whatever_value`.

However that is long and ugly. There are functions for managing container varibles.
To set a variable:
`vr("variable_name", value)`

To get the value:
`vr("variable_name")`

To add a value to a variable:
`vrp("variable_name", value)`
`vrp("variable_name")` # += 1 to the variable.

To subtract a value from a variable:
`vrm("variable_name", value)`
`vrm("variable_name")` # -= 1 to the variable.

To append a value to a variable:
`vra("variable_name", value)`

To delete a variable:
`vrd("variable_name")`

Still if you want to iterate with for, you have to do this:
```py
for pv[get_pid()]["i"] in vr("some_list"):
      vr("data", vr("i") * potatoes)
```

You can use properties of the variables just fine with `vr`.
`vr("socket").accept()`


      /---------|
----<| Packages |
      \---------|

Beryllium OS comes with an incredibly simple package manager called JPKG.
The packages themselves are jz zlib-compressed sets of files.

The following is detailing as to how to create a a jpkg package and the packaging
guidelines, if you just want to know how to use the package manager, read the man page or skip
further below.

Each .jpk package must contain a `Manifest.json` (this is case sensitive)
and that manifest must contain the following:

 - An interger specifying the minimum jpkg version supported
   (JPKG_minimum_version)
 - The package name as a string
   (package_name)
 - The package version in a 3-int list.
   (version)
 - A list of dependencies and a list of conflicts
   (dependencies and conflicts)
 - The script name to be run on a first install
   (install)
 - The script name to be run on uninstall
   (remove)
 - The script name to be run on update (may match install)
   (update)
 - The script name to be run from a desktop computer to strap onto the root
   (strap)
 - A boolean if a restart is required
   (restart_required)

The resulting json should look like this:
```json
{
    "JPKG_minimum_version": 2,
    "package_name": "package",
    "version": [0, 0, 0],
    "dependencies": [],
    "conflicts": [],
    "install": "installer.py",
    "remove": "uninstaller.py",
    "update": "updater.py",
    "strap": "strap.py",
    "restart_required": false
}
```

The install script should use shell commands for cp and mkdir.
You should not attempt to perform direct io operations without the use of the api or commands.

Example from the ducky package:
```py
# ducky files
for pv[get_pid()]["filee"] in ["ducky.lja", "duckyline.lja", "duckycat.lja"]:
    be.based.run("cp " + vr("filee") + " /bin/" + vr("filee"))

be.based.run("mkdir /bin/ducky")
for pv[get_pid()]["filee"] in [
    "ducky.py",
    "duckyline.py",
    "duckyload.py",
    "duckycat.py",
]:
    be.based.run("cp " + vr("filee") + " /bin/ducky/" + vr("filee"))

be.based.run("cp help_ducky.txt /usr/share/help/ducky.txt")
be.based.run("cp help_duckyline.txt /usr/share/help/duckyline.txt")
be.based.run("cp help_duckycat.txt /usr/share/help/duckycat.txt")

# hid files
be.based.run("mkdir /lib/adafruit_hid")

for pv[get_pid()]["filee"] in [
    "__init__.mpy",
    "consumer_control_code.mpy",
    "consumer_control.mpy",
    "keyboard_layout_base.mpy",
    "keyboard_layout_us.mpy",
    "keyboard.mpy",
    "keycode.mpy",
    "mouse.mpy",
]:
    be.based.run("cp " + vr("filee") + " /lib/adafruit_hid/" + vr("filee"))

be.api.setvar("return", "0")
```

If the installation was successful you should always end the script
with `be.api.setvar("return", "0")` or otherwise jpkg will raise an error.

For the uninstall script you should manually delete every file, since currently shell rm doesn't
support recursion.

For the strap script you should use shutil it's already imported in your local scope.
Example from ducky:
```py
# ducky files
for i in ["ducky.lja", "duckyline.lja", "duckycat.lja"]:
    shutil.copyfile(i, path.join(root, "bin", i))

try:
    mkdir(path.join(root, "bin/ducky"))
except FileExistsError:
    pass
for i in [
    "ducky.py",
    "duckyline.py",
    "duckyload.py",
    "duckycat.py",
]:
    shutil.copyfile(i, path.join(root, "bin/ducky", i))

shutil.copyfile("help_ducky.txt", path.join(root, "usr/share/help", "ducky.txt"))
shutil.copyfile("help_duckyline.txt", path.join(root, "usr/share/help", "duckyline.txt"))
shutil.copyfile("help_duckycat.txt", path.join(root, "usr/share/help", "duckycat.txt"))

# hid files
try:
    mkdir(path.join(root, "lib/adafruit_hid"))
except FileExistsError:
    pass

for i in [
    "__init__.mpy",
    "consumer_control_code.mpy",
    "consumer_control.mpy",
    "keyboard_layout_base.mpy",
    "keyboard_layout_us.mpy",
    "keyboard.mpy",
    "keycode.mpy",
    "mouse.mpy",
]:
    shutil.copyfile(i, path.join(root, "lib/adafruit_hid", i))
```

In path.join, do not end or start the string with "/".

JPKG currently only supports 3 commands:
 - `jpkg install`
 - `jpkg uninstall`
 - `jpkg list`

However performing on multiple packages at once is supported.


      /---------------------|
----<| Devices and consoles |
      \---------------------|

Beryllium os devices are stored in `be.devices`.
It's a dict.
To refer to device ttyACM0, we would go to `be.devices["ttyACM"][0]`.
To refer to device ov3660_0, we would go to `be.devices["ov3660"][0]`.

Every device category is an additional dict, so type(be.devices["ov3660"]) is dict.

To add / remove devices use the mknod and rmnod commands.
Make sure devices are deinitialized before removing.

To load a module and create a device out of it, use the modprobe command.

Modprobe expects the module to be available and path and that it has a class with the same name
as the module.

Module aliases may be applied during modprobe like:
```
    modprobe driver_wifi as network
```
This ensures that this driver will be loaded into `/dev/network0`.

To get a new blank device, use the following command snippet:
```py
be.based.run("mknod " + vr("dev"))
vr("node", be.api.getvar("return"))
be.api.subscript("/bin/stringproccessing/devid.py")
```

If this is successful, you should now have in your process's storage the container variables
"dev_id" and "dev_name" (`vr("dev_id")` & `vr("dev_name")`).

To access your new device you can then just use:
`be.devices[vr("dev_name")][vr("dev_id")]`

Consoles are stored in `pv[0]["consoles"]`.
It's highly advised you do not interface with them unless you have studied all existing
console code.

The internal `terminal` command and the api function to check consoles
are gonna be your best friends.


      /-----------------------------|
----<| Contributions to the project |
      \-----------------------------|

To commit to the project, make sure you have pre-commit installed.
On Arch Linux the packaged is named "python-pre-commit".
And then you have run `pre-commit install` inside of the repo.
Then upon committing changes, they will be immediately formatted.