# Bench tools

The computer-side tools that talk to a Kode Dot: the console over cable and Bluetooth, the live mirror, the app container and the release scripts.

## What it is

The bench is the desk where you test a Kode Dot by hand, with a cable and a
computer. The bench tools are the Python programs in the `tools/` folder of
the kodeOS repository. They run on the computer, and they talk to a board that
runs kodeOS. With them you send console commands, install an app, take a
screenshot, copy a file, change a setting and watch the screen live.

Each console client follows the console protocol exactly: one line is one
command, every answer line starts with `>`, a record starts with `>|`, a
terminator is `>ok` or `>err code=N msg=token`, and every other line is a `!`
log the computer did not ask for. A script of your own can copy what a client
does instead of rewriting the line framing, the echo removal and the ANSI
stripping. `internals/console-protocol` has the exact form.

## How it is connected

```mermaid
flowchart LR
  subgraph pc["On your computer"]
    kcli["kcli.py"]
    kble["kble.py"]
    kmirror["kmirror.py"]
    kpayload["kpayload.py"]
    kappdata["kappdata.py"]
    mkkfw["mkkfw.py"]
    kcli --> kpayload
    kble --> kpayload
    kcli --> kappdata
    mkkfw --> kappdata
  end
  subgraph dot["On the Kode Dot"]
    board["AO_Console"]
  end
  kcli -- "USB-C or charging base" --> board
  kble -- "Bluetooth LE" --> board
  kmirror -- "either road" --> board
  class kcli,kble,kmirror,mkkfw app
  class board ao
```

*Which tool reaches the board by which road*

`kpayload.py` and `kappdata.py` are modules, not commands.

`kcli.py` needs
pyserial, which every ESP-IDF environment carries, so run it in a terminal
where ESP-IDF is loaded. `kble.py` and the Bluetooth mode of `kmirror.py`
build a private Python environment with bleak the first time they run. Both
Bluetooth tools are written for macOS.

## Specifications

### The tools

| Tool | What it does | Main commands |
| --- | --- | --- |
| `kcli.py` | The console over the USB-C cable or the charging base. The reference client of the protocol | `-p PORT ping`, `cmd "app_list"`, `install FILE --dst D [--no-run]` (D is app, os, c5 or pending), `screenshot -o FILE`, `file-get NAME -o FILE`, `file-put LOCAL REMOTE`, `assets-install DIR`, `config`, `config-export -o FILE`, `config-import FILE`, `appdata` with `ls`, `info`, `rm` or `put` |
| `kble.py` | The same console over Bluetooth. A bridge process keeps the link open between two commands | `term`, `cmd "app_list"`, `install FILE --dst D` (D is app, os or pending), `screenshot -o FILE`, `file-get NAME`, `assets-install DIR`, `detach` |
| `kmirror.py` | The glass in a browser tab, live, with click-to-touch and a pad of six buttons | `-p PORT`, `--ble`, `--scale N` (1 or 2), `--q N` (1 to 100), `--open`, `--http-port N` |
| `kpayload.py` | Shared code: `quote()`, `fields()`, the base64 record decoder and the PNG writer. Standard library only | Imported, never run |
| `kappdata.py` | Validates an `app.json` and converts an icon to the board's `.k64` format | Imported by `kcli.py` and `mkkfw.py` |
| `kblebundle.py` | What Bluetooth needs on macOS: a Python environment with bleak and a signed `.app` that declares the Bluetooth permission | Imported by `kble.py` and `kmirror.py` |
| `mkkfw.py` | Wraps an ESP-IDF image in a `.kfw` container, optionally with an app's `app.json` and icon | `--target T` (T is os, app or c5) `[--name N] [--no-compress] [--appdata DIR] IN OUT` |

Options every `kcli.py` command takes:

- `-p PORT` (required).
- `-v` to show logs and echo while it waits.
- `--settle S` to wait S seconds after the open so a rebooting board finishes.
- `--steal` to terminate whatever holds a busy port and retry.

`kcli.py cmd` takes several lines, `--timeout S` (default 5 s) and `--gap SECONDS` for a verb that answers `>ok` and then works for a while. `kmirror.py` finds the port itself when exactly one `/dev/cu.usbmodem*` exists.

### The wire behaviour a script can rely on

| Behaviour | Value |
| --- | --- |
| Payload commands | `install` and `file-put` share one conversation: after `>\|ready chunk=N`, one chunk of raw bytes per `>\|ack`. The cable has no flow control of its own, so the sender never sends ahead of an acknowledgement |
| Chunk size | Read from `chunk=` in the ready record; 4096 when absent |
| Wait for `>\|ready` | `READY_TIMEOUT` 12 s, then one reopen of the port and a repeat of the request |
| Wait for the close after the last byte | `CLOSE_TIMEOUT` 30 s: CRC, image verify and directory commit |
| `install --dst c5` | A deadline on silence, `C5_CLOSE_IDLE` 45 s. Cable only. `>ok` means the C5 activated the image, not that it runs it |
| `install --dst os` | The board reboots into the new slot right after `>ok`; nothing answers for about 2 s |
| Screenshot and `file-get` | Base64 inside ordinary `>\|d=` records, so a `!` log line in the middle does no harm and the verbs work over Bluetooth too. The idle limit is 15 s on the cable and 30 s over Bluetooth |
| `config-import` | Reads the board's records first in the same session, then sends `config key= value=` only for keys that differ, and prints `key: old -> new` |
| Bluetooth bridge | Unix socket under `~/Library/Caches/kode-kble/`. First attach about 2.3 s, later ones about 0.18 s. It ends on `detach`, when the board drops the link, or when no board is found, never on idle |
| Mirror stream | A 14-byte little-endian header (`seq`, `x`, `y`, `w`, `h` as u16, `len` as u32) then that many JPEG bytes. The viewer sends `mirror_ack seq=N` for every frame, or the stream stops after four |

### The scripts around them

| Script | What it does | Page |
| --- | --- | --- |
| `jtag_flash.sh` | Writes the flash over JTAG with one OpenOCD that exits | `internals/building-and-flashing` |
| `build_c5.sh` | Builds the ESP32-C5 firmware into `storage_image/c5_slave.bin` and stamps its commit | `internals/release-pipeline` |
| `assemble_release.sh`, `validate_release_tag.sh`, `check_on_main.sh` | Make the release files, check a tag, check a commit is on `main` | `internals/release-pipeline` |
| `check_layers.sh` | The include rules of every layer | `internals/building-and-flashing` |
| `host_tests.sh`, `tool_tests.sh` | The SDK's host suites; the tools' own tests | `internals/host-tests` |
| `check_os_size.sh`, `ci_report.sh`, `check_todo_refs.sh` | Image size against its ceiling, one Markdown table of the build's four results, the to-do marker rule | `internals/test-pipeline` |
| `ram_budget.txt`, `misra_baseline.txt` | The internal RAM budget and the MISRA baseline that the build's checks read | `internals/test-pipeline` |
| `release/` | `flash.sh`, `FACTORY_README.txt` and `THIRD_PARTY_NOTICES.txt` for the factory pack | `internals/release-pipeline` |

### Reading a core dump

`idf.py -p PORT coredump-info` reads the `coredump` partition at `0xC20000`
(128 KB) over esptool. It needs the ELF of the image that crashed:

- `info` reports `coredump=` with `owner=kodeos`: use `build/kodeos.elf`.
- `owner=other`: the dump belongs to a sketch or another build, and you need
  that image's ELF.

For a QP/C assert, `pc=` points inside `panic_abort`, and
the backtrace names the failing code. Clear a dump after reading it with
`esptool erase-region 0xC20000 0x20000`.

## Gotchas

> **WARNING**
>
> **A loop of separate `kcli.py` calls reports "no network" for everything.**
> Opening the console port resets the board, so each call tests a board that
> has rebooted a moment ago. Anything that builds state, a Wi-Fi join or an MQTT session,
> runs inside one session: give `kcli.py cmd` several lines, or keep one pyserial
> session in a scratch file outside the repository.

> **WARNING**
>
> **`kcli.py` starts your app instead of kodeOS.** An app that came through the
> `app_pending` mailbox or an Arduino upload holds a sticky boot request, and
> opening the port is a reset that request survives. Hold BOT and LEFT for about
> 1 s first, or read `boot_info` once you have a console. `internals/boot-model`
> explains the request.

> **NOTE**
>
> **A screenshot over the cable shows a freshly booted screen.** The attach
> reset the board. Use `kble.py screenshot` (about 22 s, no reset) when the
> screen as it was is the question, and `kcli.py --steal --settle 5 screenshot`
> (about 3 s) when it is not. Leave a pause between two captures: one taken
> right after another can mix in bands that are seconds old. After any large
> transfer, check `log_dropped=0` in `ps` and `tx_drop=0` in `ble_info`.

> **NOTE**
>
> **Bluetooth attaches, then drops after half a second.** A computer that paired
> with a different build keeps its half of the bond. Make the computer forget the
> board, then run `ble_unpair` on the cable to clear the board's half. macOS also
> caches the peripheral's name, so `found KODE-DOT` while the board advertises
> its serial number comes from that cache. Only one Bluetooth central holds the
> board at a time; `pkill -f 'kble.*--hold'` ends a stuck bridge.

## Example code

Install an app with its icon and name, then look at the glass. Run each line
from the root of the kodeOS repository, with `PORT` replaced by your port.

1. Wrap the image with its metadata: `python3 tools/mkkfw.py --target app --name scope --appdata ./scope-folder build/scope.bin scope.kfw`. One line prints the sizes of the image, the container, the `app.json` and the icon.
2. Install it: `python3 tools/kcli.py -p PORT --settle 5 install scope.kfw --dst app`. The last line reads `installed and booting`.
3. Capture the glass: `python3 tools/kcli.py -p PORT --steal --settle 5 screenshot -o shot.png`. It prints `shot.png: WIDTHxHEIGHT, N bytes`.
4. Watch it live: `python3 tools/kmirror.py -p PORT --open`. A browser tab opens on `http://127.0.0.1:8420/`; a click is a tap.
5. Run the tools' own tests, which need no board: `tools/tool_tests.sh`. It ends with `tool tests, all passing`.

## In the SDK

- [`tools`](https://github.com/kodediy/KodeOS/tree/main/tools): every script on this page, in the kodeOS repository.
- [`tools/tests`](https://github.com/kodediy/KodeOS/tree/main/tools/tests): the tests for the tools. They need no board and no ESP-IDF.
- [`components/kaos/console`](https://github.com/kodediy/kode-sdk/tree/main/components/kaos/console): AO_Console, the board side of everything the clients send.