# Building and flashing

How kodeOS builds into an image, where the board settings come from, and the two ways to flash a Kode Dot: esptool or OpenOCD.

## What it is

kodeOS is an ESP-IDF project. `idf.py build` at the root of the repository
produces `build/kodeos.bin`, and `idf.py flash` writes it to a Kode Dot over
the USB-C cable. The first build of any project, and the generic flash, are
on `build/setup`, `build/first-build` and `build/flash`. This page covers what
is particular to the kodeOS tree: how the root build file is wired, which
file holds which setting, the storage image, and the JTAG route.

There are two independent ways to write the flash. One is esptool over the
serial port. The other is OpenOCD over JTAG. Both use the same USB-C cable,
because the ESP32-P4 has a USB-Serial/JTAG peripheral that presents a serial
port and a JTAG interface together. Use the serial way unless your editor is
set to flash through JTAG.

## How it is connected

The root `CMakeLists.txt` does four jobs before ESP-IDF takes over. It adds
the SDK's components, reads two default files, sets the version and the chip,
and stages the FAT image that goes into the `storage` partition.

```mermaid
flowchart LR
  subgraph settings["Settings"]
    board["kbsp<br/>sdkconfig.board"] --> cfg["sdkconfig<br/>generated"]
    defaults["sdkconfig.defaults"] --> cfg
  end
  subgraph firmware["Firmware"]
    comps["sdk/components"] -- "EXTRA_COMPONENT_DIRS" --> bin["build/kodeos.bin"]
  end
  subgraph fatimg["FAT image"]
    pack["kui/kody/pack"] --> stage["build/storage_stage"]
    img["storage_image"] --> stage
    stage -- "fatfs_create_<br/>spiflash_image" --> fat["build/storage.bin"]
  end
  cfg --> bin
  class bin,fat store
```

*From the sources to the images*

The SDK is a git submodule at `sdk/`, with the ESP32-C5's firmware nested
inside it. Fresh clones need `git submodule update --init --recursive`.
`components/` in the kodeOS tree holds only kgui, the screens that sit on top
of the SDK.

The board facts (flash size, PSRAM mode, cache, console, the SDIO
wiring to the C5) live in `sdkconfig.board` next to the partition table in
`sdk/components/kbsp`, so every project reads one copy. See `sdk/kbsp`.

## Specifications

### The commands

| Command | What it does |
| --- | --- |
| `idf.py build` | Builds `build/kodeos.bin`, the bootloader, the partition table and `build/storage.bin` |
| `idf.py -p PORT flash` | Writes the bootloader at `0x2000`, the partition table at `0x8000`, otadata at `0x1B000` and kodeOS at `0x20000`. Skips images unchanged since the last flash |
| `idf.py -p PORT flash -a` | Writes every image, changed or not. A blank or erased chip needs it |
| `idf.py -p PORT storage-flash` | Replaces the whole `storage` partition with `build/storage.bin` |
| `tools/jtag_flash.sh [build-dir]` | Writes every image of the build over JTAG with one OpenOCD that exits afterwards. The build directory defaults to `build` |
| `tools/check_layers.sh` | The include rules of every layer. Must exit 0 before a change is done |
| `tools/host_tests.sh` | The SDK's host suites; see `internals/host-tests` |

`PORT` is the serial port of the Kode Dot. On macOS it looks like
`/dev/cu.usbmodem*`. `idf.py flash` compares each image with the
`*_flashed.bin` files that the previous flash left in the build directory,
verifies the result, and falls back to a full write when the check fails.

### The root build file

| Thing | Value |
| --- | --- |
| `EXTRA_COMPONENT_DIRS` | `sdk/components/`. The folder `sdk/c5/` is an ESP-IDF project, not a component, so it must stay out of any scanned directory |
| `SDKCONFIG_DEFAULTS` | `sdk/components/kbsp/sdkconfig.board` first, then the root `sdkconfig.defaults`. Later files win |
| `PROJECT_VER` | `00.00.23` unless `-DPROJECT_VER=...` is on the command line. This is the version `app_list`, `info` and the boot banner report. A release passes the version of its tag |
| `IDF_TARGET` | `esp32p4`, set before `project()` |
| Partition table | `CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="sdk/components/kbsp/partitions.csv"` in the root `sdkconfig.defaults` |
| Storage stage | `build/storage_stage`, made at configure time: Kody's pack from `sdk/components/kui/kody/pack/` into `kode/`, then `storage_image/` without its `README.md` and `*.commit` files |
| Stamp | `storage_image/c5_slave.commit`, when present, is copied to `build/` and stays out of the FAT |
| `storage.bin` | `fatfs_create_spiflash_image(storage ...)`, not flashed by `idf.py flash` |

A changed pack or image file configures again by itself. A pack regenerated
by `kodygen.py` reaches the image on the next configure.

### The settings files

| File | What it is | What you do |
| --- | --- | --- |
| `sdkconfig.board` | The Kode Dot's settings, shared by every project | Never edit it, never copy it |
| `sdkconfig.defaults` | This project's defaults | Edit it |
| `sdkconfig` | Generated from the two above | Delete it to apply a changed default |

A value already in the generated `sdkconfig` beats a new default, every time.
A project spells out two board facts itself, because ESP-IDF needs them before
it can fetch the shared file:

| Setting | Where | Value |
| --- | --- | --- |
| `CONFIG_PARTITION_TABLE_CUSTOM_FILENAME` | `sdkconfig.defaults` | Root: `sdk/components/kbsp/partitions.csv`. An app next to its SDK submodule: `../sdk/components/kbsp/partitions.csv`. An SDK example: `managed_components/kodediy__kbsp/partitions.csv` |
| `set(IDF_TARGET esp32p4)` | `CMakeLists.txt`, before `project()` | Without it ESP-IDF guesses `esp32` and caches the guess in `CMakeCache.txt` |

### The storage image

| Thing | Value |
| --- | --- |
| Partition | `storage`, FAT, `0x1800000`, 2 MB, wear levelled. About 2,016 KB are usable |
| Ships with | `kode/` (Kody's pack, under 100 KB) and `c5_slave.bin` (the release's C5 image, about 1,310 KB) |
| Bench files | Any other file in `storage_image/`. A release refuses to package one |
| Git | `*.bin`, `*.kfw` and `*.commit` in `storage_image/` are ignored, so the folder looks empty on GitHub |
| `c5_slave.bin` | Built by `tools/build_c5.sh` from `sdk/c5/` with ESP-IDF v6.0.2 |
| `*.kfw` | Built by `tools/mkkfw.py` from the `.bin` of the same name |
| `kodeos.bin` | Does not go in the volume: at about 2.7 MB it does not fit, even alone |

A new kodeOS reaches a board in one of four ways:

- `kcli.py -p PORT install build/kodeos.bin --dst os`.
- `install src="sd:kodeos.bin" dst=os`.
- An `https://` source.
- `install src="pending" dst=os`, after a computer wrote the image into the mailbox.

See `internals/bench-tools` and `internals/command-set`.

### Flashing over JTAG

`tools/jtag_flash.sh` does three things, in order.

| Step | What happens |
| --- | --- |
| Free the adapter | Whatever listens on port 3333 owns the JTAG interface. The script prints that process, sends `kill`, then `kill -9`, and aborts with exit 2 if the port stays held |
| Read the images | Offsets and file names come from `build/flasher_args.json`, in ascending offset order, so a moved partition table needs no change in the script. Each image becomes one `program_esp <file> <offset> verify` |
| Run and leave | One OpenOCD with `board/esp32p4-builtin.cfg` (override with `OPENOCD_CFG`; add a scripts folder with `OPENOCD_SCRIPTS`), then `reset run` and `shutdown` |

Exit codes:

- `0`: flashed and verified.
- `1`: nothing to flash (no `flasher_args.json`, or `openocd` is not on the path).
- `2`: the adapter stayed busy.

A real flash of a 1.25 MB image takes about 13 s, and a no-op verify
about 1 s. JTAG and serial are separate paths: one can work while the other
fails.

### The serial port

| Fact | Consequence |
| --- | --- |
| The Kode Dot re-enumerates after every reset | The port name changes. Identify the board by its MAC address with `python -m esptool --chip esp32p4 -p PORT --no-stub read-mac` |
| Opening the console port resets the board | Clearing DTR and RTS before the open does not prevent it. Anything that builds state must run inside one session |
| A busy port is almost always a monitor still open | `lsof /dev/cu.usbmodem*`, kill the holder, retry. macOS has a `tty.*` and a `cu.*` node for each device, and a program holding one makes the other fail with EBUSY |

## Gotchas

> **WARNING**
>
> **A new line in `sdkconfig.defaults` changes nothing.** The generated
> `sdkconfig` already holds a value, and a cached value beats a default. Delete
> `sdkconfig` and build again. A cached `IDF_TARGET` survives even that, because
> it lives in `CMakeCache.txt`: delete the build folder.

> **DANGER**
>
> **`storage-flash` replaces the whole `storage` partition.** Plain
> `idf.py flash` never touches it, so files a person put on the internal volume
> survive a normal flash and vanish with `storage-flash`. Copy them off first
> with `file-get`. The command writes Kody's official pack again.

> **WARNING**
>
> **The flash succeeded and the board boots a sketch, not kodeOS.** A sketch
> uploaded from Arduino, or an image written to the `app_pending` mailbox,
> leaves a boot request that survives the resets a working app produces,
> including the USB reset that ends a flash. `idf.py flash` never writes the
> apps region, so the request stands. Hold BOT and LEFT together for about
> 1 s, which is a hardware reset that clears it. `internals/boot-model` has the
> closed list of resets that keep it.

> **NOTE**
>
> **A JTAG flash fails with `Unsupported DTM version: -1`.** A long-lived OpenOCD
> from an editor still holds port 3333 with a USB handle that went stale when
> the board re-enumerated. Retrying never helps, and while it is attached it
> also hides panics. Run `tools/jtag_flash.sh`, which kills it first.

## Example code

The commands in the order you run them, from the root of the kodeOS
repository. Each step ends in something you can see.

1. Load ESP-IDF v6.1 in the shell, for example `. ~/.espressif/v6.1/esp-idf/export.sh`. The path depends on how you installed it.
2. Fetch the SDK and the C5 firmware once: `git submodule update --init --recursive`. `sdk/c5/CMakeLists.txt` now exists.
3. Build: `idf.py build`. It ends with the size of `build/kodeos.bin`.
4. Find the port with `ls /dev/cu.usbmodem*`, then flash: `idf.py -p PORT flash`. It ends with one `Hash of data verified` per image.
5. Seed the storage volume when you need Kody's pack and the C5 image on the board: `idf.py -p PORT storage-flash`.
6. To use JTAG instead of the serial steps, run `tools/jtag_flash.sh`. It prints `flashed over JTAG; nothing left listening`.
7. Check the layers before you call a change done: `tools/check_layers.sh` exits 0.

## In the SDK

- [`CMakeLists.txt`](https://github.com/kodediy/KodeOS/blob/main/CMakeLists.txt): the root build file, in the kodeOS repository.
- [`tools/jtag_flash.sh`](https://github.com/kodediy/KodeOS/blob/main/tools/jtag_flash.sh): the JTAG flash script.
- [`components/kbsp`](https://github.com/kodediy/kode-sdk/tree/main/components/kbsp): `sdkconfig.board` and `partitions.csv`, the files every project reads.