# Host tests

The unit tests that run on a computer with no board: six Unity and CMock programs for the SDK, how the runner detects lost cases, and the tools' own tests.

## What it is

The host tests are the SDK's unit tests. They compile the real SDK sources for
the `linux` target of ESP-IDF and run them on your computer. Each test is a
**case**: a small function that calls one piece of code and checks what it
did. The cases use Unity for assertions and CMock for stand-ins generated from
a header.

They cover what a bench cannot produce on demand: a switch whose contact
bounces for exactly one sample, a reply that never arrives, a digest that does
not match, a register write that switches the board off before the next line
can be logged. A driver talks to a chip over a bus and a computer has no chip,
so a case hands the driver a stand-in bus that records the bytes written and
answers with bytes the case chooses.

Run them when you change the SDK or kodeOS. If you only write an app, you do
not need them.

## How it is connected

```mermaid
flowchart LR
  subgraph kodeos["KodeOS repository"]
    os["tools/host_tests.sh"]
    tool["tools/tool_tests.sh"] --> py["tools/tests"]
  end
  subgraph sdkrepo["SDK repository"]
    sdk["tools/host_tests.sh"] --> host["tests/host"]
    sdk --> small["tests/host_nvs, _time,<br/>_slot, _net, _pwm"]
  end
  os -- "forwards" --> sdk
  layers["tools/check_layers.sh"] --> rules["include rules"]
```

*Who runs which suite*

`tools/host_tests.sh` in the kodeOS repository is a forwarder. kodeOS owns no
C above the SDK except kgui, whose logic is entangled with LVGL objects, so it
runs `sdk/tools/host_tests.sh` and nothing else. With the submodule missing it
prints `skipped` and exits 0, and with `CI=true` in the environment it fails
instead, for a run that must not pass over nothing. Everything below describes the SDK's script.

The project is split into six programs because C allows one definition of a
function per binary. The main suite fakes or mocks some interfaces that other
suites want to test for real. Taking the stand-in away from the main suite
would break every driver and Active Object case that depends on it, so each
such interface gets a project of its own.

## Specifications

### The six suites

They run in this order. The cases column is the number of `TEST_CASE(` lines
in each project's `cases/` folder.

| Project | Binary | Cases | What it links for real |
| --- | --- | --- | --- |
| `tests/host` | `kode_host_tests` | 2717 | The drivers, the Active Objects, the console, the facade, the kit's pure parts. 145 case files |
| `tests/host_nvs` | `kode_host_nvs` | 22 | `khal_nvs_esp.c` over the real `nvs_flash` |
| `tests/host_time` | `kode_host_time` | 8 | `khal_time_esp.c` over a mocked `esp_timer` |
| `tests/host_slot` | `kode_host_slot` | 34 | `kdrv_slot.c` over a mocked `esp_ota_ops`, with `esp_partition` real |
| `tests/host_net` | `kode_host_net` | 34 | `kdrv_net.c` over mocked `esp_http_client` and `ping_sock` |
| `tests/host_pwm` | `kode_host_pwm` | 18 | `khal_pwm.c` over a mocked `khal_pwm_port.h` |

### The runner

| Thing | Value |
| --- | --- |
| Command | `tools/host_tests.sh`, with ESP-IDF v6.1 loaded and Ruby installed (CMock is a Ruby generator; without Ruby the mocks are silently not produced) |
| Green run | One last line: `ok    N cases across 6 suites, all passing` |
| Failure output | A `-- project --` banner and Unity's own output, so a failing case names its file and line. A build or configure failure prints the last 30 lines of `<project>/build.log` |
| Continues after a failure | Yes. Every project builds and runs, and the closing lines name each one `ok` or `FAIL`. Exit 1 if any failed |
| Failure count | Read from Unity's `N Tests M Failures` line, because a process status holds only 8 bits |
| `IDF_PATH` unset | Stops at once and prints the export line |
| `IDF_TARGET` exported | Unset by the script. Every project builds for `linux`, and `idf.py` refuses a target that contradicts the environment |
| `set-target linux` | Run only when `CONFIG_IDF_TARGET="linux"` is not in the project's `sdkconfig`, because it forces a full rebuild |
| Build folder | `build/` per project; a folder configured by another ESP-IDF is dropped and rebuilt |

### Lost cases

A linker drops a case file that nothing references, and Unity then reports
`OK` over what survived. These checks work together.

| Defence | How it works |
| --- | --- |
| `WHOLE_ARCHIVE` | Every project registers its component with it, so the link keeps every case |
| Declared against ran | The script counts `TEST_CASE(` in `<project>/cases` and compares with the first number of Unity's summary line. A mismatch fails with `N cases declared, M ran - the link dropped some`, per project |
| Case-list stamp | The case and fake lists are globbed, and `CONFIGURE_DEPENDS` is unavailable under ESP-IDF's component file. The script writes the sorted list of `cases/test_*.c` and `main/fakes/fake_*.c` to `build/.case-list` and reconfigures when it moves |
| `test_harness.c` | A deliberately trivial case in its own file. If the link drops files, it goes with them and the count check notices |

A run that looks too clean needs three checks:

- The last line says six suites.
- The total matches the sum of the `TEST_CASE(` counts.
- No project printed `declared, ... ran`.

Symbols from Kconfig are a fourth gap the count cannot see: `kaos` and `kdrv` arrive as loose sources, so `main/CMakeLists.txt` defines each `CONFIG_KODEOS_*` the sources test, and a missing one compiles a block out silently.

### The harness

| Part | What it does |
| --- | --- |
| Fakes, `main/fakes/fake_*.c` | Hand-written stand-ins: NVS, sync, media, time, panic, system, the console transport, a card in a slot with a sparse sector table, the radio link, esp-hosted, BLE, the worker queue, the Wi-Fi events. Controlled through `fakes.h` |
| `fakes_reset()` | The runner calls it before every case, in a fixed order, so no case inherits another's world |
| Mocks, `mocks/*` | CMock output for `khal_i2c`, `khal_gpio`, `khal_clk`, `khal_exp_port`, `kdrv_es8311`, `kdrv_display`, `kdrv_ir`, `kdrv_jpeg`, `kdrv_ppa`, `kdrv_st25r3916`, `kdrv_slot`, `kdrv_net`, `kdrv_kfw`, plus ESP-IDF's shipped mocks for `esp_wifi`, `esp_netif`, `esp_event` and `lwip` |
| Chip drivers | The BQ25896, CW2217, TCA6408A, QMI8658A, QMC5883P, CST820 and buttons drivers are compiled for real over the mocked bus, so Active Object cases drive a register file |
| `qf_support.h` | Starts QF once per binary and gives an Active Object a queue with no task: `qf_support_init()`, `_give_queue()`, `_pump()`, `_tick()`, `_background()`, `_claim_ao()` |
| `ao_spy.h` | A real `QActive` with a real queue and no task, so events arrive as they would on the board |
| Direct dispatch | A case calls an Active Object's dispatch function itself. It tests which event is handled in which state and which transition fires. Priority, queue depth and preemption are properties of the running system and are not tested here |
| Emulated flash | `tests/host/sdkconfig.defaults` and the slot project's declare the board's own partition table, so the apps region really sits at `0xC40000`, `0xBC0000` long, ending at `0x1800000` |
| Power cuts | `CONFIG_ESP_PARTITION_ENABLE_STATS` enables `esp_partition_fail_after()`: the n-th write or erase fails and every later one too. Cases tagged `[power_cut]` cut the directory commit and the boot request write at every step, then read both copies as a boot would |

The main suite does not mock `esp_timer`: klog stamps every line with
`esp_timer_get_time()`, so a mock would need an expectation per log line. The
time suite makes the opposite choice, because its subject is which
`esp_timer` calls the code makes.

### Sanitizers

`tools/host_tests.sh --sanitize` builds each project in `build_san/` with
AddressSanitizer and UndefinedBehaviorSanitizer on the code under test (the
`main` component only), through `tests/sanitize.cmake`. A read past a buffer,
a `bool` holding 232 or a signed overflow stops the case with a report. Leak
detection is off, because cases keep their fixtures for the life of the
process. The plain build is left alone.

### The other runners

| Runner | What it covers | Needs |
| --- | --- | --- |
| `tools/host_tests.sh` | The SDK's C, off the board | ESP-IDF, the `linux` target, Ruby |
| `tools/tool_tests.sh` | Every test under `tools/tests` plus the SDK's `test_check_ram.sh` and `test_check_misra.sh` | Python 3 and Pillow for two of them. No ESP-IDF, no board |
| `tools/check_layers.sh` | The include rules of every layer | A shell |

`tool_tests.sh` lists its tests by hand and fails when a `tools/tests/test_*`
file exists that is not in the list, so a new test cannot sit in the tree
looking like coverage. It sets `PYTHONDONTWRITEBYTECODE` so a stale
`__pycache__` cannot report on code that is no longer in the tree.

The tests it runs:

- `test_kble.py`
- `test_kcli.py`
- `test_kmirror.py`
- `test_kappdata.py`
- `test_kpayload.py`
- `test_mkkfw.sh`
- `test_assemble_release.sh`
- `test_validate_release_tag.sh`
- `test_check_on_main.sh`
- `test_ci_report.sh`
- `test_check_todo_refs.sh`

One header is written twice by hand on purpose. The 14-byte mirror frame header
is in `tools/tests/test_kmirror.py` and in `tests/host/cases/test_kmirror_proto.c`.
The firmware writes it and the Python viewer reads it, and each program's tests
check it against itself only, so both can pass while the programs disagree. A
header generated by one program would repeat that program's errors.
`test_mkkfw.sh` does the same for the `.kfw` header.

## Gotchas

> **WARNING**
>
> **A case fails only on another computer.** The suite is order-coupled: reversing the case
> list breaks about fifteen cases, because some cases depend on state an earlier
> one left behind. The globs are sorted so every machine runs one order, but the
> coupling stays. A failure that depends on order is a finding, so read the case
> twice before you blame the environment, and do not re-run a red suite hoping
> for green.

> **WARNING**
>
> **Mocks vanish, or a stale header fakes a compile error.** Without Ruby, CMock
> produces nothing. After a header changed, delete the project's `build` folder
> (`tests/host*/build`): stale mocks report errors that are not in your code.

> **NOTE**
>
> **A fake that relies on a permission bit passes on your machine and fails as
> root.** Some containers run the suite as root, and root writes into a read-only
> directory.
> The card fake therefore refuses files another way: its mount point sits below
> a symlink that points at itself, so every open fails with `ELOOP`.

> **NOTE**
>
> **A board row has no twin in the host suite.** The `test` command's rows that
> need a real chip stay on the board: `test apps` checks the live category store
> against real NVS, and `test kfw` needs the ROM's real inflater. Where a host
> case can fail on its own and name the broken condition, a coarser copy on the
> board proves nothing extra. `internals/production-test` covers the board rows.

## Example code

Run the whole SDK suite, then one suite with the sanitizers, then the tools'
tests. Run each from the root of the repository you are in.

1. Load ESP-IDF v6.1 and check Ruby: `ruby --version` prints a version.
2. In the SDK repository run `tools/host_tests.sh`. The last line reads `ok    N cases across 6 suites, all passing`.
3. Run it again with memory checking: `tools/host_tests.sh --sanitize`. The builds land in `build_san/`.
4. In the kodeOS repository run `tools/host_tests.sh`. It forwards to the SDK's script, or prints `skipped` when the submodule is missing.
5. Run `tools/tool_tests.sh`. It prints one `ok` line per test file and ends with `tool tests, all passing`.

## In the SDK

- [`tests/host`](https://github.com/kodediy/kode-sdk/tree/main/tests/host): the main suite, its cases, fakes and mocks. The five smaller suites sit beside it.
- [`tools/host_tests.sh`](https://github.com/kodediy/kode-sdk/blob/main/tools/host_tests.sh): the runner, with the lost-case checks.
- [`tools/tool_tests.sh`](https://github.com/kodediy/KodeOS/blob/main/tools/tool_tests.sh): the tools' runner, in the kodeOS repository.