kodeOS internals

Host tests

The unit tests that run on a computer with no board: six Unity and CMock programs for the SDK, how the runner detects lost cases, and the tools' own tests.

What it is

The host tests are the SDK's unit tests. They compile the real SDK sources for the linux target of ESP-IDF and run them on your computer. Each test is a case: a small function that calls one piece of code and checks what it did. The cases use Unity for assertions and CMock for stand-ins generated from a header.

They cover what a bench cannot produce on demand: a switch whose contact bounces for exactly one sample, a reply that never arrives, a digest that does not match, a register write that switches the board off before the next line can be logged. A driver talks to a chip over a bus and a computer has no chip, so a case hands the driver a stand-in bus that records the bytes written and answers with bytes the case chooses.

Run them when you change the SDK or kodeOS. If you only write an app, you do not need them.

How it is connected

flowchart LR
subgraph kodeos["KodeOS repository"]
  os["tools/host_tests.sh"]
  tool["tools/tool_tests.sh"] --> py["tools/tests"]
end
subgraph sdkrepo["SDK repository"]
  sdk["tools/host_tests.sh"] --> host["tests/host"]
  sdk --> small["tests/host_nvs, _time,<br/>_slot, _net, _pwm"]
end
os -- "forwards" --> sdk
layers["tools/check_layers.sh"] --> rules["include rules"]
Who runs which suite

tools/host_tests.sh in the kodeOS repository is a forwarder. kodeOS owns no C above the SDK except kgui, whose logic is entangled with LVGL objects, so it runs sdk/tools/host_tests.sh and nothing else. With the submodule missing it prints skipped and exits 0, and with CI=true in the environment it fails instead, for a run that must not pass over nothing. Everything below describes the SDK's script.

The project is split into six programs because C allows one definition of a function per binary. The main suite fakes or mocks some interfaces that other suites want to test for real. Taking the stand-in away from the main suite would break every driver and Active Object case that depends on it, so each such interface gets a project of its own.

Specifications

The six suites

They run in this order. The cases column is the number of TEST_CASE( lines in each project's cases/ folder.

ProjectBinaryCasesWhat it links for real
tests/hostkode_host_tests2717The drivers, the Active Objects, the console, the facade, the kit's pure parts. 145 case files
tests/host_nvskode_host_nvs22khal_nvs_esp.c over the real nvs_flash
tests/host_timekode_host_time8khal_time_esp.c over a mocked esp_timer
tests/host_slotkode_host_slot34kdrv_slot.c over a mocked esp_ota_ops, with esp_partition real
tests/host_netkode_host_net34kdrv_net.c over mocked esp_http_client and ping_sock
tests/host_pwmkode_host_pwm18khal_pwm.c over a mocked khal_pwm_port.h

The runner

ThingValue
Commandtools/host_tests.sh, with ESP-IDF v6.1 loaded and Ruby installed (CMock is a Ruby generator; without Ruby the mocks are silently not produced)
Green runOne last line: ok N cases across 6 suites, all passing
Failure outputA -- project -- banner and Unity's own output, so a failing case names its file and line. A build or configure failure prints the last 30 lines of <project>/build.log
Continues after a failureYes. Every project builds and runs, and the closing lines name each one ok or FAIL. Exit 1 if any failed
Failure countRead from Unity's N Tests M Failures line, because a process status holds only 8 bits
IDF_PATH unsetStops at once and prints the export line
IDF_TARGET exportedUnset by the script. Every project builds for linux, and idf.py refuses a target that contradicts the environment
set-target linuxRun only when CONFIG_IDF_TARGET="linux" is not in the project's sdkconfig, because it forces a full rebuild
Build folderbuild/ per project; a folder configured by another ESP-IDF is dropped and rebuilt

Lost cases

A linker drops a case file that nothing references, and Unity then reports OK over what survived. These checks work together.

DefenceHow it works
WHOLE_ARCHIVEEvery project registers its component with it, so the link keeps every case
Declared against ranThe script counts TEST_CASE( in <project>/cases and compares with the first number of Unity's summary line. A mismatch fails with N cases declared, M ran - the link dropped some, per project
Case-list stampThe case and fake lists are globbed, and CONFIGURE_DEPENDS is unavailable under ESP-IDF's component file. The script writes the sorted list of cases/test_*.c and main/fakes/fake_*.c to build/.case-list and reconfigures when it moves
test_harness.cA deliberately trivial case in its own file. If the link drops files, it goes with them and the count check notices

A run that looks too clean needs three checks:

Symbols from Kconfig are a fourth gap the count cannot see: kaos and kdrv arrive as loose sources, so main/CMakeLists.txt defines each CONFIG_KODEOS_* the sources test, and a missing one compiles a block out silently.

The harness

PartWhat it does
Fakes, main/fakes/fake_*.cHand-written stand-ins: NVS, sync, media, time, panic, system, the console transport, a card in a slot with a sparse sector table, the radio link, esp-hosted, BLE, the worker queue, the Wi-Fi events. Controlled through fakes.h
fakes_reset()The runner calls it before every case, in a fixed order, so no case inherits another's world
Mocks, mocks/*CMock output for khal_i2c, khal_gpio, khal_clk, khal_exp_port, kdrv_es8311, kdrv_display, kdrv_ir, kdrv_jpeg, kdrv_ppa, kdrv_st25r3916, kdrv_slot, kdrv_net, kdrv_kfw, plus ESP-IDF's shipped mocks for esp_wifi, esp_netif, esp_event and lwip
Chip driversThe BQ25896, CW2217, TCA6408A, QMI8658A, QMC5883P, CST820 and buttons drivers are compiled for real over the mocked bus, so Active Object cases drive a register file
qf_support.hStarts QF once per binary and gives an Active Object a queue with no task: qf_support_init(), _give_queue(), _pump(), _tick(), _background(), _claim_ao()
ao_spy.hA real QActive with a real queue and no task, so events arrive as they would on the board
Direct dispatchA case calls an Active Object's dispatch function itself. It tests which event is handled in which state and which transition fires. Priority, queue depth and preemption are properties of the running system and are not tested here
Emulated flashtests/host/sdkconfig.defaults and the slot project's declare the board's own partition table, so the apps region really sits at 0xC40000, 0xBC0000 long, ending at 0x1800000
Power cutsCONFIG_ESP_PARTITION_ENABLE_STATS enables esp_partition_fail_after(): the n-th write or erase fails and every later one too. Cases tagged [power_cut] cut the directory commit and the boot request write at every step, then read both copies as a boot would

The main suite does not mock esp_timer: klog stamps every line with esp_timer_get_time(), so a mock would need an expectation per log line. The time suite makes the opposite choice, because its subject is which esp_timer calls the code makes.

Sanitizers

tools/host_tests.sh --sanitize builds each project in build_san/ with AddressSanitizer and UndefinedBehaviorSanitizer on the code under test (the main component only), through tests/sanitize.cmake. A read past a buffer, a bool holding 232 or a signed overflow stops the case with a report. Leak detection is off, because cases keep their fixtures for the life of the process. The plain build is left alone.

The other runners

RunnerWhat it coversNeeds
tools/host_tests.shThe SDK's C, off the boardESP-IDF, the linux target, Ruby
tools/tool_tests.shEvery test under tools/tests plus the SDK's test_check_ram.sh and test_check_misra.shPython 3 and Pillow for two of them. No ESP-IDF, no board
tools/check_layers.shThe include rules of every layerA shell

tool_tests.sh lists its tests by hand and fails when a tools/tests/test_* file exists that is not in the list, so a new test cannot sit in the tree looking like coverage. It sets PYTHONDONTWRITEBYTECODE so a stale __pycache__ cannot report on code that is no longer in the tree.

The tests it runs:

One header is written twice by hand on purpose. The 14-byte mirror frame header is in tools/tests/test_kmirror.py and in tests/host/cases/test_kmirror_proto.c. The firmware writes it and the Python viewer reads it, and each program's tests check it against itself only, so both can pass while the programs disagree. A header generated by one program would repeat that program's errors. test_mkkfw.sh does the same for the .kfw header.

Gotchas

Example code

Run the whole SDK suite, then one suite with the sanitizers, then the tools' tests. Run each from the root of the repository you are in.

  1. Load ESP-IDF v6.1 and check Ruby: ruby --version prints a version.
  2. In the SDK repository run tools/host_tests.sh. The last line reads ok N cases across 6 suites, all passing.
  3. Run it again with memory checking: tools/host_tests.sh --sanitize. The builds land in build_san/.
  4. In the kodeOS repository run tools/host_tests.sh. It forwards to the SDK's script, or prints skipped when the submodule is missing.
  5. Run tools/tool_tests.sh. It prints one ok line per test file and ends with tool tests, all passing.

In the SDK