# AO_Autotest

The Active Object that runs the self-test rows of the Kode Dot in order: the groups all, main and user, how a verdict is read, the hold and the kept report.

## What it is

A self-test **row** is one test of one part of the board, such as `pmic` or `sd`. **AO_Autotest** holds the table of rows and runs them in order. It owns no hardware: a row belongs to the Active Object that owns the part it judges. AO_Autotest decides only when each row is asked, how long its owner has to answer, and how the answer is read.

Whatever asks for a run is a **client**. There are three: the console command `test` (`AUTOTEST_CLIENT_CONSOLE`), the Self-test page of the settings screen (`AUTOTEST_CLIENT_GUI`) and AO_TestDot (`AUTOTEST_CLIENT_TEST_DOT`), which asks for one row at a time in the E5 final test (`sdk/kaos/test-dot`). A client posts one request that carries its id, then reads what AO_Autotest publishes: the answer to the request, each row as it starts and ends, the wait for the client, and the end of the run with a tally. Every event carries the id of its client, and a client reads only its own. One run is in flight at a time, whoever asks.

Two words select a group of rows. `main` is what a bare main board answers. `user` is the profile for the owner of a unit. The console's `test component=all` is not a group of this AO: it is the E5 final test, which AO_TestDot runs, asking this AO for its rows by name.

## How it is connected

```mermaid
sequenceDiagram
  autonumber
  participant C as Client
  box "kodeOS"
    participant A as AO_Autotest
    participant O as The row's owner
  end
  C->>A: AUTOTEST_RUN_REQ_SIG
  A-->>C: AUTOTEST_BEGIN_SIG, rows
  A->>O: the row's request
  A-->>C: AUTOTEST_ROW_SIG, running
  O-->>A: TEST_RESULT_SIG
  A-->>C: AUTOTEST_ROW_SIG, result
  A-->>C: AUTOTEST_END_SIG, tally
```

*One run, from the request to the tally*

AO_Autotest in numbers:

| Thing | Value |
| --- | --- |
| Priority | 10, right under AO_Console (11), its first client |
| Queue | 16 events |
| Stack | 4096 bytes |
| Claims | Nothing |
| Compiled under | `KODEOS_AUTOTEST`, off by default and selected by `KODEOS_CONSOLE` |
| States | One |

Waiting for an owner costs nothing: it posts one request, arms one timer and sleeps. Because it runs below the console, the console prints a row's record before the sequencer asks the next row, so the log lines of one row never land above the verdict of the row before.

### The table

`TESTS[]` has one entry per row: `name`, a permanent `id` (`AUTOTEST_ID_*`, never reused because the kept report stores it), the function that runs or asks, a `kind`, its group bits, `budget_ms`, an optional `answer` name (several rows of one owner answer under the owner's name), flags, a `label` and a `problem` sentence for a person. `autotest_row_label()`, `autotest_row_problem()` and `autotest_row_name()` read them by id. A group runs every row whose mask has its bit, in table order.

### Two kinds of row

A row is synchronous or asynchronous.

| Kind | How it runs | Rows |
| --- | --- | --- |
| `TEST_SYNC` | No owner. AO_Autotest runs it in its own step and publishes the verdict at once | `screenshot`, `jpeg`, `clocks`, `mem`, `boot`, `apps`, `kfw`, `id`, `ble_console` |
| `TEST_ASYNC` | It belongs to another AO. AO_Autotest asks the owner and arms the budget, and the owner answers later with `TEST_RESULT_SIG` or `TEST_SKIP_SIG` | `fs`, `sd`, `pmic`, `gauge`, `ioexp`, `imu`, `mag`, `haptic`, `c5`, `ir`, `nfc`, `rfid`, `audio`, `buttons`, `led`, `display`, `tp`, `net` |

The `clocks` row holds the core for tens of milliseconds while it counts the crystal. `psram` is of the asynchronous kind with AO_Autotest as its own owner: it takes the free PSRAM less a reserve, walks it on the `psram_step` timer, publishes `PSRAM_TEST_SIG` with what it found and gives the blocks back when it ends or runs out of budget.

### The run

The rows run one at a time, because several need a person at the board and two rows asking at once would each judge the other's answers. For each row AO_Autotest publishes `AUTOTEST_ROW_SIG` twice, with `AUTOTEST_ROW_RUNNING` and the budget, then with the result. A row whose owner stays silent for its whole budget ends as a timeout and the run goes on. While a run is in flight AO_Autotest subscribes to the verdicts and the step events of the guided rows; while idle it hears nothing but requests. It compares the name a verdict arrives under with the row in flight and ignores any other, with one warning, such as a row answering after its budget.

### The user profile

`user` is not a mask. `order_build()` has its own list, `USER_ORDER`, and looks each id up in the table. A row this build lacks is not in the run. Rows that need nobody come first, then the hold, then the rows that need hands.

The table lists the order of a `user` run.

| Place | Row | Owner |
| --- | --- | --- |
| 1 to 4 | `ioexp`, `clocks`, `imu`, `mag` | AO_IoExp, AO_Autotest, AO_Imu |
| 5 | `haptic` | AO_Haptic |
| 6 | `sd_user`: the card row, answering as `sd`; a slot with no card is skipped | AO_Storage |
| 7 | `c5_user`: the link, a Wi-Fi sweep that completes and the BLE controller, no floor on what the scans hear | AO_Radio |
| 8 | `ir` | AO_Ir |
| 9, 10 | `nfc_field`, `rfid_field`: the part names itself and its field stands for one window | AO_Nfc, AO_Rfid |
| 11 | `audio` | AO_Audio |
| hold | The run waits for its client | |
| 12 | `power_plug`: the charger plugged in, both power chips judged, the charger taken out | AO_Power |
| 13 to 16 | `buttons`, `led`, `display`, `tp` | AO_Buttons, AO_LedMgr, AO_Display, AO_Touch |
| 17, 18 | `nfc_card`, `rfid_card`: any tag held to the back passes | AO_Nfc, AO_Rfid |
| 19, 20 | `psram`, `mem` | AO_Autotest |

A run of `user` holds 20 rows, or 19 without the radio; the hold is not a row. A run holds at most `AUTOTEST_RUN_MAX` (24) entries.

### How a verdict is read

An owner answers pass or fail. AO_Autotest reads that together with what the steps of the row said (`verdict_state()`):

1. A pass is a pass.
2. A fail of a row flagged `ROW_F_SKIP_NO_CARD`, when the first step of the card found no card, is **skipped**.
3. A fail when a step went unanswered and none was answered "no" is a **timeout**: nobody was there to press, or the row had no witness.
4. Any other fail is a **fail**.

`TEST_SKIP_SIG` from an owner ends the row as skipped whatever it says, and a budget that ends with no answer is a timeout.

The step events that set the flags are `BTN_TEST_SIG`, `TOUCH_TEST_SIG`, `HAPTIC_TEST_SIG`, `DISPLAY_TEST_SIG`, `LED_TEST_SIG`, `POWER_TEST_SIG` and `STORAGE_TEST_SIG`. The owners publish them, and AO_Autotest does not forward them.

**Whose run it is.** The two guided scripts, buttons and touch, can also be started by another client (`buttons_test_request_from()`, `touch_test_request_from()`). Their steps and their verdict carry an `origin`, and AO_Autotest counts only those of `TEST_ORIGIN_AUTOTEST`: a step of another client's run does not mark the row now running, and a verdict of another origin is ignored (the log says so at debug level). The steps of the other owners have no origin, because only AO_Autotest asks for them.

### The hold

In the `user` profile the walk publishes `AUTOTEST_HOLD_SIG`, arms the budget timer for `HOLD_MS` (300000 ms, 5 min) and waits for `autotest_continue()` from the client that owns the run.

The console does not wait for a person: it prints `>|hold=continue` and continues at once. With no answer in 5 min the run ends with `cancelled` set. If the pool has no event for the hold, nobody could answer, so the run goes on.

### The cancel

`autotest_cancel_request()` ends the run after the row in flight and at once during a hold. A row is never torn down halfway.

### The kept report

When a `user` run ends and was not cancelled, AO_Autotest keeps one blob in NVS: namespace `ktest`, key `last`, version 1. It holds each row's id and final state, and the UTC time of the end (0 when the clock was not set).

A cancelled run keeps nothing. `all` and `main` keep nothing. A run of one row by name replaces that row's state in the report and leaves the rest. `autotest_report_request()` publishes `AUTOTEST_REPORT_SIG` once per row and a closing marker with the row count, the problem count (fail or timeout) and the time.

### The budget timer

`QTimeEvt_disarm()` returns false both for a timer never armed and for one that has expired with its event already queued. Without a guard that late event would time out the next row. AO_Autotest counts each such case when it stops the timer and drops that many `AUTOTEST_BUDGET_SIG` events. The hold uses the same timer and guard.

### Resting with the board

With `CONFIG_KODE_LOWPOWER`, AO_Autotest joins the rounds of AO_LowPower as `autotest` (`sdk/kaos/lowpower`). It owns no device, so it has nothing to put away or to restore. It answers a round busy with `test row` while a run is in flight, a hold included, because the operator at the pad would find the board gone in the middle of a row. It answers ready otherwise.

## Specifications

### Signals in

| Signal | Event | From |
| --- | --- | --- |
| `AUTOTEST_RUN_REQ_SIG` | `AutotestRunReqEvt` (`client`, `group`, `name`) | `autotest_run_request()` |
| `AUTOTEST_CANCEL_REQ_SIG`, `AUTOTEST_CONTINUE_SIG`, `AUTOTEST_REPORT_REQ_SIG` | `AutotestClientEvt` (`client`) | `autotest_cancel_request()`, `autotest_continue()`, `autotest_report_request()` |
| `TEST_RESULT_SIG`, `TEST_SKIP_SIG` | `TestResultEvt` (`name`, `ok`) | The owner of the row, during a run only |
| `BTN_TEST_SIG`, `TOUCH_TEST_SIG`, `HAPTIC_TEST_SIG`, `DISPLAY_TEST_SIG`, `LED_TEST_SIG`, `POWER_TEST_SIG`, `STORAGE_TEST_SIG` | The owners' step events | The owners, during a run only |
| `AUTOTEST_BUDGET_SIG`, `AUTOTEST_PSRAM_STEP_SIG` | none (timers) | Itself |
| `LOWPOWER_ROUND_SIG` | `LowPowerRoundEvt` | AO_LowPower, with `CONFIG_KODE_LOWPOWER`: answered busy `test row` during a run |

### Signals out

| Signal | Event | When |
| --- | --- | --- |
| `AUTOTEST_BEGIN_SIG` | `AutotestBeginEvt` (`client`, `status`, `rows`) | The answer to a run request: `AUTOTEST_TAKEN` (0), `AUTOTEST_BUSY` (1) or `AUTOTEST_UNKNOWN` (2). A hold is not a row |
| `AUTOTEST_ROW_SIG` | `AutotestRowEvt` (`client`, `id`, `pos`, `state`, `name`, `budget_ms`) | A row starts and ends. `budget_ms` is 0 for a row judged on the spot and in every state but running |
| `AUTOTEST_HOLD_SIG` | `AutotestHoldEvt` (`client`) | A `user` run waits for its client |
| `AUTOTEST_END_SIG` | `AutotestEndEvt` (`client`, `tally[5]`, `cancelled`) | The run is over. The tally has one count per state, capped at 255 |
| `AUTOTEST_REPORT_SIG` | `AutotestReportEvt` (`client`, `last`, `id`, `state`, `rows`, `problems`, `when`) | The answer to a report request |
| `PSRAM_TEST_SIG` | `PsramTestEvt` | What the PSRAM row found, once per run, before its verdict |

If the pool has no event for `AUTOTEST_BEGIN_SIG`, AO_Autotest logs an error and does not start the run. A client counts the rows that end against the number in that event.

### The groups and states

| Group | Constant | Rows |
| --- | --- | --- |
| `main` | `AUTOTEST_G_MAIN` | `pmic`, `gauge`, `ioexp`, `clocks`, `imu`, `mag` |
| `user` | `AUTOTEST_G_USER` | The ordered list above |
| By name | `AUTOTEST_G_NONE` | Any row, including the ones in no group: `haptic`, `sd`, `c5`, `ir`, `nfc`, `rfid`, `audio`, `buttons`, `led`, `display`, `tp`, `psram`, `mem`, `fs`, `net`, `boot`, `apps`, `kfw`, `id`, `screenshot`, `jpeg`, `ble_console` and the user rows |

A row ends in one of five states: `AUTOTEST_ROW_RUNNING` (0), `_PASS` (1), `_FAIL` (2), `_TIMEOUT` (3) or `_SKIPPED` (4). The default budget of a row is `KODE_TEST_ROW_BUDGET_MS`, 5000 ms, and the slower rows have budgets of their own in `kqp_config.h`.

### The calls

| Call | What it does |
| --- | --- |
| `autotest_run_request(client, group, name)` | Asks for a run. `group` is `AUTOTEST_G_MAIN` or `_USER`, or `_NONE` with a row name |
| `autotest_cancel_request(client)` | Ends the run after the row in flight. Only the run's client may |
| `autotest_continue(client)` | Answers a hold |
| `autotest_report_request(client)` | Publishes the kept report. Allowed during a run |
| `autotest_row_label(id)`, `autotest_row_problem(id)`, `autotest_row_name(id)` | Constant strings; empty for an id this build lacks |

The console verbs are `test [component=…]` and `test_report`. The only stored data is the report blob. The option is `KODEOS_AUTOTEST`.

## Gotchas

> **WARNING**
>
> **A row that nobody answered ends as a timeout and says nothing about the hardware.** `fail=` in the summary counts failed rows and unjudged ones together, and `timeout=` says how many of them nobody judged. Ship a unit on `fail=0`, and read `timeout=` to learn whether a person was missing.

> **WARNING**
>
> **A second run request is refused while one is in flight.** The answer is `AUTOTEST_BUSY` and the console prints `>err code=4 msg=test`. A cancel takes effect after the row in flight, so it can take as long as that row's budget.

> **NOTE**
>
> **The guided rows need hands.** `buttons`, `led`, `display`, `tp` and the card rows wait on the pad or the glass for up to their budget. The `led` and `display` rows take the top button as yes and the bottom button as no. `sd_verify` is destructive, and it is not a row: it publishes its own `TEST_RESULT_SIG` that an idle sequencer never sees.

> **NOTE**
>
> **A `user` run keeps its report only when it ends whole.** A cancelled run leaves the report of the run before.

## Example code

A run answers with events and has no return value, so the console is where to see one. Each step ends in something you can read.

1. Open the console on the cable and type `test component=main`. Six records arrive, one `>|test=<row> result=<state>` per row (`pmic`, `gauge`, `ioexp`, `clocks`, `imu`, `mag`).
2. Read the last two lines: `>|summary pass= fail= timeout=` and the bare `>ok`. A verdict block with `ALL PASSED` or the failing rows precedes them in the log.
3. Type `test component=user` with the unit on a table. The console prints `>|hold=continue` after the rows that need nobody. The rows after it ask for presses on the pad and taps on the glass.
4. Type `test_report`. One `>|row="<name>" result=<state>` record per row arrives, then `>|when=<utc> rows=<n> problems=<n>`.

## In the SDK

- [`components/kaos/autotest`](https://github.com/kodediy/kode-sdk/tree/main/components/kaos/autotest): `autotest_ao.c` (the table, the walk, the report) and `autotest_psram.c` (the PSRAM walk).
- [`kqp_signals.h`](https://github.com/kodediy/kode-sdk/blob/main/components/kqp/include/kqp_signals.h): the events, the group bits, the row ids and the states.
- [`kaos.h`](https://github.com/kodediy/kode-sdk/blob/main/components/kaos/core/include/kaos.h): the five calls.