kodeOS internals

Apps region

How kodeOS stores installed apps in flash: the region, the mailbox, install and eviction, and the app data folder with the icon and colour of each app.

What it is

The apps region is the part of the flash where kodeOS keeps the apps you install. It is the apps partition, 0xC40000, 11.75 MB, and it holds up to 64 apps. Each app is a complete firmware image. The bootloader starts one in place, without copying it, so installing an app is writing it once and starting it is a reset.

Three things sit around the region.

• The mailbox (app_pending, 6 MB at 0x1A00000) is where a computer leaves an image with one esptool command.
• The boot request is the sector at the start of the region that says which app starts next (internals/boot-model).
• The app data folder is a small folder on the FAT volume or the microSD card that gives an app its icon, colour, version and description without rebuilding it.

sdk/kdrv/apps has the byte formats: the directory, the request, the .kfw container and the manifest. This page is how kodeOS uses them. AO_AppMgr owns the content of the region and the mailbox (KBSP_RES_APPS_REGION, KBSP_RES_APP_PENDING). AO_Boot owns the boot request sector (KBSP_RES_APPS_BOOTREQ). AO_Storage reads the app data folder.

How it is connected

flowchart LR
host["Computer<br/>esptool"]
subgraph board["On the Kode Dot"]
  src["Launcher, console,<br/>URL"]
  mb["app_pending"]
  am["AO_AppMgr"]
  reg["apps region"]
  boot["AO_Boot<br/>boot request"]
  bl["Bootloader<br/>override"]
  fat["FAT<br/>.kode/appsdata"]
  mb -- "adopt" --> am
  src -- "install" --> am
  am -- "write" --> reg
  am -- "request" --> boot
  boot --> bl
  bl -- "jump" --> reg
  am --> fat
end
host -- "write-flash" --> mb
class host ext
class mb,reg,fat store
class am,boot ao

How an install behaves

An install works by name. The identity of an app is the project_name in its image header. An install with the name of an installed app replaces it in one directory commit, so a second install does not use a second entry. A new name is added. The installs that come from a file, the card, a URL, the console or the mailbox all end in the same code in AO_AppMgr.

The allocator is next-fit and the eviction is first in, first out. kdrv_apps_plan() puts a new image right after the newest one by install order, searching to the end of the region and then wrapping to the start. When nothing fits, the old copy of the app being replaced goes first, then the oldest apps until the image fits. So a new version of an installed app costs no other app unless it is larger than the room its old copy leaves. kodeOS keeps no record of which app you used last, so it has nothing else to rank by.

When all 64 directory entries hold other names, the oldest entry goes too. AO_AppMgr logs each eviction, evicting '<name>' to make room; its copy is on the card as a notice or evicting '<name>' to make room; no copy kept as a warning, publishes APP_EVICTED_SIG for it, and commits the directory, with the victims' flags cleared, before it touches the flash. The old copy of the app being replaced is not an eviction: the log says the old copy of the app being replaced makes room for its new version.

An eviction keeps a copy on the card when it can. If an install from a file, the card, a URL or the mailbox must evict, AO_AppMgr first copies each victim to /sdcard/<project>.bin (skipping one that is already there) and then restarts the install. The victim shows in the launcher as a file you can install again. If a copy fails and the install came from the card, the install fails and nothing is evicted. A terminal install cannot pause for this, so its victims are evicted without a copy.

The write is paced. AO_AppMgr writes an image in 32 KB chunks, spaced by a 10 ms gap, so the lower priorities (the screen included) get the CPU between erases. A write holds the cache down while it erases, and nothing in flash runs then.

Adoption copies the mailbox into the region at boot. In its initial transition AO_AppMgr asks whether the mailbox holds something worth copying. If it does, it posts APP_ADOPT_SIG to itself, because a copy of several megabytes must not run inside an initial transition. The copy goes through a static 4 KB buffer.

The app data folder is read by the owner of the media. AO_Storage finds the folder, reads app.json and the icon and posts the answer to kgui as events. A row in the launcher resolves each field on its own. For each one it takes the first source that has a value.

Specifications

The region

The region ends where the FAT begins and crosses the 16 MB mark on the way. That is legal because the cache maps 32-bit addresses (CONFIG_BOOTLOADER_CACHE_32BIT_ADDR_QUAD_FLASH). To grow the region, space has to come from storage.

Adoption of the mailbox

AO_AppMgr consumes the mailbox (it erases the first sector and the intent sector) only when the outcome is final, so a failed directory write is retried at the next boot.

install src="pending" dst=os|app runs the same adoption with the destination named by the command. For dst=app, run=on writes a NEW request, never ALWAYS.

Verification

The app data folder

app.json is one flat object of string values. These are its keys:

Unknown keys are ignored. The whole file is refused with size, syntax or revision on any fault, and the launcher then draws the app as if it had no folder. internals/app-icon has the steps.

Who writes the folder, and what an install does to it:

The install ends only after AO_Storage has answered, so a launch never runs before the folder exists. A write that fails is one warning and the install still succeeds. An erased app takes its folder with it. An install never touches a folder on the card.

Which source gives each field in the launcher:

The identity of a row is always the project name. The folder changes only its label.

The answer to a request for one folder is three posted events, in this order:

1. APPDATA_FIELDS_SIG with every field and two statuses (json_status: ok, none, size, syntax, revision; icon_status: ok, none, size, path, busy).
2. Zero or three APPDATA_ICON_SIG blocks of 4096 bytes each.
3. APPDATA_END_SIG.

AO_Storage takes the three icon blocks before it posts anything, so an icon arrives complete or not at all. While a console upload holds the jumbo pool, the fields event still goes out with icon_status=busy and kgui asks again at the next scan.

kgui keeps one 12 KB PSRAM buffer for each of its 128 list entries (64 installed apps and 64 files), about 1.5 MB, taken once.

Console

Gotchas

Example code

Install an app over the cable, start it and read the board's list. Each command prints what it did.

1. Install and start: python3 tools/kcli.py -p <port> --settle 5 install build/my_app.bin --dst app. The tool prints installed and booting and the app starts.
2. Hold BOT and LEFT for about one second to go back to kodeOS. Apps opens with the app in focus, in its folder if it names a category.
3. List the region: python3 tools/kcli.py -p <port> --settle 5 cmd app_list. One record per installed app comes back, then ok.
4. Start it from the console: python3 tools/kcli.py -p <port> --settle 5 cmd 'app_run name="my_app"'. The name is the project() name of the build.

TBD: these steps have not been run on a Kode Dot from this page.

In the SDK

• components/kdrv/apps: the region driver, kapps_fmt.h, the container decoder and the app.json reader.
• components/kaos/appmgr/src/appmgr_ao.c: install, adoption, eviction and keeping a copy on the card.
• tools/kappdata.py: the host side of the app data folder.