kodeOS internals

Release pipeline

How a version tag becomes a kodeOS release, built by hand from a clean clone of the tag: the order of the three repositories, the checks, the C5 image, the welcome app, the factory pack and the publish.

What it is

A release is a numbered kodeOS that people can download. A maintainer makes one by hand. They push a git tag named vMAJOR.MINOR.PATCH on main, clone that tag into an empty folder, run the checks, build the C5 image, kodeOS and the welcome app, and assemble the release files. They try the factory pack on a Kode Dot, and publish the files as a GitHub release with gh release create.

Four files make a release:

• a factory pack: a zip with the six images and a flashing script;
• the standalone kodeos-vX.bin for updates;
• SHA256SUMS.txt;
• BUILD_INFO.txt, the record of what the release was built from.

The clean clone is the point. A build in a working tree can pick up a file nobody committed, a submodule left on another commit or a dependencies.lock from last week. A clone of the tag holds only what the tag names.

The checks a change goes through before it is merged are on internals/test-pipeline.

How it is connected

flowchart LR
c5["C5 firmware<br/>tag, when it changed"] --> sdk["SDK<br/>tag"]
sdk --> pins["kodeOS: kgui pins,<br/>PROJECT_VER, submodule"]
pins --> tag["kodeOS tag"]
tag --> clone["clean clone<br/>of the tag"]
clone --> build["checks, build_c5.sh,<br/>idf.py, build_welcome.sh"]
build --> dist["assemble_release.sh<br/>BUILD_INFO.txt"]
dist --> try["flash a Kode Dot"]
try --> pub["gh release create"]
class c5,sdk,tag ext
class dist store
class try chip

Three repositories release, always in this order, because each pins the one before it: the C5's firmware (when it changed), the SDK, which pins the C5 as its c5 submodule, and kodeOS, which pins the SDK as its sdk submodule and by tag in kgui's manifest. The SDK tag and the kodeOS tag are two separate numbers.

An annotated tag is its own git object and is not the commit. Resolve a tag to its commit before you trust it: git rev-parse TAG^{commit}. A pin resolved against the wrong object builds code nobody reviewed.

dependencies.lock is not in the repository, so the tag named in kgui's manifest alone decides what a clean clone builds. If that tag does not exist, the build fails.

Specifications

The order of a kodeOS release

1. If the C5's firmware changed, release it first (below). Then move the SDK's c5 submodule to that commit.
2. Release the SDK (below): tag vAA.BB.CC on its main.
3. In kodeOS, on a branch, set the four version: pins of components/kgui/idf_component.yml (kodediy/kui, kodediy/kode, kodediy/kbsp, kodediy/kdrv) to that tag. They are the only per-release manifest edit in the repository.
4. Set PROJECT_VER in the root CMakeLists.txt to the new kodeOS number, and move the sdk submodule to the commit the SDK tag names.
5. Run the checks (internals/test-pipeline), open the pull request and merge it.
6. Tag vXX.YY.ZZ on the merged commit of main and push the tag.
7. Build the release from a clean clone of the tag, try it and publish it (below).

The build from a clean clone

From an empty folder. Each command stops the release when it fails.

The C5 image comes before kodeOS's build, because kodeOS's build puts it in the FAT image.

The scripts

build_c5.sh needs ESP-IDF v6.0.2, because the C5 firmware builds with it. Host and slave must run the same esp-hosted version, so building from the pinned commit is what keeps the pair true. If idf.py is a shell function (some installers define it so), both build scripts run $IDF_PATH/tools/idf.py.

The welcome app lives in kode-app-onboarding, a private repository, so build_welcome.sh needs a key with read access to it. A fork points welcome.pin at a welcome app of its own.

What assemble_release.sh checks and writes

Files are copied by name from an explicit list, so a key or a stray file in the folder cannot reach a release file.

BUILD_INFO.txt

Written in the clone's root after the assembly:

{
  echo "kodeOS XX.YY.ZZ"
  echo "kodeos commit  $(git rev-parse HEAD)"
  echo "sdk commit     $(git -C sdk rev-parse HEAD)"
  echo "c5 stamp       $(cat build/c5_slave.commit)"
  echo "welcome pin    $(cat apps_image/welcome.pin)"
  echo "welcome stamp  $(cat apps_image/*.commit)"
  echo "ESP-IDF        $(python3 -c 'import json; print(json.load(open("build/project_description.json"))["git_revision"])')"
  echo
  echo "--- dependencies.lock"
  cat dependencies.lock
} > dist/BUILD_INFO.txt

The resolved dependencies.lock is the only record of which component versions the release was built with, because the repository keeps none.

The factory pack

flash.sh PORT needs Python 3 with esptool 5 or newer and checks the version before it touches the board. It makes one call: esptool --chip esp32p4 -p PORT write-flash --erase-all with these images.

The erase removes everything on the chip: settings, remembered networks, BLE bonds, installed apps, the mailbox and every file on the internal volume. Afterwards otadata is blank, so the board boots factory, and the apps region holds the welcome app with no boot request. boot_info answers running=factory and otadata=blank. See internals/boot-model.

Trying it on a Kode Dot

Before publishing, flash a unit with the pack the release will carry and read it back over the cable.

Publishing

--verify-tag refuses a tag that is not on GitHub, so a release never names a tag that exists only on one computer.

The C5 firmware's release

Only an admin can push a v* tag on kode_slave_c5 (the ruleset "Release tags: admins only"). From a clean clone of the tag, on its main:

1. With ESP-IDF v6.0.2: idf.py -B build_kode set-target esp32c5 build and tools/check_image.sh build_kode. Copy kode_c5.bin, the bootloader, the partition table, ota_data_initial.bin and flasher_args.json into dist/, and write dist/c5_slave.commit as <commit> <sha256 of kode_c5.bin>, the format build_c5.sh writes in kodeOS.
2. With ESP-IDF v6.1: the closed build into build_closed and tools/check_image.sh build_closed closed (internals/test-pipeline).
3. SIGN_TOKEN=<token> tools/sign_closed.py build_closed kode_c5-closed. The factory's signing service holds the two Secure Boot keys and signs the bootloader and the app; the script checks both signatures against the keys in pki/ before it writes anything. It needs espsecure on the path.
4. Zip the folder as dist/kode_c5-closed-vC.zip, copy its kode_c5.bin to dist/kode_c5-closed.bin, and write SHA256SUMS.txt over dist/.
5. Publish with the command above.

kodeOS builds the C5 image itself, from the commit the SDK pins, and does not read this release. The release is the record of what that commit produces, and the production line takes the closed image from it: the line's station writes it to a virgin C5, and its first boot closes the C5 for good.

Gotchas

Example code

A dry run of a release on your own computer, from a clean clone of main, with no tag. It builds version 00.00.00, and the pin check may find the submodule ahead of the last tag, which is normal between two releases.

1. Check the tag format: tools/validate_release_tag.sh v0.1.0 prints 0.1.0. v1.2.3-rc1 prints an error and exits 1.
2. Load ESP-IDF v6.0.2 and build the C5 image: tools/build_c5.sh. It prints the commit and size, and writes storage_image/c5_slave.commit.
3. Load ESP-IDF v6.1 and build kodeOS: idf.py -DPROJECT_VER=00.00.00 build.
4. Build the welcome app: tools/build_welcome.sh.
5. Assemble: tools/assemble_release.sh 00.00.00. It lists the zip, the bin and SHA256SUMS.txt in dist/.
6. Check the sums: cd dist && sha256sum -c SHA256SUMS.txt reports OK twice.
7. Flash a test unit from the pack: unzip it and run ./flash.sh PORT. The last lines tell you to run boot_info and look for running=factory.

In the SDK

• tools/assemble_release.sh: the release files, with tools/release holding the pack's templates.
• tools/build_welcome.sh and apps_image: the welcome app's pin and build.
• components/kdrv/apps: kapps_fmt.h, where the mailbox ceiling the size check reads is defined.