kodeOS internals

Buttons and gestures

What every button and every touch gesture does on every kodeOS screen, the buzzes the interface plays, and the recognizer that reads a touch.

What it is

The Kode Dot has six buttons. Four form the pad: up, down, left and right. Two round buttons sit to the right of it on the front: the blue TOP button and, lower, the red BOT button. Together they are the side buttons. This page lists what each button and each touch gesture does on each screen. Every row is an action or nothing (by design), which means the system chooses to do nothing for that input.

The short version:

• The face is home. A tap anywhere, TOP or the pad's right opens Apps. The pad's up, TOP held or a finger that drags down pulls Settings over it. BOT pressed and let go sends the face to Rest. The pad's down does nothing there.
• A pad button moves the cursor and acts when it goes down.
• TOP goes in or confirms and acts when you let go. Held for 0.6 s on an app, it opens the app's details. Held on the Wi-Fi or Bluetooth circle in Settings, it opens their page.
• BOT goes back and acts when you let go. In a folder it goes back to Apps' root, at the root to the face. On the face it sends the face to Rest. On Settings and on a page it goes back one level.
• Hold BOT for 0.6 s to go home, to Apps' root. A hold never locks: the face comes back when you go back from Apps' root, or by itself after the auto-lock time.
• On the face, hold BOT instead. From 2 s a red arc fills along the top, and at 5 s the card "Turn off?" rises.
• At Rest, TOP pressed twice wakes the face. The first press shows "Press ● again", where the blue dot is TOP and beats once a second. BOT and the pad show "Press ● twice". On a cable any button or touch wakes the face at once.
• A finger that moves more than 16 px is not a tap and acts on nothing.
• The interface buzzes lightly when something acts, refuses or arms. The Settings list sets how much.

How it is connected

flowchart LR
btn["AO_Buttons"] -- "BUTTON_EVT_SIG<br/>BUTTON_HOLD_SIG" --> gui["AO_Gui"]
boot["AO_Boot"] -- "HOME_SIG at 0.6 s" --> gui
touch["AO_Touch"] -- "TOUCH_EVT_SIG" --> rec["recognizer<br/>kui_gesture"]
rec -- "tap, claim, rejection" --> gui
console["console: btn, tap, touch, swipe"] -. "injected" .-> gui
gui -- "kode_haptic_play()" --> hap["AO_Haptic"]
class btn ao
class boot ao
class touch ao
class gui ao
class hap ao
class console ext

Where inputs come from

• AO_Buttons publishes each press, release and hold of the six buttons. sdk/kaos/buttons has the debounce and the hold.
• AO_Boot turns a held BOT into HOME_SIG.
• AO_Touch publishes touch samples on change only. Every sample goes through the recognizer of the kit before LVGL reads it, and AO_Gui creates an LVGL input device in event mode so each sample triggers exactly one read.
• gui_face_navigate(), gui_navigate(), gui_settings_navigate() and gui_sheet_navigate() in kgui_ao.c read the buttons of their screens.
• The console's btn, tap, touch and swipe (internals/command-set) drive the same inputs.

An injected press starts no hold, so btn id=bot state=press on the face begins the count of the red arc but never produces HOME_SIG, and btn id=bot state=hold is home alone.

A gesture only records an intent

LVGL calls a click callback inside its input processing, and a repaint there would delete objects the input device is still walking. A callback first ends its press with lv_indev_wait_release() where it must, then records an intent: a back (pending_back), a door, or a tap. The next tick applies it.

The arc counts edges, never the hold

On the face AO_Gui times BOT itself, from its press (bot_hold_ms, on khal_time_ms(), read by gui_off_arc_tick()). BUTTON_HOLD_SIG is not an edge and leaves the count running, although it disarms every armed press. From 2,000 ms (GUI_OFF_ARC_FROM_MS) the red arc fills, and at 5,000 ms (GUI_OFF_ARC_TO_MS) with BOT still down the card "Turn off?" rises. No hold on any screen locks the board.

5 s sits far past the 0.6 s home hold and 11 s below the shortest full reset the charger can make. BOT is the BQ25896's QON pin, and holding it for tQON_RST, 16 to 23 s, with no USB cuts and closes the battery path.

Specifications

Words

The face

gui_face_navigate() reads the buttons in Gui_awake, the awake look of Gui_face.

• A tap is the recognizer's (gui_face_tap()): a press that travelled more than 16 px, or that the recognizer claimed, opens nothing. A tap on Kody does not make him look.
• The press of a side button arms it and the release acts. The hold disarms the press it grew out of.
• BOT posts the door to Rest on the release, because the answer refuses while a button is held. The face goes to Rest only when it is bare: no card, no curtain, no finger, no held button and no write.
• The recognizer claims a drag down that starts anywhere on the face. A swipe left or right opens Apps, and a swipe up is a zone rejection and no tap.
• gui_off_arc_tick() counts the hold from the press: the arc fills toward 5 s, the hold is spent with a mark buzz when the card rises, and the hold disarms the press, so the release opens nothing.
• Kody looks when the face comes back from Apps or from Settings, not on an input of the face.
• The scene of Kody plays on while the face is seen. When the kody_sleep time has gone by, the face goes to Rest. A finger on the screen, a held button, a sheet, a moving screen, a running write and a look of Kody hold the count.

The face at Rest

At Rest the face is dimmed. The state is Gui_rest. Nothing a person does reaches a screen from it. On the battery only TOP, pressed twice, wakes it. On a cable any button or touch wakes it at once.

Gui_rest records the waking button in wake_btn, and Gui_awake swallows the rest of that press (gui_wake_swallow()). Gui_rest takes every touch sample, so LVGL and the recognizer never see it, and the log writes no gesture: line. A touch that is down when the face wakes stays swallowed until the finger lifts.

Apps

The launcher is the rail of folders and apps (internals/gui/launcher). gui_navigate() reads the buttons, in Gui_browsing.

• A pad button that stays down repeats: nothing for 400 ms (KGUI_PAD_HOLD_MS), then one row every 115 ms, and after 8 repeats two rows each time. At an end the repeat stops.
• Kody's door is the empty launcher's switch between Kody and the QR code. It changes no screen and does nothing while "Looking for apps" shows or Kody slides.
• HOME_SIG closes an open folder (gui_level_home()) and leaves Apps at its root. The release goes back nowhere, because the hold disarmed the press. The hint already home shows only for a hold that counts for nothing, such as btn id=bot state=hold with no press.
• A swipe right is claimed 12 px from the landing; the level goes back when the finger lifts, if it went 40 px along the claim (KGUI_SIDE_STEP_PX). A shorter slide is no tap and goes nowhere. It works on the list and on the empty box alike.
• A drag down that starts in the top 44 rows (KGUI_LIST_PULL_BAND) always pulls Settings, wherever the list stands. Lower down it pulls Settings only when the press landed on the list, the first row is the selection, the list is at rest there and nothing has moved it for 500 ms (gui_zone_lock_pull()). If one fails, the drag moves the list.
• The auto-lock does not happen while a sheet is up, a finger rests, a button is held, a write runs or a screen moves.

A launch is one gesture. gui_launch_selection() has two callers: gui_level_enter() on an app row, and Run again on the Stopped card. TOP and a tap on an app row launch it at once, with no card between. Remove sits behind the details and behind a question of its own, so the one act that cannot be undone asks first.

The settings screen

The control centre. gui_settings_navigate() reads the buttons, in Gui_settings.

A page

Wi-Fi, Bluetooth, Settings, Screen, Battery, About, Kody, Self-test and the two lists of choices. gui_sheet_navigate() and gui_sheet_body() read them.

Rows with no value, where left and right do nothing: every row of the Wi-Fi, Bluetooth, Battery and About pages, the Auto-lock row, and every choice in a list. On the Settings list, Sleep after, Kody's scene and Vibration step through their values and wrap, and the switches While charging, Header 3.3 V and Header 5 V have no step. TOP or a tap changes any of them where it stands.

The keypad

The pad moves the focus over the controls of the card, and TOP presses the control with the focus. The ring marks the focus.

Down from the pill or from the Show password row lands on the column where the focus last stood. The Show password row is skipped for a hidden network's name. While the join is out, the focus still moves, the pill is dimmed and no control acts.

A system card

The pad does nothing on a system card. One table serves them all, and the last column says where a card differs.

A hold that reached its end acts once; the Ring stays full and takes no second hold until the card is closed.

The details

The details of an app open over Apps. gui_details_navigate() reads the buttons while they are up.

The hold that opened the details disarmed the press it grew out of, so its release is no answer. The question takes no answer for 400 ms after it is written.

The guided tests and the self-test

During a guided factory test the operator presses what a script asks for. Gui_btn_test and Gui_tp_test swallow BUTTON_EVT_SIG, BUTTON_HOLD_SIG, HOME_SIG and every door, because AO_Buttons, AO_Touch, AO_LedMgr and AO_Display score those very presses. Every input does nothing (by design).

While a run of the Self-test page lasts, AO_Gui is in Gui_selftest on the loading screen.

Haptics

The interface buzzes at the moments that act, refuse and arm. The buzz is the LRA motor on the antenna board, which AO_Haptic drives. kgui_haptic_play(effect) in kgui_haptic.c is the one call every moment makes. It applies the level, keeps silent while a recording would pick the buzz up, spaces the step, and asks AO_Haptic by event with kode_haptic_play().

The setting haptic_ui sets how much of the interface buzzes: off, light or normal, default light. The Vibration row of the Settings list sets it too.

The moments:

• click: a circle acts; Apps goes in on a row; the one answer of a card; a row of a page that acts; the Auto-lock row and each change of a Screen setting; a key of the keypad that typed, deleted or changed the mode; Kody leaves for the QR code and comes back.
• thud: the pad past the first or last row, and a finger that takes the list past an end, once for each pull; a key the full text refuses; BOT held where home is refused.
• tick: a settings pull passes the point where letting go commits; a card taken by its header passes the point where letting go closes it; a hold begins on a hold card, and the red arc starts to fill at 2 s.
• mark: BOT held for 0.6 s where home does something; a hold fills its Ring; the arc is full at 5 s; TOP held opens the details or the page of the Wi-Fi or Bluetooth circle.
• step: the pad moves the selection of Apps, the controls of Settings or the ring between the rows of a page; a tap on a row not yet selected; left or right steps the brightness bar.

A buzz that is not played is counted and said at debug level with the reason: haptic: <effect> held back (<reason>). The reasons are level (the level does not play this effect), recording (the microphone or the IMU records), pace (a step came less than 72 ms after the last) and AO_Haptic is busy, no event in the pool or AO_Haptic refused it. The silence for a recording follows the rows of AO_Autotest alone: AUTOTEST_ROW_SIG raises it at the start of the audio, IMU or haptic row and the verdict lowers it. A recording nobody started through a row is not seen.

config key="haptic" is a percentage AO_Haptic applies to every gain it is asked for, the interface's included, so haptic=0 silences the interface as well. haptic_ui=off is the setting meant for the interface alone.

gui_info ends its interface line with ui: haptics <level> (silenced by <nothing|the microphone|the IMU|both>), motion <word>, awake on USB <on|off>; <n> buzzes asked, <m> held back. The bench board 6c:0a has no AW86233, so its evidence is the count. While a secret is typed, kgui writes no haptic: line for a buzz, but the counts still count.

The four rules

The rules behind every table above.

A press that is not a tap acts on nothing. It logs not a tap: the press on <what> was <outcome>; nothing acts at debug level, with the recognizer's word: claim, diagonal, axis, zone or blocked.

A held button counts as input for as long as it is down. Nothing publishes while a hand rests on one, so the face, Apps, Settings and any card over them track which buttons are down (btn_held, gui_btn_track()), and the auto-lock and the rest hold their counts meanwhile.

The recognizer

Every touch sample goes through the recognizer of the kit (kui_gesture, sdk/kui) before LVGL reads it. The recognizer decides what a press is, from where the finger has gone, and it owns every door and every tap: the curtains a finger pulls or catches, the card a finger takes down by its header, the way back a swipe right makes, the Settings pull over Apps, and the tap on the face and on every row, circle and button. LVGL keeps one thing: the scroll of the list page of a card. The rail of Apps is the interface's own, so no LVGL scroll is involved.

Each sample is measured from the point where the finger landed, never from the sample before, so the answer does not depend on how often the touch sensor reports.

A rejection is noted at first and becomes the state only when the finger goes farther than 16 px, so a sloppy tap still works where nothing drags. The input device of AO_Gui starts a scroll one pixel after the claim distance, at 13 px, so the recognizer has decided before a scroll can begin.

gui_gesture_zone() says what a press may become. It reads the screen, the card over it and the landing point on the press edge, once.

The top band. The first 60 rows of the glass are the top band. The corners are cut away, so at each row the band starts where the glass does. The band draws nothing of its own and takes no press: the content may stand in it, and a tap there is the content's tap. The hint stands in it. The face's drag down starts anywhere on it, and over a card a drag down from the band is the card's.

At the end of every press AO_Gui logs one line at debug level (gui_gesture_end()), such as gesture: claim up scroll at 205,400, 0,-180 px or gesture: tap at 205,300, 0,0 px. While a secret is typed, every gesture line is written without coordinates. The list logs its own lines at INFO level: gesture: caught the list at <p>, gesture: list released at <v> px/s -> row <r>. A curtain logs gesture: caught the curtain at <p>% and gesture: release at <p>% <v> px/s -> open|back, and a card logs gesture: card release at <p>% <v> px/s -> stay|close.

The counters. gui_info reports >|gesture. The counters count presses since boot and stop at 65535:

• taps, claims and the four rejections diagonal, axis, zone and blocked.
• commits: curtains and cards a finger followed that landed on the far side. shorts counts the ones that went back to where they started. commits + shorts never exceeds claims.

Gotchas

Example code

Walk the screens with the buttons:

1. Boot the Kode Dot. The face shows. Press TOP and let go. Apps slides in from the right.
2. Press BOT and let go. The face slides back in from the left. Press the pad's up button. Settings comes down over it. Press BOT to close it.
3. Press the pad's right button. Apps opens. Move the pad's down button until an app is in focus, then hold TOP for a moment. The details of the app open. Press BOT to close them.
4. On the face, press BOT and let go. The face dims. Press BOT: "Press ● twice" shows. Press TOP, then TOP again, to wake the face.
5. On the face, press and hold BOT. From 2 s a red arc fills along the top, and at 5 s "Turn off?" rises. Press BOT to answer no.

In the SDK

• kgui_ao.c: the button maps, gui_gesture_zone(), gui_off_arc_tick() and gui_click_is_tap().
• kgui_haptic.c: the level, the silences and the pace of the buzzes.
• kui_gesture.h: the recognizer.
• kqp_config.h: KODE_BUTTON_HOLD_MS and the other timing constants.
• components/kaos/buttons: AO_Buttons, the owner of the six buttons.