# Jessica — Kitchen Mode Step 4: Visual step-card + live timers (plan)

**Status:** Built 2026-05-31. Companion to `kitchen-mode-refinement.md` (steps 1–3 shipped
the same day). The open items below were resolved during the build (notes inline). What's
left is a real-device voice smoke test — the headless browser has no mic, so the live
agent↔client wire (attribute push, RPC round trip, voice-advance updating the card) was
verified by API/unit/render checks but not over a live room.

This is the build spec for step 4 of the refinement build order: a propped-phone screen
that shows the current step as a big card and any running timers as live countdowns,
advanced by voice **or** tap, with the agent as the single source of truth.

## Decisions locked (Maitland, 2026-05-31)

- **Target screen:** **phone-optimized**, not the 240×282 R1 frame. Jessica runs as a
  PWA on a phone propped on the counter. Kitchen mode gets a full phone-screen layout.
- **Tap semantics:** **tap speaks + updates the screen.** Tapping Next / Details / Timer
  routes through the agent and triggers the same behavior as saying it aloud — she reads
  the action / detail, and the card updates. Voice and screen stay in lockstep.

## Architecture (verified against LiveKit docs, 2026-05-31)

Two LiveKit data primitives, one per direction. Confirmed current API via `lk docs`
(`/transport/data/state/participant-attributes`, `/transport/data/rpc`):

| Direction | Mechanism | Why |
|-----------|-----------|-----|
| **agent → client** (push state) | **Participant attributes** — agent sets `kitchen.state` (a JSON string) on its own participant; client listens on `RoomEvent.ParticipantAttributesChanged`. | Key-value state sync, auto-delivered to late joiners. Single source of truth lives on the agent. |
| **client → agent** (commands) | **RPC** — agent registers `kitchen.next` / `kitchen.prev` / `kitchen.repeat` / `kitchen.details` / `kitchen.timer`; client calls `localParticipant.performRpc({destinationIdentity, method})`. | Request/response, structured, awaitable. The button does exactly what the voice command does. |

**Critical constraint (from the docs):** participant attributes are *not* for
high-frequency updates ("more than once every few seconds"). So **do not** stream
per-second timer ticks. Push each timer's `endsAt` **once** when it starts/stops, and let
the client compute the live `mm:ss` countdown locally (`setInterval`). Attribute writes
then only happen on discrete events — step change, timer start, timer cancel/fire — well
within budget.

### State payload shape (`kitchen.state` attribute, JSON)

```json
{
  "cooking": true,
  "recipe": "Tonight's Dinner Cook Timeline",
  "idx": 9,
  "total": 23,
  "action": "Put the squash on the grill.",
  "detail": "Turn the pieces halfway through.",
  "offeredTimer": { "label": "squash", "seconds": 840 },
  "activeTimers": [
    { "label": "grill preheat", "endsAt": 1748697600000 },
    { "label": "skewers", "endsAt": 1748698200000 }
  ]
}
```

- `action` is what the card shows big. `detail` is revealed on the Details tap.
- `offeredTimer` is the timer attached to the *current* step (drives the "Set timer"
  button); `null` if the step has none.
- `activeTimers` is every running timer with an absolute `endsAt` (epoch ms). Client
  renders each as a chip counting down; drop a chip when it reaches zero (the agent also
  removes it on fire and re-pushes).
- When cooking ends or she leaves the kitchen, push `{ "cooking": false }` (or clear the
  key with `""`) so the card disappears and the main view returns.

## Build order

### Agent (`agent/src/agent.py`)

1. **Active-timer registry.** Extend the step-3 timer work: `_arm_timer` records
   `{label, endsAt}` in a per-agent dict and removes the entry when the timer fires or is
   cancelled. This is what `activeTimers` is built from.
2. **State push helper.** `_push_kitchen_state()` builds the payload above from
   `_steps`/`_idx` + the timer registry and calls
   `room.local_participant.set_attributes({"kitchen.state": json.dumps(...)})`. Call it
   from `start_cooking`, `next_step`, `previous_step`, `repeat_step`, `start_step_timer`,
   and on timer fire.
3. **RPC methods.** Register `kitchen.next` / `.prev` / `.repeat` / `.details` / `.timer`
   in `KitchenAgent.on_enter`; each handler invokes the *same* logic as the matching voice
   tool **and** has her speak (tap-speaks decision) via `session.generate_reply`/`say`,
   then pushes state. Unregister + clear `kitchen.state` in `leave_kitchen` (and on the
   handoff back to `Assistant`) so no stale card lingers.

### Client (`creation/index.html`)

4. **Phone layout + kitchen view.** Switch the viewport to `width=device-width` and add a
   kitchen card view (big action text, Details button, timer chips, Prev/Repeat/Next).
   Show it when `kitchen.state.cooking` is true; otherwise keep the existing voice-orb
   view. Tap-to-talk (step 3) stays available.
5. **Consume state.** On `ParticipantAttributesChanged` for the agent participant, parse
   `kitchen.state` and render. Compute timer countdowns locally from `endsAt`.
6. **Send commands.** Wire the buttons to `localParticipant.performRpc({ destinationIdentity:
   <agent identity>, method: 'kitchen.next' | ... })`. The agent identity is the remote
   participant we already track in `TrackSubscribed` / `room.remoteParticipants`.
7. **Keep-awake.** `navigator.wakeLock.request('screen')` when cooking starts; re-acquire
   on `visibilitychange`. A propped phone must not sleep mid-cook.

## Open implementation items — resolved during the build

- **`canUpdateOwnMetadata` permission** → *no change needed.* The agents framework already
  calls `set_attributes` on the agent's own participant to publish `lk.agent.state`
  (`room_io.py`), so the grant is present. Attribute writes merge by key, so `kitchen.state`
  sits alongside the framework's state key. The browser client uses RPC only — its token is
  untouched.
- **Room access from inside the Agent** → `get_job_context().room`, wrapped in `_room()`
  which returns `None` outside a job (so unit/behavior tests, which have no room, skip the
  push cleanly).
- **RPC registration lifecycle** → registered in `KitchenAgent.on_enter` (handlers are bound
  methods, so they capture the live instance), torn down in `leave_kitchen`. Re-entry just
  re-registers (same name overwrites), so kitchen→main→kitchen is safe.
- **Layout scope** → second view toggled on `cooking`, not a rewrite. Viewport switched to
  `device-width`; the existing voice-orb view was made responsive (fills the viewport) and
  the kitchen card is a full-screen overlay shown only while cooking.

## What shipped

- **Agent (`agent/src/agent.py`):** active-timer registry with `endsAt` in `_arm_timer`;
  `_build_kitchen_state` / `_push_kitchen_state` (sets the `kitchen.state` attribute);
  pushes on `start_cooking` / `next_step` / `previous_step` / `start_step_timer` and on
  timer fire (`_after_timer_fired`); RPC methods `kitchen.next/prev/repeat/details/timer`
  that advance, speak via `session.say`, and re-push; `leave_kitchen` clears the card and
  unregisters.
- **Client (`creation/index.html`):** phone viewport + full-screen kitchen card;
  `ParticipantAttributesChanged` → `renderKitchen`; local per-second timer countdown;
  Back / Repeat / Next / Details / Set-timer buttons over `performRpc`; whole-screen
  tap-to-talk disabled in kitchen view (buttons own their taps; a dedicated mic button
  remains); `navigator.wakeLock` to keep the propped phone awake.
- **Tests:** deterministic `_build_kitchen_state` payload + timer-registry tests; full
  suite green (64). Client render verified live in a headless browser (card content, live
  countdown tick 5:04→5:01, Details reveal, hide on `cooking:false`, phone layout).

## Testing

- **Deterministic (unit):** timer-registry add/remove; `_push_kitchen_state` payload shape
  given `_steps`/`_idx` + timers (pure, no LiveKit). Extend `tests/test_tools.py`.
- **Behavior (judge):** an RPC `kitchen.next` advances the step and she speaks the next
  action (mirror of the step-2 tests, driving via the tool path). Extend `tests/test_agent.py`.
- **Live:** on the phone via the Funnel URL — start the dinner timeline, confirm the card
  tracks voice advances, tap Next advances + speaks, a set timer counts down on screen and
  announces at zero, and the screen stays awake.

## Deferred to step 5

Wake word "Hey Jessica" + the short follow-up listen-window build on top of this view (e.g.
a barge-in "Hey Jessica, stop"). Out of scope here.