# Jessica — Kitchen Mode Refinement

**Status:** Designed, not yet built. Captured 2026-05-30 after a live dinner-party cook.
**Owner context:** Maitland cooked a full grilled dinner using Jessica as a hands-free
sous-chef. The menu/timeline worked; the *interaction model* needs work.

## Problem (from the live retro)

Two complaints, one root cause:
1. **Mic is always on.** In a kitchen with multiple people talking, ambient conversation
   gets picked up and muddles her.
2. **She reads too many steps at once.** Partly recipe formatting (compound steps — one
   numbered item held 5 actions), partly behavior (she keeps going).

**Root cause:** Jessica behaves like a *continuous, autonomous narrator* when a live,
social, hands-busy kitchen needs a *turn-based, addressed-only sous-chef* — speaks only
when addressed, advances only when told, never runs ahead.

## Decisions (agreed)

| Dimension | Decision |
|-----------|----------|
| **Listening** | **Wake word "Hey Jessica"** to engage, **+ a short follow-up listen-window** after she speaks so you can say "next"/"details"/"yes" without re-waking her; she re-arms the wake word when you go quiet. Wake word starts the *conversation*, not every sentence. |
| **Step detail** | **Action, then offer detail.** She reads the terse action and stops; you say "details" to hear quantities/temps/why. |
| **Visual** | **Voice + screen.** Big current-step card + live timers on the propped phone; advance by voice OR tap. Screen is the source of truth when the room's loud. |
| **Timers** | **Ask first.** If a step carries a timer, she offers it and waits for "yes." |
| **Interim mic fix** | Ship **tap-to-talk toggle first** (trivial, reliable), layer wake word on top once tuned. Deferred to implementer's judgment. |

## The keystone: a structured "step" object

Three of the four decisions collapse into one change. A recipe step becomes a small
structured unit instead of a sentence:

```
action: "Put the squash and corn on the grill."
detail: "~475°F, 14 min, turn halfway."
timer:  { label: "squash", seconds: 840 }
```

This single convention powers:
- **Pacing** — read `action`, hard-stop; "details" reads `detail`.
- **Ask-first timers** — if `timer` present, offer it and wait for yes.
- **Visual card** — render `action` big, `detail` on tap, one-tap timer button.

It also fixes the "reads a paragraph" problem and improves `/pa` retrieval. **Start here.**

### Proposed markdown encoding (human-readable, backward-compatible)

Keep the numbered action line; attach optional indented sub-lines. Steps without
sub-lines still parse exactly as today.

```markdown
## Steps
1. Put the squash and corn on the grill.
   - detail: ~475°F, ~14 min, turn halfway to char.
   - timer: squash, 14 min
2. Make the Creamy Oregano Dressing with an immersion blender.
   - detail: Reserve half for serving; use half as the chicken marinade.
```

Parser rule: a numbered line = `action`; following indented `- detail:` / `- timer:`
lines attach to it. `timer` value is `<label>, <duration>` (parse "14 min" → 840s).

## Build order (value-first)

1. **Structured atomic steps** *(foundation — do first)*
   - `Templates/Recipe.md` (vault) — document the new step encoding.
   - `agent/src/recipes.py` — extend `_parse_note` / step parsing to emit
     `{action, detail, timer}` per step (currently steps are flat strings from a
     `\d+[.)]\s+(.+)` regex under `## Steps`).
   - `Recipes/README.md` — document the convention.
   - Re-author `Recipes/Web/Tonight's Dinner Cook Timeline.md` as the proof case
     (break its compound steps into atomic action/detail/timer units).
2. **Kitchen behavior**
   - `agent/src/agent.py` `KitchenAgent`: `start_cooking` / `next_step` / `repeat_step`
     read `action` only and hard-stop; add a "details" path that reads `detail`; when a
     step has a `timer`, offer it and wait for "yes" before calling `set_timer`.
   - Tighten the `KitchenAgent` system prompt: read exactly one step, never preview or
     combine, stop and wait.
3. **Tap-to-talk toggle** *(immediate mic fix)*
   - `creation/index.html` — mic track muted/unpublished by default; a big toggle
     publishes for a turn then re-mutes (LiveKit client `setMicrophoneEnabled`).
4. **Visual step-card + live timers**
   - Agent pushes state (current step, active timers w/ remaining) to the client over a
     LiveKit data channel / RPC; `creation/index.html` renders a step card + timers panel
     with Next / Details / Timer buttons that send commands back. Advance by voice or tap.
5. **Wake word "Hey Jessica"** *(highest effort, last)*
   - In-browser keyword spotter — Picovoice **Porcupine Web** with a custom "Hey Jessica"
     keyword; runs locally so ambient talk never reaches her. On detection, enable mic
     publish for one turn; keep the **short follow-up window** open after she speaks for
     "next"/"details"/"yes"; re-arm on silence. Support barge-in ("Hey Jessica, stop").
   - Needs a Picovoice access key + false-trigger tuning. This is why tap-to-talk ships
     first.

## Risks / open questions

- Porcupine access key + custom keyword training; false-trigger rate in a loud kitchen.
- Follow-up-window length needs tuning (too short = re-waking constantly; too long = back
  to ambient-pickup).
- Visual push: confirm LiveKit data-channel/RPC shape between `agent.py` and the client.
- Device/placement: phone vs tablet, where it's propped, screen-wake/keep-awake.

## Relevant files (today's reality)

- `agent/src/agent.py` — agent (`agent_name="jessica"`), `JessicaBase` shared tools
  (`set_timer` supports multiple named concurrent timers), `KitchenAgent`
  (`start_cooking`, `next_step`, `previous_step`, `repeat_step`, `find_recipe`).
- `agent/src/recipes.py` — `find_recipe` (fuzzy title match), `load_recipe`, `_parse_note`
  (parses `## Ingredients` / `## Steps`).
- `creation/index.html` — working voice UI (`AGENT_NAME='jessica'`, token fetch). **Use
  `/creation/` — NOT `/creation/app/`, which is an unwired PWA shell.**
- `web-server.py` — serves repo root + `/getToken` (port 8081), exposed via Tailscale
  Funnel `https://maitlands-mac-mini.tailb4c264.ts.net/creation/`.
- Run both: `cd agent && uv run python src/agent.py dev` + `uv run python ../web-server.py`.
