Merge pull request 'Local model: compact cached prompt so typed replies take ~6 s (fixes 45 s timeouts)' (#8) from fix/local-fast-prompt into main

Root cause of the 45 s timeouts: the local model got the full hosted prompt (~3.1k tokens with examples). Any prompt-cache
miss meant ~40 s of CPU prefill on the single slot, and every request queued behind one paid it too.

- LOCAL_PROMPT/_local_system_prompt: terse action list, one-line examples, default page text, one line per listing
  (L-refs, dollar prices). No per-visitor data in it, so the cache always hits; it only changes with the catalog.
- _local_user: look overrides (if any) + keyword/budget MATCHING LISTINGS hint + 'VISITOR: <message>'.
- max_tokens 200 for the local model; over-budget card picks dropped when some fit (_respect_budget).
- Buy vs sell examples separated (sell my own thing -> listing form, never checkout).
- 18/18 correct on typical prompts, warm median 5.9 s, max 7.7 s; cold prefill ~13 s.

README.md

@@ -120,14 +120,19 @@ never get rule-based answers. The header badge shows the model that answered.
 ### Local model (last link of the chain)
 A small model runs on the box with llama.cpp's `llama-server` (the build that ships inside Ollama v0.40.1 under
 `~/ollama/lib/ollama/`; any llama.cpp build works). The server tries it after the hosted chain, through the same
-OpenAI-compatible endpoint (`/v1/chat/completions`), with no key. For the local model the request differs in three ways:
-- a GBNF grammar (`JSON_GRAMMAR` in `market/agent.py`) constrains the output to one JSON object;
-- the system prompt adds a block of worked examples (`LOCAL_EXAMPLES`: text edits, hide sections, reset, light mode,
-  list layout, chat position, fonts, gradients, presets, a listing form). It sits before the catalog, so it's cached;
-- the visitor's message is labelled (`VISITOR MESSAGE ...`), because Gemma's chat template folds the system prompt into the
-  first user turn.
+OpenAI-compatible endpoint (`/v1/chat/completions`), with no key.
 
-No chat history is sent to it by default (small models copy earlier answers).
+**It gets its own compact prompt**, because CPU prefill costs about 13 ms per prompt token:
+- `LOCAL_PROMPT` + `_local_system_prompt()`: a terse action list, a dozen one-line examples, the default page text and a
+  one-line-per-listing catalog with short refs (`L1`, `L2`, ...; small models miscopy 8-hex ids, the server maps refs
+  back) and dollar prices. About 1.2k tokens instead of about 3k, and it is the same for every visitor, so
+  llama-server's prompt cache always hits. It only changes when the catalog does (the primer re-warms it then).
+- `_local_user()`: the per-visitor part (a few dozen tokens): the visitor's look overrides if any, a
+  `MATCHING LISTINGS` hint (listings sharing a word with the message, within any stated budget), and the message.
+- A JSON GBNF grammar, `max_tokens` 200, no chat history and no tool round. Over-budget picks are dropped when some fit.
+
+Measured on the demo catalog (12 typical prompts, warm): 12/12 correct, median 5.9 s, max 7.7 s. A turn right
+after a catalog change, before the primer runs, pays about 13 s of prefill once.
 
 | Variable | Meaning |
 |---|---|
@@ -137,6 +142,7 @@ No chat history is sent to it by default (small models copy earlier answers).
 | `AGENT_MARKET_LLM_TOTAL_BUDGET` | hosted chain + local model per turn (60 s); the local model gets `min(45, max(25, 60 - time already spent))` |
 | `AGENT_MARKET_LOCAL_LLM_CONCURRENCY` | 2 = one running + one queued; more simultaneous turns get the busy reply instead of waiting |
 | `AGENT_MARKET_LOCAL_LLM_HISTORY` | chat turns sent to the local model (0) |
+| `AGENT_MARKET_LOCAL_LLM_MAX_TOKENS` | output cap for the local model (200; replies are one sentence + one action) |
 | `AGENT_MARKET_LOCAL_LLM_PRIME` | 1 = warm the prompt cache in the background (see below) |
 
 `deploy.sh` loads these from `~/.config/agent-market/local-llm.env`. The behaviour:
@@ -160,14 +166,16 @@ No chat history is sent to it by default (small models copy earlier answers).
   `~/llm/gemma3-4b-it-Q4_K_M.gguf`.
 - **Resources (measured on this box: 8 vCPU, no GPU, 16 GB RAM).** Idle: 0% CPU, 2.9 GB RSS, of which 0.6 GB is private
   and 2.4 GB is the mmapped model file (clean page cache the kernel can reclaim). While answering: about 6 cores at nice 10
-  for 4–13 s, RSS unchanged.
-- **Latency (real chain, handle_chat, warm cache).** Exact logo phrase 5 s, other text and look edits 4–6 s, search 9 s,
-  compare 10 s, prefilled listing form 13 s (10/10 correct). Cold (no primer yet): about 43 s for the first turn.
+  for 4–8 s, RSS unchanged.
+- **Latency (real chain, handle_chat, warm cache, compact prompt).** Exact logo phrase 4.5 s, look edits 4–6 s, search
+  6 s, compare 6 s, checkout 4 s, cash meetup 4.5 s, prefilled listing form 7–8 s. Cold prefill of the 1.2k-token
+  prompt: about 13 s, once per catalog change. The single slot is shared by demo and staging, so a staging test run
+  makes a demo visitor wait for at most one other reply.
 - **Model choice.** Benchmarked on this box with the real prompt and catalog:
 
   | model | correct | warm latency | notes |
   |---|---|---|---|
-  | **gemma3:4b** (chosen) | 15/15 (Ollama), 10/10 (llama-server + examples) | 4–13 s, cold 30–45 s | 2.9 GB RSS |
+  | **gemma3:4b** (chosen) | 18/18 (compact prompt), 15/15 (Ollama, long prompt) | 4–8 s, cold about 18 s | 2.9 GB RSS |
   | qwen2.5:3b | 7/15 | 3–8 s | dropped actions, invented ids, flat payloads |
   | phi4-mini | 4/5 | 10–25 s | missed the logo edit |
   | qwen3:4b (no think) | 4/5 | 12–22 s | slow, missed the logo edit |

market/agent.py

@@ -412,7 +412,7 @@ def _call_llm(cfg, model, messages, json_mode=True, timeout=None):
     payload = {"model": model, "messages": messages, "temperature": 0.2, "max_tokens": 1100}
     headers = {"Content-Type": "application/json", "X-Title": "Agent Market demo"}
     if cfg.get("local") is True:   # the local server itself (a hosted cfg carries its local backup as a dict)
-        payload["max_tokens"] = 700
+        payload["max_tokens"] = int(os.environ.get("AGENT_MARKET_LOCAL_LLM_MAX_TOKENS", "200"))   # replies are short
         if json_mode:
             payload["grammar"] = JSON_GRAMMAR   # llama-server: constrain the output to one JSON object
     else:
@@ -483,30 +483,84 @@ def _page_text(look):
                        "chips": [x.get("label") for x in c["chips"]], "bullets": c["bullets"]}, ensure_ascii=False)
 
 
-LOCAL_EXAMPLES = """
-EXAMPLES (visitor message => your whole answer). Follow the pattern; use the visitor's own words and CATALOG ids.
-"change the name in the top left to Bazaar Hub" => {"reply":"Done: the top-left logo now says Bazaar Hub.","actions":[{"type":"set_theme","payload":{"copy":{"logo":"Bazaar Hub"}}}]}
-"swap Agent Market for Corner Shop in the header" => {"reply":"The header now says Corner Shop.","actions":[{"type":"set_theme","payload":{"copy":{"logo":"Corner Shop"}}}]}
-"make the big title say Hello neighbours" => {"reply":"Headline updated.","actions":[{"type":"set_theme","payload":{"copy":{"headline":"Hello neighbours"}}}]}
-"get rid of the bullet points" => {"reply":"Bullet points hidden.","actions":[{"type":"set_theme","payload":{"sections":{"hidden":["bullets"]}}}]}
-"go back to the original look" => {"reply":"Back to the default look.","actions":[{"type":"set_theme","payload":{"reset":true}}]}
-"light theme please, with an orange accent" => {"reply":"Light mode with an orange accent.","actions":[{"type":"set_theme","payload":{"mode":"light","palette":{"accent":"#f76b15"}}}]}
-"one item per row instead of a grid" => {"reply":"Results now show as a list.","actions":[{"type":"set_theme","payload":{"layout":"list"}}]}
-"put the chat on the left side" => {"reply":"Chat moved to the left.","actions":[{"type":"set_theme","payload":{"chat":{"position":"left"}}}]}
-"use a mono font, slightly larger" => {"reply":"Mono font, a bit larger.","actions":[{"type":"set_theme","payload":{"font":"mono","fontScale":1.15}}]}
-"a teal to purple gradient behind everything" => {"reply":"Teal to purple gradient background.","actions":[{"type":"set_theme","payload":{"background":{"kind":"gradient","colors":["#12a594","#8e4ec6"],"angle":135}}}]}
-"make it neon" => {"reply":"Neon look applied.","actions":[{"type":"set_theme","payload":{"preset":"neon"}}]}
-"I'd like to sell my old sofa for $60, cash only" => {"reply":"I've filled in the listing form. Check it and press Publish.","actions":[{"type":"show_listing_form","payload":{"defaults":{"title":"Old sofa","kind":"product","price":6000,"description":"Used sofa in good condition.","category":"home","area":"Demo Town","payment_options":["cash"]}}}]}
-"<a search>" => {"reply":"<one short line>","actions":[{"type":"show_cards","payload":{"heading":"<what was searched>","listing_ids":["<id of each CATALOG item that fits>"]}}]}
-"I'll buy <item> and pay online" => {"reply":"Here's the checkout (price + fee).","actions":[{"type":"show_checkout","payload":{"listing_id":"<that item's CATALOG id>"}}]}
-"can I pay cash for <item> in person?" => {"reply":"Cash meetup set up, no fee.","actions":[{"type":"show_cash_meetup","payload":{"listing_id":"<that item's CATALOG id>","suggested_place":"Demo Town square","suggested_time":"Saturday 10am"}}]}
-Only change what the visitor asked for. A visual or text request returns set_theme only (no listings).
-"""
+# ---------- local model: compact prompt ----------
+# CPU prefill costs ~13 ms per prompt token, so the local model gets its own short prompt (~900 tokens instead of ~3k):
+# a terse action list, a few examples, the default page text and a one-line-per-listing catalog with short refs
+# (L1, L2, ...; small models miscopy 8-hex ids) and dollar prices. That system message only changes when the catalog
+# does, so llama-server's prompt cache always hits; the per-visitor part (current look + message) is a few dozen tokens.
+LOCAL_PROMPT = """You run the page of Agent Market, a TEST DEMO marketplace (fake listings, test-mode payments).
+Answer the visitor with ONE JSON object: {"reply":"<one short sentence>","actions":[<one action>]}
+ACTIONS
+show_cards {"heading":"...","listing_ids":["L1"]}  listings that fit the request (respect budgets)
+show_compare {"heading":"...","listing_ids":["L1","L2"]}  side by side
+show_checkout {"listing_id":"L1"}  visitor BUYS a listing and pays online (+FEE_PERCENT fee)
+show_cash_meetup {"listing_id":"L1"}  visitor BUYS a listing and pays cash in person (no fee)
+show_listing_form {"defaults":{"title":"...","kind":"product|service","price":<cents>,"payment_options":["online","cash"]}}
+  visitor SELLS / LISTS their OWN thing (never a checkout): "$120" => 12000, "cash only" => ["cash"]
+set_theme {...}  change how the page LOOKS or what its TEXT says. Payload keys:
+  copy {"logo"|"headline"|"subhead"|"footer": "new text"}  logo = site name in the TOP-LEFT corner, headline = big title
+  preset "retro|minimal|neon|pastel|high contrast|sunset|ocean"; reset true = default look; mode "dark|light"
+  palette {"accent"|"bg"|"text": "#hex"}; background {"kind":"gradient","colors":["#hex","#hex"]}
+  font "serif|mono|Inter|Poppins|Lora|VT323|Comic Neue"; fontScale 0.75-1.75; radius 0-32; layout "grid|list"
+  sections {"hidden":["hero"|"chips"|"bullets"|"results"|"footer"]}; chat {"position":"left|right"}
+RULES: listing ids only from LISTINGS. No ids in reply. A look or text request returns only set_theme. Never refuse a visual change.
+EXAMPLES
+change the name in the top left to Bazaar Hub => {"reply":"The logo now says Bazaar Hub.","actions":[{"type":"set_theme","payload":{"copy":{"logo":"Bazaar Hub"}}}]}
+make the big title say Hello neighbours => {"reply":"Headline updated.","actions":[{"type":"set_theme","payload":{"copy":{"headline":"Hello neighbours"}}}]}
+get rid of the bullet points => {"reply":"Bullet points hidden.","actions":[{"type":"set_theme","payload":{"sections":{"hidden":["bullets"]}}}]}
+go back to the original look => {"reply":"Back to the default look.","actions":[{"type":"set_theme","payload":{"reset":true}}]}
+light theme with an orange accent => {"reply":"Light mode, orange accent.","actions":[{"type":"set_theme","payload":{"mode":"light","palette":{"accent":"#f76b15"}}}]}
+one item per row => {"reply":"Results shown as a list.","actions":[{"type":"set_theme","payload":{"layout":"list"}}]}
+put the chat on the left => {"reply":"Chat moved left.","actions":[{"type":"set_theme","payload":{"chat":{"position":"left"}}}]}
+teal to purple gradient background => {"reply":"Gradient applied.","actions":[{"type":"set_theme","payload":{"background":{"kind":"gradient","colors":["#12a594","#8e4ec6"]}}}]}
+I want to sell my sofa for $60, cash only => {"reply":"Listing form ready, press Publish.","actions":[{"type":"show_listing_form","payload":{"defaults":{"title":"Sofa","kind":"product","price":6000,"payment_options":["cash"]}}}]}
+I'll buy the <listing> and pay online => {"reply":"Here's the checkout.","actions":[{"type":"show_checkout","payload":{"listing_id":"<its id>"}}]}
+can I pay cash for the <listing>? => {"reply":"Cash meetup, no fee.","actions":[{"type":"show_cash_meetup","payload":{"listing_id":"<its id>"}}]}
+<what they want, maybe a budget> => {"reply":"Here's what fits.","actions":[{"type":"show_cards","payload":{"heading":"<topic>","listing_ids":["<each id that fits>"]}}]}
+""".replace("FEE_PERCENT", f"{FEE_RATE*100:g}%")
+
+
+def _local_system_prompt(store):
+    """Stable per catalog (no per-visitor data), so the prompt cache always hits."""
+    from .copyedit import current_copy
+    c = current_copy({}, FEE_BPS)
+    rows = [f"L{i} | {x['title']} | ${x['price_cents'] / 100:.2f} | {x['kind']} | {' '.join(x.get('pay') or [])}"
+            for i, x in enumerate(_catalog(store), 1)]
+    return (LOCAL_PROMPT + f'PAGE TEXT (defaults): logo "{c["logo"]}"; headline "{c["headline"]}"; footer "{c["footer"]}"\n'
+            "LISTINGS (id | title | price | kind | pay)\n" + "\n".join(rows))
+
+
+_BUDGET_RE = re.compile(r"(?:under|below|less than|max(?:imum)?|up to|cheaper than|<)\s*\$?\s*(\d+(?:\.\d+)?)", re.I)
+
+
+def _local_hints(store, message):
+    """Retrieval help for the small model: the listings whose title/category/tags share a word with the message, within
+    any budget it states ("under $100"). The model still decides what to show; semantic asks with no word in common
+    ("something for my garden") get no hint and use the full LISTINGS."""
+    words = {w.rstrip("s") for w in re.findall(r"[a-z]{3,}", (message or "").lower()) if w not in _STOP}
+    if not words:
+        return ""
+    m = _BUDGET_RE.search(message or "")
+    cap = int(float(m.group(1)) * 100) if m else None
+    hits = []
+    for i, c in enumerate(_catalog(store), 1):
+        L = store.get("listings", c["id"]) or {}
+        toks = {t.rstrip("s") for t in re.findall(r"[a-z]{3,}", " ".join([c["title"], c.get("category") or "", " ".join(L.get("tags") or [])]).lower())}
+        if words & toks and (cap is None or c["price_cents"] <= cap):
+            hits.append(f"L{i} {c['title']} ${c['price_cents'] / 100:.2f}")
+    return ("MATCHING LISTINGS (same words" + (", within budget" if cap is not None else "") + "): " + "; ".join(hits[:6]) + "\n") if hits else ""
+
+
+def _local_user(message, look="", store=None):
+    """Per-visitor part (a few dozen tokens): their look overrides (if any), listing hints, and the message."""
+    head = ""
+    if look and look.strip() not in ("", "{}"):
+        head = "CURRENT LOOK (this visitor's overrides): " + look.strip()[:400] + "\n"
+    if store is not None:
+        head += _local_hints(store, message)
+    return head + "VISITOR: " + message
 
 
-# Small models miscopy 8-hex ids (lst_ffe42107 vs lst_29d7f320), so the local model sees short catalog refs (L1, L2, ...)
-# that are mapped back to real ids before anything runs. Unknown refs are dropped like any invented id.
-LOCAL_SCHEMA = UI_SCHEMA.replace('"lst_..."', '"L7"').replace("internal ids (lst_...,", "internal ids (L7, lst_...,")
 ALIAS_RE = re.compile(r"\s*\(?\bL\d{1,3}\b\)?:?")
 
 
@@ -531,12 +585,9 @@ def _unalias(obj, alias):
     return obj
 
 
-def _system_prompt(store, look="", local=False):
-    # stable parts first (schema, examples, catalog) so a local model's prompt cache can reuse them across turns
-    cat = _catalog(store)
-    if local:
-        cat = [{**c, "id": f"L{i}"} for i, c in enumerate(cat, 1)]
-    return ((LOCAL_SCHEMA + LOCAL_EXAMPLES if local else UI_SCHEMA) + "\nCATALOG (all active listings):\n" + json.dumps(cat)[:9000] +
+def _system_prompt(store, look=""):
+    """Hosted models: the full schema, the catalog with real ids, the visitor's look and page text."""
+    return (UI_SCHEMA + "\nCATALOG (all active listings):\n" + json.dumps(_catalog(store))[:9000] +
             "\nCURRENT_LOOK (this visitor's theme overrides; {} = default dark look):\n" + (look or "{}") +
             "\nPAGE_TEXT (what the editable text slots show right now):\n" + _page_text(look))
 
@@ -571,12 +622,12 @@ def _prime_local_once(store):
         return False
     if _local_restarted(local):
         _PRIMED["key"] = None
-    sp = _system_prompt(store, "", local=True)
+    sp = _local_system_prompt(store)
     key = hash(sp)
     if key == _PRIMED["key"] or not _local_slots.acquire(blocking=False):
         return False
     try:
-        body = {"model": local["model"], "messages": [{"role": "system", "content": sp}, {"role": "user", "content": _visitor("hi")}],
+        body = {"model": local["model"], "messages": [{"role": "system", "content": sp}, {"role": "user", "content": _local_user("hi")}],
                 "max_tokens": 1, "temperature": 0}
         req = urllib.request.Request(local["url"], data=json.dumps(body).encode(), headers={"Content-Type": "application/json"})
         with urllib.request.urlopen(req, timeout=180) as r:
@@ -606,19 +657,16 @@ def start_local_primer(store, every=30):
     threading.Thread(target=loop, name="local-llm-primer", daemon=True).start()
 
 
-def _visitor(message):
-    # small models (and chat templates that fold the system prompt into the first user turn) need a clear boundary
-    return "\n\nVISITOR MESSAGE (answer this one with the JSON object):\n" + message
-
-
 def _llm_once(store, cfg, model, message, history, look="", timeout=None):
     local = cfg.get("local") is True
-    sys_prompt = _system_prompt(store, look, local=local)
-    msgs = [{"role": "system", "content": sys_prompt}]
+    if local:
+        msgs = [{"role": "system", "content": _local_system_prompt(store)}]
+    else:
+        msgs = [{"role": "system", "content": _system_prompt(store, look)}]
     for h in (history or [])[-6:]:
         if h.get("role") in ("user", "assistant") and h.get("content"):
             msgs.append({"role": h["role"], "content": str(h["content"])[:600]})
-    msgs.append({"role": "user", "content": _visitor(message) if local else message})
+    msgs.append({"role": "user", "content": _local_user(message, look, store) if local else message})
     t_end = time.time() + (timeout or LLM_TIMEOUT)
     alias = _aliases(store) if local else {}
     content = _call_llm(cfg, model, msgs, timeout=timeout)
@@ -673,6 +721,21 @@ def _focus_results(store, message, actions):
     return actions
 
 
+def _respect_budget(store, message, actions):
+    """Visitor said "under $X": drop listed picks above it when some within budget remain (small models overshoot)."""
+    m = _BUDGET_RE.search(message or "")
+    if not m:
+        return actions
+    cap = int(float(m.group(1)) * 100)
+    for a in actions:
+        if a.get("type") == "show_cards" and isinstance(a.get("payload"), dict):
+            ids = [i for i in (a["payload"].get("listing_ids") or []) if isinstance(i, str)]
+            keep = [i for i in ids if (store.get("listings", i) or {}).get("price", cap + 1) <= cap]
+            if keep and len(keep) < len(ids):
+                a["payload"]["listing_ids"] = keep
+    return actions
+
+
 LOCAL_HISTORY = int(os.environ.get("AGENT_MARKET_LOCAL_LLM_HISTORY", "0"))  # small models copy earlier turns; 0 = no history
 
 
@@ -740,7 +803,7 @@ def llm_agent(store, message, history, cfg, look=""):
                     out = fut.result(timeout=cap + 1)
                 except concurrent.futures.TimeoutError:
                     raise LLMError("timeout", f">{cap:.0f}s")
-                out["actions"] = _focus_results(store, message, out["actions"])
+                out["actions"] = _respect_budget(store, message, _focus_results(store, message, out["actions"]))
                 out.update(agent="llm", provider="local", local=True, model=local["model"],
                            latency_ms=int((time.time() - t0) * 1000), fallbacks=errors)
                 LAST.update(model=local["model"], at=time.time(), error=None, local=True)

tests/e2e_http.py

@@ -124,7 +124,7 @@ B = {"X-Agent-Id": "agent:e2e-buyer-bot"}
 s, r = call("POST", "/api/actions", {"action": "create_listing", "args": {"title": "Test-only data labelling (agent service)", "kind": "service", "price": 1500, "description": "Fake service listed by a test agent"}}, A)
 al = r.get("listing", {})
 check("agent API: agent lists a service", s == 201 and al.get("seller") == "agent:e2e-seller-bot" and al.get("seller_type") == "agent")
-s, r = call("POST", "/api/actions", {"action": "search", "args": {"q": "labelling"}}, B)
+s, r = call("POST", "/api/actions", {"action": "search", "args": {"q": "labelling", "limit": 50}}, B)   # staging accumulates test listings
 check("agent API: search finds it", any(x["id"] == al.get("id") for x in r.get("listings", [])))
 s, r = call("POST", "/api/actions", {"action": "quote", "args": {"listing_id": al["id"]}}, B)
 check("agent API: quote online vs cash", r["online"]["fee_cents"] == round(1500 * fee_bps / 10000) and r["cash_in_person"]["fee_cents"] == 0)

tests/mock_llm.py

@@ -25,7 +25,7 @@ class H(BaseHTTPRequestHandler):
             SEEN["auth"][mdl] = SEEN["auth"].get(mdl, 0) + 1
         if body.get("grammar"):
             SEEN["grammar"][mdl] = SEEN["grammar"].get(mdl, 0) + 1
-        if "EXAMPLES" in body["messages"][0]["content"] and "VISITOR MESSAGE" in body["messages"][-1]["content"]:
+        if "EXAMPLES" in body["messages"][0]["content"] and "VISITOR: " in body["messages"][-1]["content"]:
             SEEN["local_prompt"][mdl] = SEEN["local_prompt"].get(mdl, 0) + 1
         if body.get("model") == "daily-model":
             b = json.dumps({"error": {"message": "Rate limit exceeded: free-models-per-day.", "code": 429,
@@ -36,7 +36,7 @@ class H(BaseHTTPRequestHandler):
         if "new internet" in last.lower():
             # answers like a model that read the prompt: it only knows "top left" = copy.logo if the prompt says so
             sysmsg = body["messages"][0]["content"]
-            knows = "TOP-LEFT" in sysmsg and '"logo": "Agent Market"' in sysmsg
+            knows = "TOP-LEFT" in sysmsg and ('"logo": "Agent Market"' in sysmsg or 'logo "Agent Market"' in sysmsg)
             out = ({"reply": "Renamed the top-left logo to New Internet.", "actions": [{"type": "set_theme", "payload": {"copy": {"logo": "New Internet"}}}]}
                    if knows else {"reply": "I can't change that text.", "actions": []})
             b = json.dumps({"choices": [{"message": {"role": "assistant", "content": json.dumps(out)}}]}).encode()

tests/test_agent_units.py

@@ -39,13 +39,21 @@ cfg = agent._llm_config()
 check("local model alone is a valid chain", cfg and cfg["models"] == [] and cfg["local"]["model"] == "m" and cfg["local"]["url"].endswith("/v1/chat/completions"))
 # local model: short catalog refs in the prompt, mapped back to real ids; unknown refs left for hydrate() to drop
 al = agent._aliases(st)
-sp = agent._system_prompt(st, "", local=True)
-check("local prompt uses short refs, not hex ids", '"id": "L1"' in sp and not re.search(r"lst_[0-9a-f]{6}", sp.split("CATALOG (all active listings)")[1]) and '"L7"' in sp, len(al))
+sp = agent._local_system_prompt(st)
+check("local prompt: compact, short refs + dollar prices, no hex ids, no per-visitor data",
+      "\nL1 | " in sp and not re.search(r"lst_[0-9a-f]{6}", sp) and "$" in sp.split("LISTINGS")[1] and "CURRENT LOOK" not in sp and len(sp) < 6000, len(sp))
+check("local prompt is the same for every visitor (prompt cache hits)", sp == agent._local_system_prompt(st))
+u = agent._local_user("show me bikes under $100", '{"mode":"light"}', st)
+check("per-visitor part: look, budget-aware hints, message", u.startswith("CURRENT LOOK") and "MATCHING LISTINGS" in u and "within budget" in u and u.endswith("VISITOR: show me bikes under $100"), u)
+check("no hint for a text edit", "MATCHING" not in agent._local_user("replace agent market to new internet on the top left corner", "", st))
 plan = agent._unalias({"actions": [{"type": "show_cards", "payload": {"listing_ids": ["L1", "L999"]}},
                                    {"type": "show_checkout", "payload": {"listing_id": "L2"}}]}, al)
 check("refs mapped back (unknown kept for hydrate to drop)", plan["actions"][0]["payload"]["listing_ids"] == [al["L1"], "L999"] and
       plan["actions"][1]["payload"]["listing_id"] == al["L2"])
-check("hosted prompt keeps real ids", al["L1"] in agent._system_prompt(st, "", local=False))
+check("hosted prompt keeps real ids", al["L1"] in agent._system_prompt(st, ""))
+cheap = min(st.all("listings"), key=lambda L: L["price"]); dear = max(st.all("listings"), key=lambda L: L["price"])
+b = agent._respect_budget(st, f"anything under ${cheap['price'] // 100 + 1}", [{"type": "show_cards", "payload": {"listing_ids": [dear["id"], cheap["id"]]}}])
+check("budget: over-budget picks dropped when some fit", b[0]["payload"]["listing_ids"] == [cheap["id"]], b)
 
 # primer: re-prime after the local server restarts (llama-server /metrics counter goes back down) or is down
 import http.server, threading

tests/test_copy_defaults.py

@@ -18,8 +18,8 @@ check("DEFAULT_COPY in sync with web/src/App.tsx", ok)
 check("model prompt explains top-left = logo and includes PAGE_TEXT", "TOP-LEFT" in agent.UI_SCHEMA and '"logo": "Agent Market"' in agent._page_text("{}"))
 check("PAGE_TEXT follows the visitor's overrides", '"logo": "New Internet"' in agent._page_text('{"copy":{"logo":"New Internet"}}'))
 check("fee in bullets follows FEE_BPS", current_copy({}, 250)["bullets"][0].startswith("2.5% "))
-check("local prompt carries examples (incl. a top-left text edit), hosted prompt doesn't",
-      "EXAMPLES" in agent.LOCAL_EXAMPLES and '"copy":{"logo"' in agent.LOCAL_EXAMPLES and "EXAMPLES (" not in agent.UI_SCHEMA)
+check("compact local prompt maps TOP-LEFT = logo and has a logo example; hosted prompt has no examples block",
+      "TOP-LEFT" in agent.LOCAL_PROMPT and '"copy":{"logo"' in agent.LOCAL_PROMPT and "EXAMPLES" not in agent.UI_SCHEMA)
 gone = [n for n in ("fallback_agent", "_rules_check") if hasattr(agent, n)]
 import market.theme as th, market.copyedit as ce
 gone += [n for n in ("parse_look_command",) if hasattr(th, n)] + [n for n in ("parse_copy_edit",) if hasattr(ce, n)]