diff --git a/README.md b/README.md index 5e9a78c..08504a1 100644 --- a/README.md +++ b/README.md @@ -11,8 +11,8 @@ sell products or services. ## How it works ``` -browser (React SPA) ─────/api/chat──▶ tiny server ──▶ hosted LLM (OpenAI-compatible) - ▲ renders UI actions │ or the rule-based fallback agent +browser (React SPA) ─────/api/chat──▶ tiny server ──▶ free hosted LLMs (OpenAI-compatible) + ▲ renders UI actions │ then the local model on this box (llama-server) │ ├── /api/payments/* ── mock test-mode provider ── signed webhook └── card form posts straight ────────┘ to the payment endpoint (never to the LLM) @@ -66,13 +66,10 @@ never changes the page for anyone else. **Reset look** in the header, or saying - Protected panels use the theme's surface colour as a solid fill, with black or white text, whichever contrasts more, so they always clear 4.5:1. The banner and badge use fixed high-contrast colours. - Text sitting directly on the page background switches to readable ink. If no ink clears 4.5:1 against every gradient stop, it gets a scrim behind it. - **Page text.** The logo (top-left brand text), headline, subtitle, footer, quick-action buttons and bullets are editable - through `copy`. The built-in agent (`market/copyedit.py`) understands "replace/change/rename/swap A to/with/into B", - locations such as top left, header, logo, brand, site name, title, headline, subtitle and footer, quoted or unquoted text, - and a generic find-and-replace when A is the current text of exactly one slot. It keeps the case you type for B; text typed - all in lowercase takes the style of the text it replaces ("new internet" replacing "Agent Market" gives "New Internet"). - When it can't tell what to change, it says what it understood and offers the closest option. The model prompt gets the - same slot map plus the visitor's current page text (`PAGE_TEXT`). -- If the model is unavailable, the built-in agent understands the common look commands: presets, light/dark, background colours, gradients and patterns, accent and text colour, fonts, bigger or smaller text, corners, density, card style, list/grid, columns, chat position and size, hiding or showing sections, headline, logo and subtitle, animations, and reset. + through `copy`. The prompt maps where each slot is ("top left" = `copy.logo`, title, subtitle, footer, ...) and includes the + visitor's current page text (`PAGE_TEXT`, from `market/copyedit.py`), so "replace agent market to new internet on the top + left corner" becomes `{"copy":{"logo":"New Internet"}}`. There is no rule-based text or look editing: every typed request + is answered by a model (see below). ### Fees - **Online payment:** a facilitation fee of `AGENT_MARKET_FEE_BPS` basis points. The default is 500, which is 5%. @@ -88,7 +85,7 @@ never changes the page for anyone else. **Reset look** in the header, or saying | anything else | refused (`not_a_test_card`) | ## LLM configuration -The model has to be hosted. The server talks to any OpenAI-compatible `/chat/completions` endpoint, and the +The server talks to any OpenAI-compatible `/chat/completions` endpoint, and the key stays on the server: it never goes to the browser, never into the repo, and never into a URL. | Variable | Meaning | @@ -104,7 +101,9 @@ key stays on the server: it never goes to the browser, never into the repo, and `deploy.sh` loads `~/.config/agent-market/llm.env` and `~/.config/agent-market/.env` if they exist. Both sit outside the repo. The box runs OpenRouter free models: -`nvidia/nemotron-3-super-120b-a12b:free` → `apodex/apodex-1.1-mini:free` → `google/gemma-4-31b-it:free` → rule-based agent +`nvidia/nemotron-3-super-120b-a12b:free` → `apodex/apodex-1.1-mini:free` → `google/gemma-4-31b-it:free` → **local `gemma3:4b`** + +That is the whole chain. There are no keyword rules for typed messages. The order came from a benchmark of natural-language prompts, scored on valid UI actions and latency. In that run: - **nemotron-3-super**: 8/8 valid, about 2–3 s. @@ -113,14 +112,77 @@ The order came from a benchmark of natural-language prompts, scored on valid UI - **inkling**: 403, because it's limited to agentic harnesses. - **nemotron-3.5-lightning**: 24–55 s, and returned non-JSON. -If a model returns 429, it times out, or it returns invalid JSON, the server moves on to the next model in the chain. When every model fails, -the labelled rule-based agent answers, so the page never breaks. The header badge shows the model that answered. +If a model returns 429, it times out, or it returns invalid JSON, the server moves on to the next model in the chain. When every +model fails (or none is configured), the reply is an honest **"The assistant is busy, please try again in a moment."** with a +**Retry** button that resends the same message (`agent: "unavailable"`, no actions; the badge says `AI: busy`). Typed messages +never get rule-based answers. The header badge shows the model that answered. + +### Local model (last link of the chain) +A small model runs on the box with llama.cpp's `llama-server` (the build that ships inside Ollama v0.40.1 under +`~/ollama/lib/ollama/`; any llama.cpp build works). The server tries it after the hosted chain, through the same +OpenAI-compatible endpoint (`/v1/chat/completions`), with no key. For the local model the request differs in three ways: +- a GBNF grammar (`JSON_GRAMMAR` in `market/agent.py`) constrains the output to one JSON object; +- the system prompt adds a block of worked examples (`LOCAL_EXAMPLES`: text edits, hide sections, reset, light mode, + list layout, chat position, fonts, gradients, presets, a listing form). It sits before the catalog, so it's cached; +- the visitor's message is labelled (`VISITOR MESSAGE ...`), because Gemma's chat template folds the system prompt into the + first user turn. + +No chat history is sent to it by default (small models copy earlier answers). + +| Variable | Meaning | +|---|---| +| `AGENT_MARKET_LOCAL_LLM_URL` | e.g. `http://127.0.0.1:11434/v1` (loopback only) | +| `AGENT_MARKET_LOCAL_LLM_MODEL` | e.g. `gemma3:4b` (the server's `--alias`) | +| `AGENT_MARKET_LOCAL_LLM_TIMEOUT` / `_MIN` | its own cap (45 s; CPU inference is slower) and the minimum it always gets (25 s) | +| `AGENT_MARKET_LLM_TOTAL_BUDGET` | hosted chain + local model per turn (60 s); the local model gets `min(45, max(25, 60 - time already spent))` | +| `AGENT_MARKET_LOCAL_LLM_CONCURRENCY` | 2 = one running + one queued; more simultaneous turns get the busy reply instead of waiting | +| `AGENT_MARKET_LOCAL_LLM_HISTORY` | chat turns sent to the local model (0) | +| `AGENT_MARKET_LOCAL_LLM_PRIME` | 1 = warm the prompt cache in the background (see below) | + +`deploy.sh` loads these from `~/.config/agent-market/local-llm.env`. The behaviour: +- **Quota stop.** When OpenRouter reports the daily free quota is used up, every hosted model is parked until the reset + time and turns go **straight to the local model**, with no hosted calls in between. The stop is saved in the data + directory (`llm_quota.json`, keyed by the base URL, capped at 36 h), so a restart doesn't spend a call to rediscover it. +- **Badge.** It honestly shows `Local model (gemma3:4b)` when the local model answered (chat header: `Local model · gemma3:4b · 8.2s`), + and also on page load while the hosted quota is used up. +- **Prompt cache.** The first call with a new prompt prefix processes about 2.4k prompt tokens on the CPU (about 30–40 s). + A background primer sends the current prompt once (1 output token) at start-up, whenever the catalog changes, and after + the model server restarts (detected from llama-server's `/metrics` counter). Otherwise it only reads `/metrics` every + 30 s, so an idle server stays at 0% CPU. `--swa-full` keeps Gemma's sliding-window layers reusable, so a warm turn only + processes the new tokens (about 0.3 s). `--cache-ram 1024` keeps a few prompt states (demo and staging) in RAM. +- **Running it.** `ops/local-llm-watchdog.sh --ensure` (idempotent; `deploy.sh` calls it) starts a private copy of the + watchdog under `~/llm/`. It keeps `llama-server` on `127.0.0.1:11434` only, restarts it if it exits (backing off to 30 s + if it keeps crashing) or fails 3 health checks 15 s apart, and rotates its log. Stop it with `kill $(cat ~/llm/watchdog.pid)`. + Lean flags so the rest of the box keeps headroom: text weights only (`--no-mmproj`, the 0.8 GB vision projector is never + loaded), mmapped and not repacked (`--no-repack`, no second copy of the weights), `-c 6144 -np 1`, q8 KV cache, + flash attention, `-t 6 -tb 6` at `nice 10`, `--poll 0` (threads sleep when idle). Overrides go in + `~/.config/agent-market/local-llm-server.env`. Model file: the gemma3:4b Q4_K_M text GGUF (2.4 GB), linked as + `~/llm/gemma3-4b-it-Q4_K_M.gguf`. +- **Resources (measured on this box: 8 vCPU, no GPU, 16 GB RAM).** Idle: 0% CPU, 2.9 GB RSS, of which 0.6 GB is private + and 2.4 GB is the mmapped model file (clean page cache the kernel can reclaim). While answering: about 6 cores at nice 10 + for 4–13 s, RSS unchanged. +- **Latency (real chain, handle_chat, warm cache).** Exact logo phrase 5 s, other text and look edits 4–6 s, search 9 s, + compare 10 s, prefilled listing form 13 s (10/10 correct). Cold (no primer yet): about 43 s for the first turn. +- **Model choice.** Benchmarked on this box with the real prompt and catalog: + + | model | correct | warm latency | notes | + |---|---|---|---| + | **gemma3:4b** (chosen) | 15/15 (Ollama), 10/10 (llama-server + examples) | 4–13 s, cold 30–45 s | 2.9 GB RSS | + | qwen2.5:3b | 7/15 | 3–8 s | dropped actions, invented ids, flat payloads | + | phi4-mini | 4/5 | 10–25 s | missed the logo edit | + | qwen3:4b (no think) | 4/5 | 12–22 s | slow, missed the logo edit | + | llama3.2:3b | 1/5 | 4–8 s | mostly empty plans | + | qwen2.5:1.5b | 0/5 | 3–6 s | no usable plans | + + 7B models were left out: they need about 5 GB more RAM, and on this CPU they would be about 2–3x slower. + Ollama itself was dropped in favour of running `llama-server` directly: Ollama loaded the vision projector, a 16k context + with 2 slots and an 8 GiB prompt cache, which grew to about 10 GB RSS. **Free text goes to the LLM. Structured UI events stay deterministic.** Button clicks, form submits and payment results each have exactly one outcome, so the server handles them deterministically. That keeps them instant and saves free-tier quota. The landing view doesn't call the model either. Set `AGENT_MARKET_LLM_EVENTS=1` to send events to the LLM as well. -With no LLM configured, the demo runs the **rule-based fallback agent**. The page labels this. +With no model configured at all, clicks, forms and payments still work, and typed messages get the busy reply. ## Frontend (React + Vite + TypeScript) @@ -152,7 +214,8 @@ reproducible) **before** it stops the running server, so a failed build leaves t Locally: `cd web && npm ci && npm run build`, then run `server.py`. For hot reload, run the server on :8795 and `npm run dev` (Vite proxies `/api`, `/agent.json`, `/openapi.json` to it). -**How the page renders.** Typed messages go to `/api/chat` (the model, or the built-in agent when it's unavailable). +**How the page renders.** Typed messages go to `/api/chat` (a hosted model, then the local model; if both fail, a busy +reply with a Retry button). Clicks, form submits and payment results are sent as events that the server handles deterministically. The reply's `actions` go through `render()`: every action type is looked up in the registry. Effects (`set_theme`, `toast`, `clear_panel`) run first; the panels of one reply replace the results area together. If the visitor's theme hides @@ -168,7 +231,7 @@ theme object. The theme is stored per browser in `localStorage["am_theme_v1"]`; ### Adding a UI component 1. **Server.** Add the action type to `ALLOWED_ACTIONS` and describe its payload in `UI_SCHEMA` in `market/agent.py`. If the payload references store objects, resolve them in `hydrate()` in `server.py` (never trust model-made ids or prices), - and add a rule to `fallback_agent`/`fallback_event` if the built-in agent should be able to show it. + and add a case to `fallback_event` if a click or form should open it deterministically. 2. **Component.** Create it in `web/src/components/`, e.g. ```tsx export function Reviews({ payload }: { payload: { heading?: string; reviews?: Review[] } }) { @@ -191,7 +254,8 @@ The first run seeds `data/store.json` with fake listings. Delete that file to re `deploy.sh [ref]` checks out a ref, builds the frontend, and restarts the service on the box. See the script for details. For test runs without real model calls, point the LLM at the fake model: run `python3 tests/mock_llm.py 18797` and pass `AM_EXTRA_ENV=` with `AGENT_MARKET_LLM_BASE_URL=http://127.0.0.1:18797/v1` and `AGENT_MARKET_LLM_MODELS=daily-model` -(plus a dummy key file); the chain parks on the fake daily-quota 429 and the built-in agent answers. +(plus a dummy key file); the chain parks on the fake daily-quota 429 and the local model answers (or, without one, the +busy reply). ## Agent API quick start ```bash @@ -206,14 +270,15 @@ curl -s $BASE/api/actions -H 'X-Agent-Id: agent:my-bot' -H 'Content-Type: applic ## Tests ```bash -python3 tests/e2e_http.py http://127.0.0.1:8795 # HTTP end-to-end incl. SPA shell + bundle (24 checks) -bash tests/llm_proxy_check.sh # fake LLM: 429 chain, cooldown, fallback, key handling, rate limit -python3 tests/nl_check.py http://127.0.0.1:8795 # real model: 6 natural-language prompts +python3 tests/e2e_http.py http://127.0.0.1:8795 # HTTP end-to-end incl. SPA shell + bundle (REQUIRE_LLM=1: typed turns must reach a model) +bash tests/llm_proxy_check.sh # fake LLM: 429 chain, cooldown, local model, busy reply (never rules), key handling, rate limit +python3 tests/nl_check.py http://127.0.0.1:8795 # real model (hosted or local): 6 natural-language prompts python3 tests/test_theme.py # theme spec + customCss sanitiser -python3 tests/test_copyedit.py # built-in copy edits (logo/headline/... phrasings) +python3 tests/test_copy_defaults.py # page-text defaults in sync with the app, prompt shows PAGE_TEXT, no rule code left +python3 tests/test_agent_units.py # local-model normalisers (result focus, preset+reset, listing-form shape), primer restart detection NODE_PATH=... node tests/theme_check.js BASE ./screenshots # chat restyles, guardrail contrast, persistence, hostile stored theme, reset (REQUIRE_LLM=1 to insist on the model) -NODE_PATH=/path/to/node_modules node tests/browser_e2e.js http://127.0.0.1:8795 ./screenshots # headless Chrome (14 checks) -NODE_PATH=/path/to/node_modules bash tests/fake_model_browser.sh # React UI vs fake model: badges, fallback text, quota, rate-limit toast, sanitised set_theme +NODE_PATH=/path/to/node_modules node tests/browser_e2e.js http://127.0.0.1:8795 ./screenshots # headless Chrome, needs a model (16 checks) +NODE_PATH=/path/to/node_modules bash tests/fake_model_browser.sh # React UI vs fake model: badges, busy reply + Retry, quota, rate-limit toast, sanitised set_theme ``` ## Known gaps (demo) @@ -221,5 +286,5 @@ NODE_PATH=/path/to/node_modules bash tests/fake_model_browser.sh # React UI vs - There's no real messaging between buyer and seller for cash meetups. - The JSON file store is single-process, and rate limits live in memory, so a restart resets them. - Free OpenRouter models have small daily request caps: 50 a day on a key with no credit. They also get rate-limited upstream. When the daily cap hits, - the server parks the whole model chain until OpenRouter's reset time, and the built-in agent answers, labelled as such. + the server parks the hosted chain until OpenRouter's reset time, and the local model answers, labelled as such. - The mock payment provider is the only one. A real provider would need its own test keys and webhook secret. diff --git a/deploy.sh b/deploy.sh index fe2eff7..c65f987 100755 --- a/deploy.sh +++ b/deploy.sh @@ -20,11 +20,17 @@ echo "deploying $ENVN @ $(git rev-parse --short HEAD) on :$PORT" CONF=$HOME/.config/agent-market set -a [ -f "$CONF/llm.env" ] && . "$CONF/llm.env" +# optional local model, the last link of the chain (llama-server on this box, see README "Local model") +[ -f "$CONF/local-llm.env" ] && . "$CONF/local-llm.env" [ -f "$CONF/$ENVN.env" ] && . "$CONF/$ENVN.env" # Optional extra env file for offline test runs, e.g. point the LLM at tests/mock_llm.py so no real model calls are made: # AM_EXTRA_ENV=/path/offline.env ./deploy.sh staging my-branch [ -n "${AM_EXTRA_ENV:-}" ] && . "$AM_EXTRA_ENV" set +a +# The local model answers when the hosted free models can't, so make sure its watchdog is running (idempotent). +if [ -n "${AGENT_MARKET_LOCAL_LLM_URL:-}" ] && [[ "$AGENT_MARKET_LOCAL_LLM_URL" == http://127.0.0.1:${LLM_PORT:-11434}/* ]]; then + ops/local-llm-watchdog.sh --ensure || echo "warning: local model watchdog not started" +fi # Build the React frontend (web/ -> static/dist) BEFORE stopping the running server, so a failed build # leaves the current version up. Reproducible: exact versions from web/package-lock.json via `npm ci`. command -v npm >/dev/null || { echo "npm not found: Node 20+ is needed to build the frontend"; exit 1; } diff --git a/market/agent.py b/market/agent.py index db22754..cc90949 100644 --- a/market/agent.py +++ b/market/agent.py @@ -4,7 +4,8 @@ Returns {reply, actions} where actions is a list of UI action objects from a fix component set the frontend knows how to render. """ from __future__ import annotations -import concurrent.futures, json, os, re, time, urllib.request, urllib.error +import concurrent.futures +import threading, json, os, re, time, urllib.request, urllib.error from .store import new_id, now FEE_BPS = int(os.environ.get("AGENT_MARKET_FEE_BPS", "500")) # 5.00% @@ -209,12 +210,32 @@ def run_tools(store, tools): return results -# ---------- deterministic fallback (no LLM key) ---------- +# ---------- deterministic handlers for clicks, forms and payments (typed messages always go to a model) ---------- def _search_q(store, q, kind="any", limit=6): return run_tools(store, [{"name": "search", "args": {"q": q, "kind": kind, "limit": limit}}])[0]["listings"] +def featured_listings(store, limit=8): + """Landing view: the first active listings, no model involved.""" + hits = _search_q(store, "", limit=limit) + if not hits: + return {"reply": "No listings yet. Say what you want to sell and I'll set up the form.", "actions": []} + return {"reply": f"Here are {len(hits)} listings. Tell me what you're after, or click a card.", + "actions": [{"type": "show_cards", "payload": {"heading": "Featured listings", "listing_ids": [h["id"] for h in hits]}}]} + + +def sell_form(): + """The "Sell something" button: an empty listing form (the user fills it in and presses Publish).""" + return {"reply": "Fill in the listing form. I'll publish it as test data. The online fee only applies when a buyer pays online.", + "actions": [{"type": "show_listing_form", "payload": {"defaults": { + "title": "", "kind": "product", "price": 2500, "description": "", "category": "misc", "area": "Demo Town", + "payment_options": ["online", "cash"], "emoji": "📦"}}}]} + + +BUSY_REPLY = "The assistant is busy, please try again in a moment." + + def fallback_event(store, ev: dict, actor: str): t = ev.get("type") lid = ev.get("listing_id") @@ -245,7 +266,7 @@ def fallback_event(store, ev: dict, actor: str): "actions": [{"type": "show_compare", "payload": {"listing_ids": ids, "heading": "Compare"}}]} return {"reply": "Pick at least two listings to compare.", "actions": []} if t == "sell": - return fallback_agent(store, "i want to sell", [], actor) + return sell_form() if t == "submit_listing": f = ev.get("fields") or {} try: @@ -281,155 +302,6 @@ def fallback_event(store, ev: dict, actor: str): return {"reply": "I didn't recognise that action.", "actions": []} -def fallback_agent(store, message: str, history, actor: str = "demo-buyer", look=None): - m = (message or "").strip() - low = m.lower() - actions = [] - reply = "" - - from .theme import parse_look_command - th = parse_look_command(m, look if isinstance(look, dict) else {}) - if th: - # payload None = understood a copy edit but couldn't apply it; the reply says why (no listings, no silent no-op) - return {"reply": th[0], "actions": [{"type": "set_theme", "payload": th[1]}] if th[1] else []} - - # welcome / help - if not m or low in ("hi", "hello", "hey", "help", "start"): - reply = ("Welcome to Agent Market (TEST DEMO). I drive the page: ask me to search, " - "compare, list something for sale, check out online (test card 4242…), or arrange a cash meetup. " - f"Online fee is {FEE_RATE*100:.1f}%; cash in person is free.") - actions = [{"type": "show_hero", "payload": { - "title": "Agent Market", - "subtitle": "An AI agent runs this page. List, buy, or meet for cash — all demo data.", - "bullets": [ - "Chat to search, list, buy, or arrange a meetup", - f"Online payments: test mode + {FEE_RATE*100:.1f}% facilitation fee", - "Cash in person: $0 fee — we just connect you", - "Agents can buy and sell via the open JSON API", - ], - }}] - return {"reply": reply, "actions": actions, "agent": "fallback"} - - # sell / list - if any(w in low for w in ("sell", "list something", "create a listing", "post a listing", "i want to sell")): - reply = "Fill in the listing form. I'll publish it as test data. Online fee only applies when a buyer pays online." - actions = [{"type": "show_listing_form", "payload": {"defaults": { - "title": "", "kind": "product", "price": 2500, "description": "", - "category": "misc", "area": "Demo Town", "payment_options": ["online", "cash"], "emoji": "📦", - }}}] - return {"reply": reply, "actions": actions, "agent": "fallback"} - - # cash meetup - if "cash" in low or "meetup" in low or "meet in person" in low or "in person" in low: - listings = _search_q(store, "") - cashable = [L for L in listings if "cash" in (L.get("payment_options") or [])] - # try to pick by keyword - pick = None - for L in cashable: - if any(t in low for t in (L.get("tags") or []) + [L.get("title", "").lower().split()[0]]): - pick = L - break - if not pick and cashable: - pick = cashable[0] - if not pick: - return {"reply": "No cash-friendly listings right now. Try searching first.", "actions": [], "agent": "fallback"} - res = run_tools(store, [{"name": "start_cash_deal", "args": { - "listing_id": pick["id"], "buyer": actor, - "place": pick.get("area") or "Demo Town square", - "time": "Saturday afternoon", - }}])[0] - deal = res["deal"] - reply = (f"Cash deal arranged for “{pick['title']}” at {money(pick['price'])}. " - f"Fee: $0. Meet at {deal['place']} — {deal['time']}. (TEST DEMO, no real meetup.)") - actions = [{"type": "show_cash_meetup", "payload": { - "listing_id": pick["id"], "deal_id": deal["id"], - "suggested_place": deal["place"], "suggested_time": deal["time"], - "fee_cents": 0, "price_cents": pick["price"], - }}] - return {"reply": reply, "actions": actions, "agent": "fallback", "tools": res} - - # buy / checkout - if any(w in low for w in ("buy", "checkout", "purchase", "pay for", "i'll take")): - listings = store.all("listings") - pick = None - # explicit id - m_id = re.search(r"lst_[a-f0-9]+", low) - if m_id: - pick = store.get("listings", m_id.group(0)) - if not pick: - for L in listings: - title_words = [w for w in L.get("title", "").lower().split() if len(w) > 3] - if any(w in low for w in title_words[:3]) or any(t in low for t in (L.get("tags") or [])): - if "online" in (L.get("payment_options") or []): - pick = L - break - if not pick: - online = [L for L in listings if "online" in (L.get("payment_options") or []) and L.get("status") == "active"] - pick = online[0] if online else None - if not pick: - return {"reply": "I couldn't match a listing to buy. Try “show bikes” first.", "actions": [], "agent": "fallback"} - res = run_tools(store, [{"name": "start_checkout", "args": {"listing_id": pick["id"], "buyer": actor}}])[0] - if not res.get("ok"): - return {"reply": f"Can't start online checkout: {res.get('error')}. Try cash instead.", "actions": [], "agent": "fallback"} - intent = res["intent"] - reply = (f"Checkout for “{pick['title']}”. Subtotal {money(intent['subtotal_cents'])} + " - f"{FEE_RATE*100:.1f}% fee {money(intent['fee_cents'])} = {money(intent['total_cents'])}. " - "TEST MODE: use card 4242 4242 4242 4242.") - actions = [{"type": "show_checkout", "payload": { - "listing_id": pick["id"], "intent_id": intent["id"], "quantity": 1, - "subtotal_cents": intent["subtotal_cents"], "fee_cents": intent["fee_cents"], - "fee_bps": FEE_BPS, "total_cents": intent["total_cents"], - }}] - return {"reply": reply, "actions": actions, "agent": "fallback"} - - # compare - if "compare" in low: - q = re.sub(r"compare|vs|versus|and", " ", low) - hits = _search_q(store, q.strip() or "bike", limit=4) - if len(hits) < 2: - hits = _search_q(store, "bike", limit=3) or _search_q(store, "", limit=3) - ids = [h["id"] for h in hits[:3]] - reply = f"Comparing {len(ids)} listings." - actions = [{"type": "show_compare", "payload": {"listing_ids": ids, "heading": "Compare"}}] - return {"reply": reply, "actions": actions, "agent": "fallback"} - - # search / show - q = low - for prefix in ("show me ", "show ", "find ", "search for ", "search ", "looking for ", "list "): - if q.startswith(prefix): - q = q[len(prefix):] - break - kind = "any" - if "service" in q: - kind = "service" - q = q.replace("services", "").replace("service", "") - if "product" in q: - kind = "product" - q = q.replace("products", "").replace("product", "") - max_price = None - mp = re.search(r"(?:under|below|less than|max)\s*\$?(\d+(?:\.\d+)?)", q) - if mp: - max_price = int(float(mp.group(1)) * 100) - q = q[:mp.start()] + q[mp.end():] - stop = {"everything", "all", "anything", "me", "a", "an", "the", "some", "any", "for", "i", "want", "need", "find", "show", "please"} - q = " ".join(w for w in re.sub(r"[^a-z0-9 ]", " ", q).split() if w not in stop) - q = " ".join(w[:-1] if w.endswith("s") and len(w) > 4 and w not in ("glass",) else w for w in q.split()) - hits = _search_q(store, q.strip(), kind=kind, limit=8) - if not hits and q.strip(): - # broader: any token - hits = _search_q(store, "", kind=kind, limit=8) - hits = [h for h in hits if any(tok in (h.get("title","")+h.get("description","")).lower() for tok in q.split() if len(tok)>2)][:6] - if max_price is not None: - hits = [h for h in hits if h["price"] <= max_price] - if hits: - reply = f"Found {len(hits)} listing(s). Click a card, or say “buy the bike” / “cash meetup for the lamp”." - actions = [{"type": "show_cards", "payload": {"heading": f"Results for “{q.strip() or 'everything'}”", "listing_ids": [h["id"] for h in hits]}}] - else: - reply = "Nothing matched. Try “bikes”, “tutoring”, “lamp”, or “lawn”." - actions = [{"type": "toast", "payload": {"level": "warn", "text": "No matches"}}] - return {"reply": reply, "actions": actions, "agent": "fallback"} - - # ---------- LLM path ---------- LLM_TIMEOUT = float(os.environ.get("AGENT_MARKET_LLM_TIMEOUT", "12")) # hard cap per model attempt @@ -437,8 +309,39 @@ LLM_BUDGET = float(os.environ.get("AGENT_MARKET_LLM_BUDGET", "28")) # h _pool = concurrent.futures.ThreadPoolExecutor(max_workers=8, thread_name_prefix="llm") COOLDOWN_429 = float(os.environ.get("AGENT_MARKET_LLM_COOLDOWN", "60")) _cooldown = {} # model -> unix time until which we skip it after a 429 -LAST = {"model": None, "at": None, "error": None} +LAST = {"model": None, "at": None, "error": None, "local": False} QUOTA = {"until": 0} +# Local model (llama.cpp llama-server on this box, OpenAI-compatible): the last link of the chain, after the hosted models. +LOCAL_TIMEOUT = float(os.environ.get("AGENT_MARKET_LOCAL_LLM_TIMEOUT", "45")) # its own cap (CPU inference is slower) +LOCAL_MIN = float(os.environ.get("AGENT_MARKET_LOCAL_LLM_MIN", "25")) # always give the local model at least this long +TOTAL_BUDGET = float(os.environ.get("AGENT_MARKET_LLM_TOTAL_BUDGET", "60")) # hosted chain + local model, per turn +_local_slots = threading.Semaphore(int(os.environ.get("AGENT_MARKET_LOCAL_LLM_CONCURRENCY", "2"))) # 1 running + 1 queued +_QUOTA_FILE = os.path.join(os.environ["AGENT_MARKET_DATA"], "llm_quota.json") if os.environ.get("AGENT_MARKET_DATA") else None + + +def _load_quota(): + """Remember a hosted daily-quota stop across restarts, so a redeploy doesn't spend a call to rediscover it.""" + try: + with open(_QUOTA_FILE) as f: + d = json.load(f) + until = min(float(d.get("until") or 0), time.time() + 36 * 3600) # a daily cap never lasts longer than this + if until > time.time() and d.get("base") == os.environ.get("AGENT_MARKET_LLM_BASE_URL", ""): + QUOTA["until"] = until + except (OSError, ValueError, TypeError, AttributeError): + pass + + +def _save_quota(): + if not _QUOTA_FILE: + return + try: + with open(_QUOTA_FILE, "w") as f: + json.dump({"until": QUOTA["until"], "base": os.environ.get("AGENT_MARKET_LLM_BASE_URL", "")}, f) + except OSError: + pass + + +_load_quota() class LLMError(Exception): @@ -464,31 +367,61 @@ def _read_key(): return None +def _local_config(): + """Local OpenAI-compatible server (no key), e.g. AGENT_MARKET_LOCAL_LLM_URL=http://127.0.0.1:11434/v1.""" + base = (os.environ.get("AGENT_MARKET_LOCAL_LLM_URL") or "").strip() + model = (os.environ.get("AGENT_MARKET_LOCAL_LLM_MODEL") or "").strip() + if not base or not model: + return None + return {"provider": "local", "url": base.rstrip("/") + "/chat/completions", "model": model, "models": [model], "local": True} + + def _llm_config(): - """Hosted LLM config (OpenAI-compatible chat completions) with an ordered model fallback chain.""" - if not _read_key(): + """Hosted LLM config (OpenAI-compatible chat completions) with an ordered model fallback chain, + plus an optional local backup model. None when neither is configured.""" + remote = None + if _read_key(): + base = os.environ.get("AGENT_MARKET_LLM_BASE_URL") or "" + if not base: + base = "https://api.x.ai/v1" if os.environ.get("XAI_API_KEY") else ("https://api.openai.com/v1" if os.environ.get("OPENAI_API_KEY") else "") + models = [m.strip() for m in (os.environ.get("AGENT_MARKET_LLM_MODELS") or os.environ.get("AGENT_MARKET_LLM_MODEL") or "").split(",") if m.strip()] + if base and models: + remote = {"provider": base.split("//")[-1].split("/")[0], "url": base.rstrip("/") + "/chat/completions", + "models": models, "model": models[0]} + local = _local_config() + if not remote and not local: return None - base = os.environ.get("AGENT_MARKET_LLM_BASE_URL") or "" - if not base: - base = "https://api.x.ai/v1" if os.environ.get("XAI_API_KEY") else ("https://api.openai.com/v1" if os.environ.get("OPENAI_API_KEY") else "") - models = [m.strip() for m in (os.environ.get("AGENT_MARKET_LLM_MODELS") or os.environ.get("AGENT_MARKET_LLM_MODEL") or "").split(",") if m.strip()] - if not base or not models: - return None - return {"provider": base.split("//")[-1].split("/")[0], "url": base.rstrip("/") + "/chat/completions", - "models": models, "model": models[0]} + cfg = remote or {"provider": "local", "url": None, "models": [], "model": local["model"]} + cfg["local"] = local + return cfg -def _call_llm(cfg, model, messages, json_mode=True): +# GBNF for "exactly one JSON object" (llama.cpp grammar syntax). Used instead of response_format for the local model: +# it is cheap to sample with and the object ends at the closing brace (no trailing whitespace run). +JSON_GRAMMAR = r'''root ::= "{" ws ( string ":" ws value ("," ws string ":" ws value)* )? "}" +value ::= object | array | string | number | ("true" | "false" | "null") ws +object ::= "{" ws ( string ":" ws value ("," ws string ":" ws value)* )? "}" ws +array ::= "[" ws ( value ("," ws value)* )? "]" ws +string ::= "\"" ( [^"\\\x7F\x00-\x1F] | "\\" (["\\bfnrt] | "u" [0-9a-fA-F]{4}) )* "\"" ws +number ::= ("-"? ([0-9] | [1-9] [0-9]{0,15})) ("." [0-9]+)? ([eE] [-+]? [0-9]{1,3})? ws +ws ::= | " " | "\n" [ ]{0,12} +''' + + +def _call_llm(cfg, model, messages, json_mode=True, timeout=None): payload = {"model": model, "messages": messages, "temperature": 0.2, "max_tokens": 1100} - if json_mode: - payload["response_format"] = {"type": "json_object"} - req = urllib.request.Request(cfg["url"], data=json.dumps(payload).encode(), method="POST", headers={ - "Authorization": "Bearer " + _read_key(), - "Content-Type": "application/json", - "X-Title": "Agent Market demo", - }) + headers = {"Content-Type": "application/json", "X-Title": "Agent Market demo"} + if cfg.get("local") is True: # the local server itself (a hosted cfg carries its local backup as a dict) + payload["max_tokens"] = 700 + if json_mode: + payload["grammar"] = JSON_GRAMMAR # llama-server: constrain the output to one JSON object + else: + if json_mode: + payload["response_format"] = {"type": "json_object"} + headers["Authorization"] = "Bearer " + _read_key() + req = urllib.request.Request(cfg["url"], data=json.dumps(payload).encode(), method="POST", headers=headers) try: - with urllib.request.urlopen(req, timeout=LLM_TIMEOUT) as r: + with urllib.request.urlopen(req, timeout=timeout or LLM_TIMEOUT) as r: data = json.loads(r.read().decode()) except urllib.error.HTTPError as e: body = e.read().decode(errors="replace")[:2000] @@ -497,10 +430,12 @@ def _call_llm(cfg, model, messages, json_mode=True): if "per-day" in body or "daily" in body: raise LLMError("daily_quota", m.group(1) if m else "") raise LLMError("rate_limited", body[:300]) - if e.code == 400 and json_mode and "response_format" in body: - return _call_llm(cfg, model, messages, json_mode=False) + if e.code == 400 and json_mode and ("response_format" in body or "grammar" in body or "sampler" in body): + return _call_llm(cfg, model, messages, json_mode=False, timeout=timeout) raise LLMError(f"http_{e.code}", body[:300]) except (urllib.error.URLError, TimeoutError, OSError) as e: + if isinstance(e, TimeoutError) or "timed out" in str(e): + raise LLMError("timeout", type(e).__name__) raise LLMError("timeout_or_network", type(e).__name__) if isinstance(data, dict) and data.get("error"): err = data["error"] @@ -548,41 +483,213 @@ def _page_text(look): "chips": [x.get("label") for x in c["chips"]], "bullets": c["bullets"]}, ensure_ascii=False) -def _llm_once(store, cfg, model, message, history, look=""): - sys_prompt = (UI_SCHEMA + "\nCATALOG (all active listings):\n" + json.dumps(_catalog(store))[:9000] + - "\nCURRENT_LOOK (this visitor's theme overrides; {} = default dark look):\n" + (look or "{}") + - "\nPAGE_TEXT (what the editable text slots show right now):\n" + _page_text(look)) +LOCAL_EXAMPLES = """ +EXAMPLES (visitor message => your whole answer). Follow the pattern; use the visitor's own words and CATALOG ids. +"change the name in the top left to Bazaar Hub" => {"reply":"Done: the top-left logo now says Bazaar Hub.","actions":[{"type":"set_theme","payload":{"copy":{"logo":"Bazaar Hub"}}}]} +"swap Agent Market for Corner Shop in the header" => {"reply":"The header now says Corner Shop.","actions":[{"type":"set_theme","payload":{"copy":{"logo":"Corner Shop"}}}]} +"make the big title say Hello neighbours" => {"reply":"Headline updated.","actions":[{"type":"set_theme","payload":{"copy":{"headline":"Hello neighbours"}}}]} +"get rid of the bullet points" => {"reply":"Bullet points hidden.","actions":[{"type":"set_theme","payload":{"sections":{"hidden":["bullets"]}}}]} +"go back to the original look" => {"reply":"Back to the default look.","actions":[{"type":"set_theme","payload":{"reset":true}}]} +"light theme please, with an orange accent" => {"reply":"Light mode with an orange accent.","actions":[{"type":"set_theme","payload":{"mode":"light","palette":{"accent":"#f76b15"}}}]} +"one item per row instead of a grid" => {"reply":"Results now show as a list.","actions":[{"type":"set_theme","payload":{"layout":"list"}}]} +"put the chat on the left side" => {"reply":"Chat moved to the left.","actions":[{"type":"set_theme","payload":{"chat":{"position":"left"}}}]} +"use a mono font, slightly larger" => {"reply":"Mono font, a bit larger.","actions":[{"type":"set_theme","payload":{"font":"mono","fontScale":1.15}}]} +"a teal to purple gradient behind everything" => {"reply":"Teal to purple gradient background.","actions":[{"type":"set_theme","payload":{"background":{"kind":"gradient","colors":["#12a594","#8e4ec6"],"angle":135}}}]} +"make it neon" => {"reply":"Neon look applied.","actions":[{"type":"set_theme","payload":{"preset":"neon"}}]} +"I'd like to sell my old sofa for $60, cash only" => {"reply":"I've filled in the listing form. Check it and press Publish.","actions":[{"type":"show_listing_form","payload":{"defaults":{"title":"Old sofa","kind":"product","price":6000,"description":"Used sofa in good condition.","category":"home","area":"Demo Town","payment_options":["cash"]}}}]} +"" => {"reply":"","actions":[{"type":"show_cards","payload":{"heading":"","listing_ids":[""]}}]} +"I'll buy and pay online" => {"reply":"Here's the checkout (price + fee).","actions":[{"type":"show_checkout","payload":{"listing_id":""}}]} +"can I pay cash for in person?" => {"reply":"Cash meetup set up, no fee.","actions":[{"type":"show_cash_meetup","payload":{"listing_id":"","suggested_place":"Demo Town square","suggested_time":"Saturday 10am"}}]} +Only change what the visitor asked for. A visual or text request returns set_theme only (no listings). +""" + + +# Small models miscopy 8-hex ids (lst_ffe42107 vs lst_29d7f320), so the local model sees short catalog refs (L1, L2, ...) +# that are mapped back to real ids before anything runs. Unknown refs are dropped like any invented id. +LOCAL_SCHEMA = UI_SCHEMA.replace('"lst_..."', '"L7"').replace("internal ids (lst_...,", "internal ids (L7, lst_...,") +ALIAS_RE = re.compile(r"\s*\(?\bL\d{1,3}\b\)?:?") + + +def _aliases(store): + """ref -> real listing id, in catalog order (stable while the catalog is unchanged, so the prompt stays cacheable).""" + return {f"L{i}": c["id"] for i, c in enumerate(_catalog(store), 1)} + + +def _unalias(obj, alias): + """Map L-refs back to real listing ids in action payloads / tool args (in place).""" + if isinstance(obj, dict): + for k, v in list(obj.items()): + if k == "listing_id" and isinstance(v, str): + obj[k] = alias.get(v.strip(), v) + elif k == "listing_ids" and isinstance(v, list): + obj[k] = [alias.get(x.strip(), x) if isinstance(x, str) else x for x in v] + else: + _unalias(v, alias) + elif isinstance(obj, list): + for x in obj: + _unalias(x, alias) + return obj + + +def _system_prompt(store, look="", local=False): + # stable parts first (schema, examples, catalog) so a local model's prompt cache can reuse them across turns + cat = _catalog(store) + if local: + cat = [{**c, "id": f"L{i}"} for i, c in enumerate(cat, 1)] + return ((LOCAL_SCHEMA + LOCAL_EXAMPLES if local else UI_SCHEMA) + "\nCATALOG (all active listings):\n" + json.dumps(cat)[:9000] + + "\nCURRENT_LOOK (this visitor's theme overrides; {} = default dark look):\n" + (look or "{}") + + "\nPAGE_TEXT (what the editable text slots show right now):\n" + _page_text(look)) + + +_PRIMED = {"key": None, "decodes": None} + + +def _local_restarted(local): + """True when the local server is down or was restarted since we last looked (its llama-server /metrics counter went + back down), so its prompt cache is cold again and must be re-primed. Unknown servers (no /metrics) -> False.""" + root = local["url"].split("/v1/")[0] + try: + with urllib.request.urlopen(root + "/metrics", timeout=2) as r: + m = re.search(r"^llamacpp:n_decode_total (\d+)", r.read().decode(errors="replace"), re.M) + except urllib.error.HTTPError: + return False + except Exception: + _PRIMED["decodes"] = None + return True + if not m: + return False + n, last = int(m.group(1)), _PRIMED["decodes"] + _PRIMED["decodes"] = n + return last is not None and n < last + + +def _prime_local_once(store): + """Process the current system prompt on the local model in the background (1 output token), so the next visitor + turn reuses llama-server's prompt cache instead of paying ~30 s of CPU prompt processing.""" + local = _local_config() + if not local: + return False + if _local_restarted(local): + _PRIMED["key"] = None + sp = _system_prompt(store, "", local=True) + key = hash(sp) + if key == _PRIMED["key"] or not _local_slots.acquire(blocking=False): + return False + try: + body = {"model": local["model"], "messages": [{"role": "system", "content": sp}, {"role": "user", "content": _visitor("hi")}], + "max_tokens": 1, "temperature": 0} + req = urllib.request.Request(local["url"], data=json.dumps(body).encode(), headers={"Content-Type": "application/json"}) + with urllib.request.urlopen(req, timeout=180) as r: + r.read() + _PRIMED["key"] = key + _local_restarted(local) # remember the counter after our own call + return True + except Exception: + return False + finally: + _local_slots.release() + + +def start_local_primer(store, every=30): + """Background thread: prime at start-up, whenever the catalog (and so the prompt) changes, and after the local server + restarts. Between those it only reads /metrics every 30 s, so an idle server stays idle.""" + if not _local_config() or os.environ.get("AGENT_MARKET_LOCAL_LLM_PRIME", "1") != "1": + return + + def loop(): + while True: + try: + _prime_local_once(store) + except Exception: + pass + time.sleep(every) + threading.Thread(target=loop, name="local-llm-primer", daemon=True).start() + + +def _visitor(message): + # small models (and chat templates that fold the system prompt into the first user turn) need a clear boundary + return "\n\nVISITOR MESSAGE (answer this one with the JSON object):\n" + message + + +def _llm_once(store, cfg, model, message, history, look="", timeout=None): + local = cfg.get("local") is True + sys_prompt = _system_prompt(store, look, local=local) msgs = [{"role": "system", "content": sys_prompt}] for h in (history or [])[-6:]: if h.get("role") in ("user", "assistant") and h.get("content"): msgs.append({"role": h["role"], "content": str(h["content"])[:600]}) - msgs.append({"role": "user", "content": message}) - content = _call_llm(cfg, model, msgs) - plan = _parse_agent_json(content) + msgs.append({"role": "user", "content": _visitor(message) if local else message}) + t_end = time.time() + (timeout or LLM_TIMEOUT) + alias = _aliases(store) if local else {} + content = _call_llm(cfg, model, msgs, timeout=timeout) + plan = _unalias(_parse_agent_json(content), alias) tool_results = [] tools = [t for t in (plan.get("tools") or []) if isinstance(t, dict)][:3] if tools: tool_results = run_tools(store, tools) + tr = json.dumps(tool_results) + for ref, real in alias.items(): + tr = tr.replace(real, ref) msgs += [{"role": "assistant", "content": content}, - {"role": "user", "content": "TOOL_RESULTS:\n" + json.dumps(tool_results)[:6000] + + {"role": "user", "content": "TOOL_RESULTS:\n" + tr[:6000] + "\nNow return the final {\"reply\",\"actions\"} JSON only."}] - plan = _parse_agent_json(_call_llm(cfg, model, msgs)) + left = t_end - time.time() + if left < 2: + raise LLMError("timeout", "no time left for the tool round") + plan = _unalias(_parse_agent_json(_call_llm(cfg, model, msgs, timeout=left)), alias) actions = [a for a in (plan.get("actions") or []) if isinstance(a, dict) and a.get("type") in ALLOWED_ACTIONS] reply = str(plan.get("reply") or "").strip() if not reply and not actions: raise LLMError("empty_plan", "") reply = ID_RE.sub("", reply).strip() + if local: + reply = ALIAS_RE.sub("", reply).strip() return {"reply": (reply or "Done.")[:600], "actions": actions, "tools": tool_results} +_STOP = set("show me find search look looking for want need some any the a an and or with under over below above cheap " + "please can could you i my our get buy see list all everything something anything thing things stuff near " + "options option there here have has what which that this from about like".split()) + + +def _focus_results(store, message, actions): + """Small local models sometimes pad show_cards with unrelated listings. If some chosen listings share a keyword with + the request and others share none, keep only the matching ones (semantic picks with no keyword overlap stay).""" + words = {w.rstrip("s") for w in re.findall(r"[a-z]{3,}", (message or "").lower()) if w not in _STOP} + if not words: + return actions + for a in actions: + if a.get("type") != "show_cards" or not isinstance(a.get("payload"), dict): + continue + ids = [i for i in (a["payload"].get("listing_ids") or []) if isinstance(i, str)] + def hit(i): + L = store.get("listings", i) or {} + text = " ".join([L.get("title", ""), L.get("category") or "", " ".join(L.get("tags") or []), L.get("description", "")]).lower() + toks = {t.rstrip("s") for t in re.findall(r"[a-z]{3,}", text)} + return bool(words & toks) + keep = [i for i in ids if hit(i)] + if keep and len(keep) < len(ids): + a["payload"]["listing_ids"] = keep + return actions + + +LOCAL_HISTORY = int(os.environ.get("AGENT_MARKET_LOCAL_LLM_HISTORY", "0")) # small models copy earlier turns; 0 = no history + + def llm_agent(store, message, history, cfg, look=""): - """Try each model in the chain; skip models cooling down after a 429. Raises if all fail.""" + """Hosted models in order (skipping ones cooling down after a 429, and all of them while the daily free quota is + used up), then the local model with its own timeout. Raises if everything fails (the visitor gets BUSY_REPLY).""" errors = [] now_t = time.time() + start = now_t + local = cfg.get("local") if QUOTA["until"] > now_t: - raise LLMError("all_models_failed", json.dumps([{"model": "*", "error": "daily_quota", "ms": 0}])) - chain = [m for m in cfg["models"] if _cooldown.get(m, 0) <= now_t] or cfg["models"][-1:] - start = time.time() + chain = [] + errors.append({"model": "*", "error": "daily_quota", "ms": 0}) + else: + chain = [m for m in cfg["models"] if _cooldown.get(m, 0) <= now_t] + if not chain and not local: + chain = cfg["models"][-1:] for model in chain: t0 = time.time() remaining = LLM_BUDGET - (t0 - start) @@ -597,7 +704,7 @@ def llm_agent(store, message, history, cfg, look=""): raise LLMError("timeout", f">{min(LLM_TIMEOUT, remaining):.0f}s") out.update(agent="llm", provider=cfg["provider"], model=model, latency_ms=int((time.time() - t0) * 1000), fallbacks=errors) - LAST.update(model=model, at=time.time(), error=None) + LAST.update(model=model, at=time.time(), error=None, local=False) return out except LLMError as e: if e.kind == "rate_limited": @@ -609,11 +716,39 @@ def llm_agent(store, message, history, cfg, look=""): for mm in cfg["models"]: _cooldown[mm] = max(_cooldown.get(mm, 0), until) QUOTA["until"] = until + _save_quota() errors.append({"model": model, "error": e.kind, "ms": int((time.time() - t0) * 1000)}) break errors.append({"model": model, "error": e.kind, "ms": int((time.time() - t0) * 1000)}) except Exception as e: # never let the page break errors.append({"model": model, "error": type(e).__name__, "ms": int((time.time() - t0) * 1000)}) + if local: + t0 = time.time() + cap = min(LOCAL_TIMEOUT, max(LOCAL_MIN, TOTAL_BUDGET - (t0 - start))) + if not _local_slots.acquire(blocking=False): + errors.append({"model": local["model"], "error": "local_busy", "ms": 0}) + else: + try: + hist = (history or [])[-LOCAL_HISTORY:] if LOCAL_HISTORY > 0 else [] + fut = _pool.submit(_llm_once, store, local, local["model"], message, hist, look, cap) + except Exception: + _local_slots.release() + raise + fut.add_done_callback(lambda f: _local_slots.release()) # free the slot when the call really ends + try: + try: + out = fut.result(timeout=cap + 1) + except concurrent.futures.TimeoutError: + raise LLMError("timeout", f">{cap:.0f}s") + out["actions"] = _focus_results(store, message, out["actions"]) + out.update(agent="llm", provider="local", local=True, model=local["model"], + latency_ms=int((time.time() - t0) * 1000), fallbacks=errors) + LAST.update(model=local["model"], at=time.time(), error=None, local=True) + return out + except LLMError as e: + errors.append({"model": local["model"], "error": e.kind, "ms": int((time.time() - t0) * 1000)}) + except Exception as e: + errors.append({"model": local["model"], "error": type(e).__name__, "ms": int((time.time() - t0) * 1000)}) LAST.update(error=errors[-1]["error"] if errors else "no_model", at=time.time()) raise LLMError("all_models_failed", json.dumps(errors)) @@ -624,44 +759,67 @@ def llm_agent(store, message, history, cfg, look=""): LLM_EVENTS = os.environ.get("AGENT_MARKET_LLM_EVENTS") == "1" +def busy_reply(errors=None, err="no_model", ms=0): + """Every model failed (or none is configured): say so honestly. Typed messages never get rule-based answers.""" + return {"reply": BUSY_REPLY, "actions": [], "agent": "unavailable", "busy": True, "retry": True, + "llm_error": err, "fallbacks": errors or [], "latency_ms": ms} + + def handle_chat(store, message: str, history=None, event=None, actor="demo-buyer", look=""): actor = (actor or "demo-buyer")[:60] t0 = time.time() - try: - look_d = json.loads(look) if look else {} - except ValueError: - look_d = {} if event and (event.get("type") == "landing" or not LLM_EVENTS): - if event.get("type") == "landing": - out = fallback_agent(store, "show me everything", [], actor) - else: - out = fallback_event(store, event, actor) + out = featured_listings(store) if event.get("type") == "landing" else fallback_event(store, event, actor) out.update(agent="ui-event", latency_ms=int((time.time() - t0) * 1000)) return out cfg = _llm_config() - if cfg: - text = message if not event else "UI_EVENT " + json.dumps(event)[:2000] + f"\n(actor/buyer id: {actor})" - try: - return llm_agent(store, text, history or [], cfg, look) - except Exception as e: - out = fallback_event(store, event, actor) if event else fallback_agent(store, message, history or [], actor, look_d) - out.update(agent="fallback", llm_error=getattr(e, "kind", type(e).__name__), - fallbacks=json.loads(str(e).split(": ", 1)[1]) if getattr(e, "kind", "") == "all_models_failed" else [], - latency_ms=int((time.time() - t0) * 1000)) - out["reply"] = "(AI model unavailable right now; the built-in agent answered.) " + out["reply"] + if not cfg: + if event: # AGENT_MARKET_LLM_EVENTS=1 but no model: clicks still work deterministically + out = fallback_event(store, event, actor) + out.update(agent="ui-event", latency_ms=int((time.time() - t0) * 1000)) return out - out = fallback_event(store, event, actor) if event else fallback_agent(store, message, history or [], actor, look_d) - out.update(agent="fallback", latency_ms=int((time.time() - t0) * 1000)) - return out + return busy_reply(err="no_model_configured", ms=int((time.time() - t0) * 1000)) + text = message if not event else "UI_EVENT " + json.dumps(event)[:2000] + f"\n(actor/buyer id: {actor})" + try: + return llm_agent(store, text, history or [], cfg, look) + except Exception as e: + kind = getattr(e, "kind", type(e).__name__) + errors = json.loads(str(e).split(": ", 1)[1]) if kind == "all_models_failed" else [] + if event: + out = fallback_event(store, event, actor) + out.update(agent="ui-event", llm_error=kind, fallbacks=errors, latency_ms=int((time.time() - t0) * 1000)) + return out + return busy_reply(errors, kind, int((time.time() - t0) * 1000)) + + +_LOCAL_PING = {"at": 0, "ok": None} + + +def _local_up(local): + """Is the local model server answering? (cached 15 s; loopback only)""" + if time.time() - _LOCAL_PING["at"] > 15: + try: + with urllib.request.urlopen(local["url"].rsplit("/chat/completions", 1)[0] + "/models", timeout=1.5) as r: + _LOCAL_PING["ok"] = r.status == 200 + except Exception: + _LOCAL_PING["ok"] = False + _LOCAL_PING["at"] = time.time() + return _LOCAL_PING["ok"] def agent_status(): cfg = _llm_config() if cfg: now_t = time.time() - return {"mode": "llm", "provider": cfg["provider"], "model": cfg["models"][0], "chain": cfg["models"], + local = cfg.get("local") + return {"mode": "llm", "provider": cfg["provider"], "model": (cfg["models"] or [None])[0] or (local or {}).get("model"), + "chain": cfg["models"] + ([f"local:{local['model']}"] if local else []), + "local_model": local["model"] if local else None, "local_ok": _local_up(local) if local else None, "cooling_down": [m for m in cfg["models"] if _cooldown.get(m, 0) > now_t], - "last_model": LAST["model"], "last_error": LAST["error"], "ui_events": "llm" if LLM_EVENTS else "deterministic", + "last_model": LAST["model"], "last_local": LAST["local"], "last_error": LAST["error"], + "ui_events": "llm" if LLM_EVENTS else "deterministic", "daily_quota_exhausted_until_utc": time.strftime("%Y-%m-%dT%H:%MZ", time.gmtime(QUOTA["until"])) if QUOTA["until"] > now_t else None} - return {"mode": "fallback", "provider": "deterministic", "model": None, - "note": "No hosted LLM configured (set AGENT_MARKET_LLM_API_KEY_FILE or AGENT_MARKET_LLM_API_KEY, plus AGENT_MARKET_LLM_BASE_URL and AGENT_MARKET_LLM_MODELS). Using the labelled rule-based fallback agent."} + return {"mode": "unavailable", "provider": None, "model": None, "chain": [], + "note": "No model configured (set AGENT_MARKET_LLM_API_KEY_FILE + AGENT_MARKET_LLM_BASE_URL + AGENT_MARKET_LLM_MODELS " + "and/or AGENT_MARKET_LOCAL_LLM_URL + AGENT_MARKET_LOCAL_LLM_MODEL). Typed messages get the busy reply; " + "clicks, forms and payments still work."} diff --git a/market/copyedit.py b/market/copyedit.py index bb1f77f..e156d25 100644 --- a/market/copyedit.py +++ b/market/copyedit.py @@ -1,16 +1,11 @@ -"""Rule-based UI copy edits for the built-in agent (used when no hosted/local model answers). +"""Editable page text (copy slots): the defaults and what a visitor currently sees. -Understands e.g. "replace agent market to new internet on the top left corner", "rename the logo to X", -"call the site X", "top left should say X", "change the title to X", "replace A with B" (generic find-and-replace -across the editable copy slots: logo, headline, subtitle, footer, chip labels, bullets). -Returns (reply, patch) where patch is a set_theme patch, or (reply, None) when it understood a copy edit but -couldn't apply it (the reply says what it understood and offers the closest option), or None (not a copy edit). +Used to tell the model what each text slot shows right now (PAGE_TEXT in the prompt). Text edits themselves are +made by the model with set_theme copy; there is no rule-based text editing. """ -import difflib import os -import re -# Keep in sync with copyDefaults in web/src/App.tsx (tests/test_copyedit.py checks this). +# Keep in sync with copyDefaults in web/src/App.tsx (tests/test_copy_defaults.py checks this). DEFAULT_COPY = { "logo": "Agent Market", "headline": "A marketplace you can just talk to.", @@ -23,30 +18,11 @@ DEFAULT_COPY = { "bullets": ["{fee}% facilitation fee when you pay online (test mode)", "$0 fee when you meet and pay cash in person", "Agents welcome: other AI agents buy and sell through the open API (see For agents)"], } -LIMITS = {"logo": 40, "headline": 120, "subhead": 240, "footer": 160, "chip": 40, "bullet": 140} -NAMES = {"logo": "top-left logo text", "headline": "headline", "subhead": "subtitle", "footer": "footer"} - -# location phrases -> slot (word-bounded; order matters: specific before generic) -LOCS = [ - ("subhead", r"sub-?title|sub-?head(?:ing|line)?|tag-?line|intro(?:duction)? text|(?:text|line) (?:under|below) the (?:headline|title|heading)|description"), - ("logo", r"top[- ]?left(?: corner)?(?: text)?|upper[- ]?left(?: corner)?(?: text)?|left[- ]?top|logo(?: text| name)?|brand(?:ing| name| text)?|" - r"site[- ]?(?:name|title)|website (?:name|title)|name of (?:the |this )?(?:site|website|page|store|shop|marketplace|market)|" - r"(?:store|shop|marketplace|market|app) name|header(?: text| title| name)?|word-?mark|nav(?:bar)? (?:text|title|name)|" - r"(?:corner|top) text|title (?:in|at|on) the (?:top[- ]?left|header|corner|top)"), - ("footer", r"footer(?: text| line)?|bottom(?: text| line)?|text at the bottom(?: of the page)?|(?:last|bottom) line"), - ("headline", r"(?:main |big |page |hero )?(?:headline|heading|title|header text)|big text|main text|hero(?: text)?|h1"), -] -LOC_RE = "|".join(f"(?:{p})" for _, p in LOCS) -STRONG = r"replace|rename|swap|re-?word|retitle|re-?label" -WEAK = r"change|switch|turn|update|edit|set|make|alter|modify|put|write" -SEPS = r"to|with|into|for|by|->|→|=>|=|as" -STYLE_WORDS = r"background|colou?r|font|mode|accent|gradient|theme|look|style|layout|chat|column|corners?|radius|dark|light|preset|" \ - r"pattern|spacing|density|animation|cards?|grid|list|size|bigger|smaller|order|card" def current_copy(theme, fee_bps=None): - fee_bps = fee_bps if fee_bps is not None else int(os.environ.get("AGENT_MARKET_FEE_BPS", "500")) """The text each editable slot shows right now for this visitor (defaults + their overrides).""" + fee_bps = fee_bps if fee_bps is not None else int(os.environ.get("AGENT_MARKET_FEE_BPS", "500")) fee = f"{fee_bps / 100:.{1 if fee_bps % 100 else 0}f}" c = {k: (list(v) if isinstance(v, list) else v) for k, v in DEFAULT_COPY.items()} c["bullets"] = [b.replace("{fee}", fee) for b in c["bullets"]] @@ -59,206 +35,3 @@ def current_copy(theme, fee_bps=None): elif k == "bullets" and isinstance(v, list): c["bullets"] = [x for x in v if isinstance(x, str)] return c - - -def _slot_of(phrase): - """'the top left corner' -> 'logo'; returns None unless the phrase is (only) a location.""" - p = re.sub(r"^(?:the |this |that |our |my |your |its )?(?:(?:text|words?|wording|copy|writing|label|name)\s+(?:in|on|at|of|for)\s+(?:the\s+)?)?", "", - phrase.strip().lower()) - p = re.sub(r"\s+(?:text|area|section|part|bit|corner)$", "", p).strip() - p = re.sub(r"^(?:in|on|at)\s+(?:the\s+)?", "", p) - for slot, pat in LOCS: - if re.fullmatch(f"(?:{pat})(?: of (?:the |this )?(?:page|site|screen|website))?", p): - return slot - return None - - -def _strip_loc_suffix(b): - """'new internet on the top left corner' -> ('new internet', 'logo').""" - m = re.search(r"\s+(?:on|in|at|for|up in|over in|inside|within)\s+(?:the\s+|this\s+)?(" + LOC_RE + r")" - r"(?:\s+(?:corner|area|section|bar|part|text))?(?:\s+of\s+(?:the\s+|this\s+)?(?:page|site|screen|website))?\s*$", b, re.I) - if m: - return b[:m.start()].rstrip(), _slot_of(m.group(1)) - return b, None - - -def _clean(s): - s = s.strip() - s = re.sub(r"\s+(?:please|pls|thanks|thank you|instead|now|for me)\s*[.!]*$", "", s, flags=re.I).strip() - q = re.fullmatch(r"[\"“”'‘’`](.+)[\"“”'‘’`]", s) - if q: - return q.group(1).strip(), True - return s.rstrip(" .!?"), False - - -def _match_case(new, old): - """Keep the case the user typed for the new text; only an all-lowercase entry adopts the replaced text's style - (so 'new internet' replacing 'Agent Market' becomes 'New Internet').""" - if not new or new != new.lower() or not re.search(r"[A-Za-z]", old or ""): - return new - words = [w for w in re.findall(r"[A-Za-z][\w'’-]*", old)] - if old.isupper() and len(old) > 1: - return new.upper() - if words and all(w[0].isupper() for w in words): - return re.sub(r"(^|(?<=[\s\-/]))([a-z])", lambda m: m.group(1) + m.group(2).upper(), new) - if words and words[0][0].isupper(): - return new[:1].upper() + new[1:] - return new - - -def _norm(s): - return re.sub(r"\s+", " ", re.sub(r"[\"“”'‘’`.!?]+$", "", (s or "").strip().lower())) - - -def _slots(cur): - out = [("logo", None, cur["logo"]), ("headline", None, cur["headline"]), ("subhead", None, cur["subhead"]), ("footer", None, cur["footer"])] - out += [("chip", i, ch["label"]) for i, ch in enumerate(cur["chips"])] - out += [("bullet", i, b) for i, b in enumerate(cur["bullets"])] - return out - - -def _label(slot, i, text): - """Name of a slot for replies, with its article ("the headline", "the “📐 Tutoring” button").""" - if slot == "chip": - return f"the “{text}” button" - if slot == "bullet": - return f"bullet {i + 1}" - return "the " + NAMES[slot] - - -def _apply(cur, slot, i, new, old_fragment=None): - """Build the patch. old_fragment set => replace that substring (case-insensitive) inside the slot's text.""" - def sub(text): - if old_fragment is None: - return _match_case(new, text) - rx = re.compile(re.escape(old_fragment), re.I) - return rx.sub(lambda m: _match_case(new, m.group(0)), text) - if slot in ("logo", "headline", "subhead", "footer"): - val = sub(cur[slot])[:LIMITS[slot]] - return {"copy": {slot: val}}, val - if slot == "chip": - chips = [dict(c) for c in cur["chips"]] - chips[i]["label"] = sub(chips[i]["label"])[:LIMITS["chip"]] - return {"copy": {"chips": chips}}, chips[i]["label"] - bullets = list(cur["bullets"]) - bullets[i] = sub(bullets[i])[:LIMITS["bullet"]] - return {"copy": {"bullets": bullets}}, bullets[i] - - -def _find(cur, a): - """Where does text A appear? Exact whole-slot matches first, then substring matches.""" - na = _norm(a) - if not na: - return [], [] - exact = [(s, i, t) for s, i, t in _slots(cur) if _norm(t) == na or _norm(re.sub(r"^\W+", "", t)) == na] - sub = [(s, i, t) for s, i, t in _slots(cur) if na in (t or "").lower() and (s, i, t) not in exact] - return exact, sub - - -def _closest(cur, a): - best, score = None, 0.0 - for s, i, t in _slots(cur): - r = difflib.SequenceMatcher(None, _norm(a), _norm(t)).ratio() - if r > score: - best, score = (s, i, t), r - return best if score >= 0.45 else None - - -def _candidates(text): - """Yield (A, B, kind) readings of the message; A may be a location or literal text.""" - t = re.sub(r"^(?:hey|hi|ok|okay|so)[,!]?\s+", "", text.strip(), flags=re.I) - t = re.sub(r"^(?:please|pls|can you|could you|would you|will you|i(?:'d| would) like (?:you )?to|i want (?:you )?to|let'?s)\s+", "", t, flags=re.I) - t = re.sub(r"^(?:please)\s+", "", t, flags=re.I) - # two quoted strings: replace "A" with "B" - q = re.findall(r"[\"“‘'`]([^\"“”‘’'`]+)[\"”’'`]", t) - if len(q) >= 2 and re.match(rf"(?:{STRONG}|{WEAK})\b", t, re.I): - yield q[0], q[1], "pair" - # verb A sep B (try every separator position; caller picks the reading that resolves) - m = re.match(rf"(?P{STRONG}|{WEAK})\s+(?P.+)$", t, re.I | re.S) - if m: - rest = m.group("rest") - for sm in re.finditer(rf"\s+(?:{SEPS})\s+|\s*(?:->|→|=>|=)\s*", rest, re.I): - a, b = rest[:sm.start()], rest[sm.end():] - if a.strip() and b.strip(): - yield a, b, m.group("v").lower() - # LOC should say / make LOC say / I want LOC to say / LOC: B - m = re.match(rf"(?:make|have|let|get)?\s*(?:the\s+)?(?P.+?)\s+(?:should|must|needs? to|has to|to|will|shall)?\s*" - rf"(?:say|says|read|reads|show|shows|display|displays|be|become|state)\s*:?\s+(?P.+)$", t, re.I) - if m and _slot_of(m.group("loc")): - yield m.group("loc"), m.group("b"), "say" - m = re.match(rf"(?:i want|i'd like)\s+(?:the\s+)?(?P.+?)\s+to\s+(?:say|read|show|be)\s+(?P.+)$", text.strip(), re.I) - if m and _slot_of(m.group("loc")): - yield m.group("loc"), m.group("b"), "say" - m = re.match(r"(?P[^:]{2,40}?)\s*:\s*(?P.+)$", t) - if m and _slot_of(m.group("loc")): - yield m.group("loc"), m.group("b"), "say" - # call the site X / name it X / rename the site X / retitle the site X - m = re.match(r"(?:call|name|rename|retitle|title|brand)\s+(?:the |this |our |my )?(?:site|website|page|store|shop|marketplace|market|app|it)" - r"\s+(?:to\s+|as\s+)?(?P.+)$", t, re.I) - if m: - yield "logo", m.group("b"), "call" - m = re.match(r"use\s+(?P.+?)\s+as\s+(?:the\s+)?(?P.+)$", t, re.I) - if m and _slot_of(m.group("loc")): - yield m.group("loc"), m.group("b"), "say" - - -def parse_copy_edit(text, theme=None, fee_bps=None): - fee_bps = fee_bps if fee_bps is not None else int(os.environ.get("AGENT_MARKET_FEE_BPS", "500")) - if not text or len(text) > 400: - return None - cur = current_copy(theme, fee_bps) - low = text.lower() - fallback_understood = None - for a_raw, b_raw, kind in _candidates(text): - b, hint = _strip_loc_suffix(_clean(b_raw)[0]) if kind != "pair" else (b_raw.strip(), None) - b, quoted = _clean(b) if kind != "pair" else (b, True) - a_loc_text, hint_a = _strip_loc_suffix(a_raw.strip()) - a, a_quoted = _clean(a_loc_text) if kind != "pair" else (a_raw, True) - hint = hint or hint_a - if not b or len(b) > 240: - continue - loc = "logo" if kind == "call" else (_slot_of(a) if not a_quoted else None) - if loc: - patch, val = _apply(cur, loc, None, b) - return f"Done: the {NAMES[loc]} now says “{val}”. Say “reset the look” to undo.", patch - exact, sub = _find(cur, a) - if hint: - # a location was named: replace inside that slot (whole text if it matches, else the matching words) - here = [x for x in exact + sub if x[0] == hint] - if here: - s, i, t = here[0] - frag = None if _norm(t) == _norm(a) else a - patch, val = _apply(cur, s, i, b, frag) - return f"Done: the {NAMES[s]} now says “{val}”. Say “reset the look” to undo.", patch - if kind in ("say",) or not (exact or sub): - patch, val = _apply(cur, hint, None, b) - return (f"Done: the {NAMES[hint]} now says “{val}” (I couldn't find “{a}” there, so I replaced the whole text). " - "Say “reset the look” to undo."), patch - if len(exact) == 1 or (not exact and len(sub) == 1): - s, i, t = (exact or sub)[0] - # whole-text match replaces the slot; a partial match (or a chip label minus its emoji) replaces just those words - frag = None if exact and _norm(t) == _norm(a) else a - patch, val = _apply(cur, s, i, b, frag) - return f"Done: {_label(s, i, t)} now says “{val}”. Say “reset the look” to undo.", patch - if len(exact) + len(sub) > 1: - where = ", ".join(_label(s, i, t) for s, i, t in (exact + sub)[:4]).replace("the ", "") - fallback_understood = fallback_understood or ( - f"I understood: change “{a}” to “{b}”, but that text appears in more than one place ({where}). " - f"Say where, e.g. “change the {NAMES.get((exact + sub)[0][0], 'headline')} to {b}”.") - continue - # A not on the page: only answer (instead of falling through to other commands) for clear copy-edit wording - if fallback_understood is None and (re.match(rf"\s*(?:please\s+)?(?:{STRONG})\b", low) or kind == "pair") \ - and not re.search(rf"\b(?:{STYLE_WORDS})\b", low) and not re.search(r"\b(?:buy|sell|order|cart|checkout|pay)\b", a.lower()): - near = _closest(cur, a) - if near: - s, i, t = near - fallback_understood = (f"I understood: change “{a}” to “{b}”, but I can't find “{a}” on the page. " - f"Closest is {_label(s, i, t)} (“{t[:60]}”). Say “change the {NAMES.get(s, 'headline')} to {b}” " - "if that's what you meant.") - else: - fallback_understood = (f"I understood: change “{a}” to “{b}”, but I can't find “{a}” on the page. I can edit the " - f"logo (“{cur['logo']}”), headline, subtitle, footer, quick-action buttons and bullets, " - f"e.g. “change the logo to {b}”.") - if fallback_understood: - return fallback_understood, None - return None diff --git a/market/theme.py b/market/theme.py index 0c633f8..d99fbbc 100644 --- a/market/theme.py +++ b/market/theme.py @@ -260,12 +260,15 @@ def build_theme_action(payload): """Turn a model's set_theme payload into a client action: {reset?, base?, patch}.""" if not isinstance(payload, dict): return None - if payload.get("reset"): - return {"reset": True, "patch": sanitize_patch(payload.get("patch") or {})} name = str(payload.get("preset") or "").strip().lower() + if payload.get("reset") and not name and not [k for k in payload if k not in ("reset", "patch")]: + return {"reset": True, "patch": sanitize_patch(payload.get("patch") or {})} + # {"preset": "sunset", "reset": true} means "start over from that preset" (a preset base already replaces the look) name = PRESET_ALIASES.get(name, name) patch = payload.get("patch") if isinstance(payload.get("patch"), dict) else {k: v for k, v in payload.items() if k not in ("preset", "reset")} out = {"patch": sanitize_patch(patch)} + if payload.get("reset"): + out["reset"] = True if name in PRESETS: out["preset"] = name out["base"] = sanitize_patch(PRESETS[name]) @@ -281,137 +284,3 @@ def compact_current(theme): t.pop("customCss", None) s = json.dumps(t, separators=(",", ":")) return s if len(s) <= 900 else s[:900] - - -# ---------- rule-based look commands (used when the hosted model is unavailable) ---------- -NAMED = {"red": "#e5484d", "green": "#2fb36d", "blue": "#3d7bfd", "purple": "#8e4ec6", "violet": "#8e4ec6", "pink": "#e93d82", - "orange": "#f76b15", "yellow": "#ffd23f", "teal": "#12a594", "cyan": "#00b8d9", "black": "#0b0b0f", "white": "#ffffff", - "grey": "#8b8d98", "gray": "#8b8d98", "navy": "#1b2a4a", "gold": "#d4a72c", "brown": "#8d5a3b", "beige": "#efe3cf", - "cream": "#fff8e7", "mint": "#9de0c4", "lavender": "#c9b8f0", "coral": "#ff7f6e", "lime": "#9ae600", "magenta": "#d62fd6", - "maroon": "#6d1a2b", "olive": "#7a8c3c", "sky": "#8ecdf7", "peach": "#ffcfa8", "turquoise": "#2ad4c4", "indigo": "#4b4bd6"} -GRADIENTS = {"sunset": ["#ff7e5f", "#feb47b", "#6a3093"], "ocean": ["#0f2027", "#203a43", "#2c5364"], "forest": ["#134e5e", "#71b280"], - "candy": ["#f9a8d4", "#a5b4fc"], "fire": ["#f12711", "#f5af19"], "night": ["#0f0c29", "#302b63", "#24243e"], - "peach": ["#ffecd2", "#fcb69f"], "aurora": ["#00c9a7", "#845ec2", "#0081cf"], "rainbow": ["#ff5f6d", "#ffc371", "#47e891", "#3d7bfd", "#8e4ec6"]} -SECTION_WORDS = {"hero": "hero", "headline": "hero", "title": "hero", "chips": "chips", "quick actions": "chips", "quick buttons": "chips", - "buttons": "chips", "bullets": "bullets", "bullet points": "bullets", "facts": "bullets", "results": "results", - "listings": "results", "footer": "footer"} -_CW = "|".join(sorted(NAMED, key=len, reverse=True)) - - -def parse_look_command(text, current=None): - """Map a plain-English look request to a set_theme payload. Returns (reply, payload) or None.""" - low = (text or "").lower().strip() - cur = current if isinstance(current, dict) else {} - if re.search(r"\breset\b.*\b(look|theme|style|design|page|colou?rs?)\b|\b(default|original) (look|theme)\b|^reset$", low): - return "Done: the page is back to its default look.", {"reset": True} - # UI copy edits first ("change the headline to Retro vibes" is text, not the retro preset) - from .copyedit import parse_copy_edit - ce = parse_copy_edit(text, cur) - if ce: - return ce[0], (sanitize_patch(ce[1]) if ce[1] else None) - for name in list(PRESETS) + list(PRESET_ALIASES): - if re.search(r"\b" + re.escape(name) + r"\b", low) and re.search(r"look|theme|style|make it|switch|preset|vibe|feel", low) \ - and not re.search(r"background|gradient", low): - real = PRESET_ALIASES.get(name, name) - return f"Applied the {real} look. Say “reset the look” to go back.", {"preset": real} - patch, said = {}, [] - if re.search(r"\blight (mode|theme)\b|\bmake it light\b", low): - patch["mode"] = "light"; said.append("light mode") - elif re.search(r"\bdark (mode|theme)\b|\bmake it dark\b", low): - patch["mode"] = "dark"; said.append("dark mode") - if "background" in low or "gradient" in low or re.search(r"\b(dots|polka|stripes|checks|checkered|grid pattern|waves)\b", low): - g = next((k for k in GRADIENTS if k in low), None) - cols = re.findall(r"\b(" + _CW + r")\b", low) - pat = next((p for k, p in (("polka", "dots"), ("dot", "dots"), ("stripe", "stripes"), ("check", "checks"), - ("diagonal", "diagonal"), ("wave", "waves"), ("grid", "grid")) if k in low), None) - if g: - patch["background"] = {"kind": "gradient", "colors": GRADIENTS[g], "angle": 180}; said.append(f"{g} gradient background") - elif pat and "background" in low or pat and "pattern" in low: - c = [NAMED[x] for x in cols[:2]] or ["#1d212a", "#2a2f3a"] - patch["background"] = {"kind": "pattern", "pattern": pat, "colors": c if len(c) > 1 else [c[0], "#ffffff22"]}; said.append(f"{pat} pattern background") - elif "gradient" in low and cols: - c = [NAMED[x] for x in cols[:4]] - patch["background"] = {"kind": "gradient", "colors": c if len(c) > 1 else [c[0], "#000000"], "angle": 135}; said.append("gradient background") - elif cols: - patch["background"] = {"kind": "solid", "colors": [NAMED[cols[0]]]}; said.append(f"{cols[0]} background") - m = re.search(r"\b(" + _CW + r")\s+accent|accent(?: colou?r)?\s+(?:to\s+)?(" + _CW + r")\b", low) - if m: - c = m.group(1) or m.group(2); patch.setdefault("palette", {})["accent"] = NAMED[c]; said.append(f"{c} accent") - m = re.search(r"\b(" + _CW + r")\s+text\b|\btext\s+(?:colou?r\s+)?(?:to\s+)?(" + _CW + r")\b", low) - if m: - c = m.group(1) or m.group(2); patch.setdefault("palette", {})["text"] = NAMED[c]; said.append(f"{c} text") - for name in sorted(FONTS, key=len, reverse=True): - if name != "system" and re.search(r"\b" + re.escape(name.lower()) + r"\b", low) \ - and (re.search(r"\b(font|typeface|type|lettering|letters|text)\b", low) or " " in name): - patch["font"] = name; said.append(f"{name} font"); break - else: - for alias, name in (("monospace", "mono"), ("serif", "serif"), ("handwrit", "Pacifico"), ("script", "Pacifico"), - ("pixel", "Press Start 2P"), ("comic", "Comic Neue"), ("typewriter", "Space Mono"), ("sans", "Inter")): - if alias in low and "font" in low and not ("sans" in low and alias == "serif"): - patch["font"] = name; said.append(f"{name} font"); break - scale = float(cur.get("fontScale") or 1) - if re.search(r"(bigger|larger|increase|bump up).{0,12}(text|font)|(text|font).{0,12}(bigger|larger)", low): - patch["fontScale"] = round(min(1.75, scale + 0.15), 2); said.append(f"text size {patch['fontScale']}×") - elif re.search(r"(smaller|decrease|reduce).{0,12}(text|font)|(text|font).{0,12}smaller", low): - patch["fontScale"] = round(max(0.75, scale - 0.15), 2); said.append(f"text size {patch['fontScale']}×") - if re.search(r"sharp|square corners|no rounding", low): - patch["radius"] = 0; said.append("sharp corners") - elif re.search(r"pill|very rounded|super rounded", low): - patch["radius"] = 30; said.append("pill corners") - elif re.search(r"rounded|round corners|softer corners", low): - patch["radius"] = 22; said.append("rounded corners") - if re.search(r"compact|denser|tighter|less spac", low): - patch["density"] = "compact"; said.append("compact spacing") - elif re.search(r"spacious|airy|more space|roomier|comfy", low): - patch["density"] = "comfy"; said.append("roomier spacing") - for k, v in (("flat", "flat"), ("outlined", "outlined"), ("outline", "outlined"), ("shadow", "elevated"), ("elevated", "elevated"), ("glass", "glass")): - if re.search(r"\b" + k + r"\b", low) and ("card" in low or "style" in low): - patch["cardStyle"] = v; said.append(f"{v} cards"); break - if re.search(r"as a list|list (view|layout)|not cards|as rows", low): - patch["layout"] = "list"; said.append("list layout") - elif re.search(r"grid (view|layout)|as cards|as a grid", low): - patch["layout"] = "grid"; said.append("grid layout") - m = re.search(r"\b([1-6]|one|two|three|four|five|six)\s+columns?\b", low) - if m: - n = {"one": 1, "two": 2, "three": 3, "four": 4, "five": 5, "six": 6}.get(m.group(1)) or int(m.group(1)) - patch["columns"] = n; patch.setdefault("layout", "grid"); said.append(f"{n} columns") - if "chat" in low: - ch = {} - if re.search(r"\bleft\b", low): - ch["position"] = "left" - elif re.search(r"\bright\b", low): - ch["position"] = "right" - if re.search(r"(bigger|larger|wider|taller)", low): - ch.update(width=480, height=680) - elif re.search(r"(smaller|narrower)", low): - ch.update(width=320, height=420) - if ch: - patch["chat"] = ch; said.append("chat " + ", ".join(f"{k} {v}" for k, v in ch.items())) - if re.search(r"(no|stop|turn off|disable|without) animations?|animations? off", low): - patch["animations"] = False; said.append("animations off") - elif re.search(r"(turn on|enable) animations?|animations? on", low): - patch["animations"] = True; said.append("animations on") - hide = re.search(r"\b(hide|remove|get rid of)\b(.+)", low) - show = re.search(r"\b(show|unhide|bring back)\b(.+)", low) - hidden = list((cur.get("sections") or {}).get("hidden") or []) - for verb, mm in (("hide", hide), ("show", show)): - if not mm: - continue - for w, sec in SECTION_WORDS.items(): - if re.search(r"\b" + w + r"\b", mm.group(2)): - if verb == "hide" and sec not in hidden: - hidden.append(sec); said.append(f"{sec} hidden"); patch["sections"] = {"hidden": hidden} - elif verb == "show" and sec in hidden: - hidden.remove(sec); said.append(f"{sec} shown"); patch["sections"] = {"hidden": hidden} - break - m = re.search(r"move (?:the )?(\w+(?: \w+)?) to the (top|bottom)", low) - if m and "chat" not in m.group(1): - sec = next((s for w, s in SECTION_WORDS.items() if w in m.group(1)), None) - if sec: - order = [x for x in ((cur.get("sections") or {}).get("order") or SECTIONS) if x != sec] - order = [sec] + order if m.group(2) == "top" else order + [sec] - patch.setdefault("sections", {})["order"] = order; said.append(f"{sec} moved to the {m.group(2)}") - patch = sanitize_patch(patch) - if not patch: - return None - return "Done: " + ", ".join(said) + ".", patch diff --git a/ops/local-llm-watchdog.sh b/ops/local-llm-watchdog.sh new file mode 100755 index 0000000..e543738 --- /dev/null +++ b/ops/local-llm-watchdog.sh @@ -0,0 +1,64 @@ +#!/usr/bin/env bash +# Local LLM for Agent Market (the last link of the model chain): keeps llama.cpp's llama-server running on loopback only +# and restarts it if it exits or stops answering. Lean on purpose so the box keeps headroom for everything else: +# text-only (no vision projector), weights mmapped and not repacked (no second copy in RAM), 6k context, 1 slot, +# q8 KV cache, 6 threads at nice 10, 1 GiB prompt-cache RAM. +# start / ensure running (idempotent): ops/local-llm-watchdog.sh --ensure +# foreground: ops/local-llm-watchdog.sh +# stop: kill $(cat ~/llm/watchdog.pid) (the trap stops llama-server too) +# Settings can be overridden in ~/.config/agent-market/local-llm-server.env (outside the repo). +set -u +DIR=${LLM_DIR:-$HOME/llm} +mkdir -p "$DIR" +[ -f "$HOME/.config/agent-market/local-llm-server.env" ] && . "$HOME/.config/agent-market/local-llm-server.env" +LLAMA_SERVER=${LLAMA_SERVER:-$HOME/ollama/lib/ollama/llama-server} # any llama.cpp llama-server build works +LLAMA_LIBS=${LLAMA_LIBS:-$(dirname "$LLAMA_SERVER")} +MODEL_GGUF=${MODEL_GGUF:-$DIR/gemma3-4b-it-Q4_K_M.gguf} # gemma3 4B instruct, Q4_K_M, text weights only +MODEL_ALIAS=${MODEL_ALIAS:-gemma3:4b} +PORT=${LLM_PORT:-11434} +CTX=${LLM_CTX:-6144} +THREADS=${LLM_THREADS:-6} +NICE=${LLM_NICE:-10} +CACHE_RAM_MIB=${LLM_CACHE_RAM_MIB:-1024} +LOG=$DIR/llama-server.log + +[ -x "$LLAMA_SERVER" ] && [ -f "$MODEL_GGUF" ] || { echo "missing $LLAMA_SERVER or $MODEL_GGUF (see README: Local model)"; exit 1; } + +if [ "${1:-}" = "--ensure" ]; then + if [ -f "$DIR/watchdog.pid" ] && kill -0 "$(cat "$DIR/watchdog.pid")" 2>/dev/null; then echo "local LLM watchdog running (pid $(cat "$DIR/watchdog.pid"))"; exit 0; fi + # run a private copy: a later deploy rewriting the repo file must not change a script bash is still reading + install -m 755 "$0" "$DIR/watchdog.sh" + nohup setsid "$DIR/watchdog.sh" >> "$DIR/watchdog.log" 2>&1 < /dev/null & + echo "local LLM watchdog started (pid $!)"; exit 0 +fi + +exec 9> "$DIR/watchdog.lock" +flock -n 9 || { echo "another watchdog holds $DIR/watchdog.lock"; exit 0; } +echo $$ > "$DIR/watchdog.pid" +child=0 +trap '[ $child -gt 0 ] && kill $child 2>/dev/null; exit 0' TERM INT +while true; do + # wait for a previous instance to release the port + for i in $(seq 1 15); do ss -ltn 2>/dev/null | grep -q "127.0.0.1:$PORT " || break; sleep 1; done + [ -f "$LOG" ] && [ "$(stat -c %s "$LOG")" -gt 20000000 ] && mv -f "$LOG" "$LOG.1" + LD_LIBRARY_PATH="$LLAMA_LIBS${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" nice -n "$NICE" "$LLAMA_SERVER" \ + -m "$MODEL_GGUF" --alias "$MODEL_ALIAS" --host 127.0.0.1 --port "$PORT" \ + -c "$CTX" -np 1 -t "$THREADS" -tb "$THREADS" -b 512 -ub 256 \ + -ctk q8_0 -ctv q8_0 -fa on --swa-full --no-mmproj --no-repack --cache-ram "$CACHE_RAM_MIB" \ + --poll 0 --threads-http 2 --no-webui --jinja --metrics >> "$LOG" 2>&1 & + child=$!; started=$(date +%s) + echo "$(date '+%F %T') started llama-server pid $child ($MODEL_ALIAS, ctx $CTX, $THREADS threads, nice $NICE)" + for i in $(seq 1 120); do curl -sf -m 2 "http://127.0.0.1:$PORT/health" >/dev/null && break; kill -0 $child 2>/dev/null || break; sleep 1; done + fails=0 + while kill -0 $child 2>/dev/null; do + sleep 15 + # /health is served by its own HTTP threads, so it answers while a reply is being generated + if curl -sf -m 10 "http://127.0.0.1:$PORT/health" >/dev/null; then fails=0; else fails=$((fails+1)); fi + if [ $fails -ge 3 ]; then echo "$(date '+%F %T') llama-server unresponsive, restarting"; kill $child; sleep 5; kill -9 $child 2>/dev/null; fi + [ -f "$LOG" ] && [ "$(stat -c %s "$LOG")" -gt 50000000 ] && : > "$LOG" + done + wait $child 2>/dev/null + st=$?; pause=3; [ $(( $(date +%s) - started )) -lt 60 ] && pause=30 # crashing right away: back off + echo "$(date '+%F %T') llama-server exited (status $st); restarting in ${pause}s" + sleep $pause +done diff --git a/server.py b/server.py index 00930c4..9585aa8 100755 --- a/server.py +++ b/server.py @@ -147,6 +147,14 @@ def hydrate(actions, actor): continue deal = r["deal"] p.update(deal=deal, listing=store.get("listings", deal["listing_id"]), deal_id=deal["id"]) + elif t == "show_listing_form": + # small models sometimes send the defaults flat ({"title":..., "price":...}); accept both shapes + if not isinstance(p.get("defaults"), dict): + p = {"defaults": {k: v for k, v in p.items() if k in ("title", "kind", "price", "price_cents", "description", "category", "area", "payment_options")}} + d = p["defaults"] + if d.get("price") is None and isinstance(d.get("price_cents"), (int, float)): + d["price"] = int(d["price_cents"]) + d.pop("price_cents", None) elif t == "set_theme": act = TH.build_theme_action(p) if not act: @@ -384,5 +392,6 @@ class H(BaseHTTPRequestHandler): if __name__ == "__main__": srv = ThreadingHTTPServer((HOST, PORT), H) st = A.agent_status() + A.start_local_primer(store) print(f"Agent Market [{ENV_LABEL}] on http://{HOST}:{PORT} agent={st['mode']} fee={A.FEE_BPS/100:.2f}% payments=TEST(mock)", flush=True) srv.serve_forever() diff --git a/tests/browser_e2e.js b/tests/browser_e2e.js index ef5e48f..6ec6b78 100644 --- a/tests/browser_e2e.js +++ b/tests/browser_e2e.js @@ -17,14 +17,14 @@ const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); check("landing renders agent-chosen cards", (await page.$$(".card")).length >= 4); check("React app mounted in #root", await page.evaluate(() => !!document.querySelector("#root > .testbar") && !!document.querySelector("#root main#page #stage [data-component=cards]"))); const badge = await page.$eval("#agentBadge", (n) => n.textContent); - check("agent mode badge shown", /fallback|built-in|AI:|LLM/.test(badge), badge); + check("agent mode badge shown", /^AI: |^Local model \(/.test(badge), badge); // ask(): type into the chat popup and wait for the agent's reply (LLM replies take a few seconds) const ask = async (text) => { const n = (await page.$$(".msg.bot")).length; await page.type("#chatInput", text); await page.keyboard.press("Enter"); - await page.waitForFunction((n) => document.querySelectorAll(".msg.bot").length > n, { timeout: 60000 }, n); + await page.waitForFunction((n) => document.querySelectorAll(".msg.bot").length > n, { timeout: 90000 }, n); await sleep(300); }; // chat search @@ -37,11 +37,13 @@ const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); const feeBps = (await (await fetch(BASE + "/api/status")).json()).fee_bps; const money = (c) => "$" + (c / 100).toFixed(2); - // online checkout: first card that offers online payment - const pick = await page.evaluate(() => { + // online checkout: first card that offers online payment (earlier runs may have sold the online-payable bikes) + const findOnline = () => page.evaluate(() => { const c = [...document.querySelectorAll("#stage .card")].find((c) => c.querySelector("button.btn:not(.cash)")); return c && { id: c.dataset.listing, price: c.querySelector(".price").childNodes[0].textContent }; }); + let pick = await findOnline(); + if (!pick) { await ask("show me the bike tune-up service"); pick = await findOnline(); } check("an online-payable listing is visible", !!pick, pick && pick.price); const priceC = Math.round(parseFloat(pick.price.slice(1)) * 100), feeC = Math.round(priceC * feeBps / 10000); await page.click(`.card[data-listing="${pick.id}"] button.btn:not(.cash)`); diff --git a/tests/e2e_http.py b/tests/e2e_http.py index 37da037..4f08739 100755 --- a/tests/e2e_http.py +++ b/tests/e2e_http.py @@ -1,6 +1,8 @@ #!/usr/bin/env python3 -"""End-to-end HTTP checks against a running Agent Market. Usage: e2e_http.py http://127.0.0.1:8795""" -import json, sys, urllib.request, urllib.error +"""End-to-end HTTP checks against a running Agent Market. Usage: e2e_http.py http://127.0.0.1:8795 +Typed messages always go to a model (hosted free models, then the local model). With REQUIRE_LLM=1 they must be answered +by one; otherwise an honest busy reply is accepted for them. Clicks, forms and payments are deterministic and always checked.""" +import json, os, sys, urllib.request, urllib.error BASE = (sys.argv[1] if len(sys.argv) > 1 else "http://127.0.0.1:8795").rstrip("/") results = [] @@ -11,7 +13,7 @@ def call(method, path, body=None, headers=None, raw=False): h = {"Content-Type": "application/json", **(headers or {})} req = urllib.request.Request(BASE + path, data=data, method=method, headers=h) try: - with urllib.request.urlopen(req, timeout=60) as r: + with urllib.request.urlopen(req, timeout=90) as r: b = r.read().decode() return r.status, (b if raw else json.loads(b)) except urllib.error.HTTPError as e: @@ -38,21 +40,37 @@ s, st = call("GET", "/api/status") check("status endpoint", s == 200 and st["payments"]["mode"] == "test", f"agent={st['agent']['mode']} fee_bps={st['fee_bps']}") fee_bps = st["fee_bps"] -s, out = call("POST", "/api/chat", {"message": "hi"}) -check("chat agent returns UI actions", s == 200 and out["actions"] and out["actions"][0]["type"] in ("show_hero", "show_cards"), out.get("agent")) +REQUIRE_LLM = os.environ.get("REQUIRE_LLM") == "1" +BUSY = "The assistant is busy, please try again in a moment." + +def typed(name, out, ok_fn, detail=""): + """A typed message: answered by a model (then ok_fn must hold) or, unless REQUIRE_LLM, the honest busy reply.""" + if out.get("agent") == "llm": + check(name + f" (model {out.get('model')}{', local' if out.get('local') else ''})", ok_fn(out), detail) + else: + check(name + " -> busy reply (no model available)", not REQUIRE_LLM and out.get("agent") == "unavailable" and + out.get("reply") == BUSY and out.get("actions") == [] and out.get("retry") is True, f"agent={out.get('agent')}") + + +s, out = call("POST", "/api/chat", {"event": {"type": "landing"}}) +check("landing view (deterministic featured listings)", s == 200 and out["agent"] == "ui-event" and out["actions"][0]["type"] == "show_cards" + and len(out["actions"][0]["payload"]["listings"]) >= 3) s, out = call("POST", "/api/chat", {"message": "show me bikes"}) cards = [a for a in out["actions"] if a["type"] == "show_cards"] titles = [l["title"] for l in cards[0]["payload"]["listings"]] if cards else [] -check("agent search renders cards (hydrated listings)", cards and any("bike" in t.lower() for t in titles), ", ".join(titles)) -bike = next((l for l in cards[0]["payload"]["listings"] if l["kind"] == "product" and "online" in l["payment_options"]), None) if cards else None +typed("typed search renders hydrated cards", out, lambda o: cards and any("bike" in t.lower() for t in titles), ", ".join(titles)) +s, r = call("POST", "/api/actions", {"action": "search", "args": {"q": "bike"}}, {"X-Agent-Id": "test-buyer-e2e"}) +bike = next((l for l in r.get("listings", []) if l["kind"] == "product" and "online" in l["payment_options"] and l["status"] == "active"), None) if not bike: # earlier runs may have sold the seed bike; list a fresh test-only one s, r = call("POST", "/api/actions", {"action": "create_listing", "args": {"title": "Test-only commuter bike", "kind": "product", "price": 18000, "description": "Fake bike for automated tests", "payment_options": ["online", "cash"]}}, {"X-Agent-Id": "test-seller-e2e"}) bike = r["listing"] -s, out = call("POST", "/api/chat", {"message": "I want to sell something"}) -check("agent opens listing form", any(a["type"] == "show_listing_form" for a in out["actions"])) +s, out = call("POST", "/api/chat", {"message": "I want to sell my old desk lamp for $20, cash only"}) +typed("typed sell request opens a prefilled listing form", out, lambda o: any(a["type"] == "show_listing_form" for a in o["actions"])) +s, out = call("POST", "/api/chat", {"event": {"type": "sell"}}) +check("Sell button opens the listing form (deterministic)", out["agent"] == "ui-event" and out["actions"][0]["type"] == "show_listing_form") s, out = call("POST", "/api/chat", {"event": {"type": "submit_listing", "fields": {"title": "Test-only desk lamp", "kind": "product", "price": "20", "description": "Fake test listing", "payment_options": ["online", "cash"]}}, "actor": "guest-e2e"}) new = [a for a in out["actions"] if a["type"] == "show_cards"] newl = new[0]["payload"]["listings"][0] if new else {} @@ -81,8 +99,8 @@ s, r = call("POST", "/api/payments/webhook", {"type": "payment_intent.succeeded" check("forged webhook rejected", s == 401) # Cash in person: zero fee. -s, out = call("POST", "/api/chat", {"message": "show lamp"}) -lamps = [l for a in out["actions"] if a["type"] == "show_cards" for l in a["payload"]["listings"] if "cash" in l["payment_options"] and l["kind"] == "product"] +s, r = call("POST", "/api/actions", {"action": "search", "args": {"q": "lamp"}}, {"X-Agent-Id": "test-buyer-e2e"}) +lamps = [l for l in r.get("listings", []) if "cash" in l["payment_options"] and l["kind"] == "product" and l["status"] == "active"] if not lamps: s, r = call("POST", "/api/actions", {"action": "create_listing", "args": {"title": "Test-only lamp", "kind": "product", "price": 3000, "description": "Fake lamp for automated tests", "payment_options": ["online", "cash"]}}, {"X-Agent-Id": "test-seller-e2e"}) diff --git a/tests/fake_model_browser.js b/tests/fake_model_browser.js index 0f6fb17..b453f19 100644 --- a/tests/fake_model_browser.js +++ b/tests/fake_model_browser.js @@ -1,8 +1,8 @@ -// React UI against the fake OpenAI-compatible model (tests/mock_llm.py): model badge + chat mode text, fallback badge -// texts, quota badge, rate-limit toast, and a model-sent set_theme that the server sanitised. No real model calls. -// Usage: NODE_PATH= node tests/fake_model_browser.js +// React UI against the fake OpenAI-compatible model (tests/mock_llm.py): model badge + chat mode text, busy reply with +// retry button (never rule-based answers), quota badge, rate-limit toast, and a model-sent set_theme that the server sanitised. No real model calls. +// Usage: NODE_PATH= node tests/fake_model_browser.js const puppeteer = require("puppeteer-core"); -const [OK, BUSY, DAILY, RL] = process.argv.slice(2, 6); +const [OK, BUSY, DAILY, RL, LOCAL, MOCK] = process.argv.slice(2, 8); const results = []; const check = (name, ok, d = "") => { results.push(!!ok); console.log((ok ? "PASS " : "FAIL ") + name + (d ? ` [${d}]` : "")); }; const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); @@ -43,22 +43,45 @@ const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); check("model set_theme applied (gradient, toast, saved)", /gradient/.test(st.bg) && /Look updated/.test(st.toast) && st.ls.background, st.bg.slice(0, 60)); check("model copy/CSS sanitised in the page", !st.img && /Sunset/.test(st.h) && !/ built-in agent answers, fallback badge text + // 3) every model rate-limited -> honest busy message with a retry button (no rule-based answer, no cards) p = await open(BUSY); - await ask(p, "show me bikes"); - check("fallback badge + chat mode text", (await text(p, "#agentBadge")) === "agent: built-in (AI unavailable)" && (await text(p, "#chatMode")) === "built-in agent answered (AI model unavailable)" && (await p.$$("#stage .card")).length > 0, + const cards0 = await p.$$eval("#stage .card h3", (ns) => ns.map((n) => n.textContent).join("|")); + const busy = await ask(p, "show me bikes"); + check("busy reply + retry button, nothing rendered by rules", + /^The assistant is busy, please try again in a moment\./.test(busy) && (await p.$$(".msg.bot.busy .retryBtn")).length === 1 && + (await p.$$eval("#stage .card h3", (ns) => ns.map((n) => n.textContent).join("|"))) === cards0, + `${busy} | ${await text(p, "#agentBadge")} | ${await text(p, "#chatMode")}`); + check("busy badge + chat mode text", (await text(p, "#agentBadge")) === "AI: busy" && /^Assistant busy/.test(await text(p, "#chatMode")), `${await text(p, "#agentBadge")} | ${await text(p, "#chatMode")}`); - // 4) daily quota used up -> after reload the badge says so (from /api/status) + const calls = async () => { const c = (await (await fetch(MOCK)).json()).calls; return (c["busy-model"] || 0) + (c["busy-model-2"] || 0); }; + const c0 = await calls(); + const users0 = (await p.$$(".msg.user")).length; + await p.click(".msg.bot.busy .retryBtn"); + await p.waitForFunction(() => !document.querySelector(".msg.sys") && document.querySelectorAll(".msg.bot.busy").length === 1, { timeout: 30000 }); + await sleep(300); + check("Retry resends the same message (one busy message left, no duplicate user bubble)", + (await calls()) === c0 + 1 && (await p.$$(".msg.user")).length === users0 && (await p.$$(".msg.bot.busy")).length === 1, `calls ${c0} -> ${await calls()}`); + // 4) daily quota used up, no local model -> busy reply; after reload the badge says the quota is used up p = await open(DAILY); - await ask(p, "show me bikes"); + check("quota used up -> busy reply", /^The assistant is busy/.test(await ask(p, "show me bikes"))); await p.reload({ waitUntil: "networkidle0" }); await sleep(300); - check("quota badge after reload", (await text(p, "#agentBadge")) === "agent: built-in (AI quota used up)" && /resets/.test(await p.$eval("#agentBadge", (n) => n.title)), await text(p, "#agentBadge")); + check("quota badge after reload", (await text(p, "#agentBadge")) === "AI: quota used up" && /resets/.test(await p.$eval("#agentBadge", (n) => n.title)), await text(p, "#agentBadge")); // 5) per-IP rate limit -> warning toast p = await open(RL); await ask(p, "show me bikes"); await ask(p, "show me bikes"); const toasts = await p.$$eval("#toasts .toast.warn", (ns) => ns.map((n) => n.textContent)); check("rate-limit warning toast", toasts.some((t) => /quickly/.test(t)), toasts.join("|")); + // 6) hosted quota used up -> the local model answers; badge says so honestly + if (LOCAL) { + p = await open(LOCAL); + await ask(p, "show me bikes"); + const lb = await text(p, "#agentBadge"), lm = await text(p, "#chatMode"); + check("local model badge + chat mode", lb === "Local model (local-model)" && /^Local model · local-model · \d+\.\ds$/.test(lm) && (await p.$$("#stage .card")).length > 0, `${lb} | ${lm}`); + await p.reload({ waitUntil: "networkidle0" }); await sleep(300); + check("after reload (quota used up) the badge shows the local model", (await text(p, "#agentBadge")) === "Local model (local-model)", await text(p, "#agentBadge")); + } + check("no page JS errors", errors.length === 0, errors.join("; ")); await browser.close(); const passed = results.filter(Boolean).length; diff --git a/tests/fake_model_browser.sh b/tests/fake_model_browser.sh index c07543c..e5b20eb 100755 --- a/tests/fake_model_browser.sh +++ b/tests/fake_model_browser.sh @@ -10,6 +10,7 @@ start() { # port models rpm burst AGENT_MARKET_PORT=$1 AGENT_MARKET_DATA=$D/d$1 AGENT_MARKET_LLM_API_KEY_FILE=$KEYF AGENT_MARKET_CHAT_RPM=$3 AGENT_MARKET_CHAT_BURST=$4 \ AGENT_MARKET_LLM_BASE_URL=http://127.0.0.1:18803/v1 AGENT_MARKET_LLM_MODELS=$2 python3 server.py >/dev/null 2>&1 & echo $!; } S1=$(start 18804 mock-model 0 6); S2=$(start 18805 busy-model,busy-model-2 0 6); S3=$(start 18806 daily-model 0 6); S4=$(start 18807 mock-model 1 1) -trap 'kill $M $S1 $S2 $S3 $S4 2>/dev/null; rm -rf $D' EXIT +S5=$(AGENT_MARKET_LOCAL_LLM_URL=http://127.0.0.1:18803/v1 AGENT_MARKET_LOCAL_LLM_MODEL=local-model start 18811 daily-model 0 6) +trap 'kill $M $S1 $S2 $S3 $S4 $S5 2>/dev/null; rm -rf $D' EXIT sleep 1.5 -node tests/fake_model_browser.js http://127.0.0.1:18804 http://127.0.0.1:18805 http://127.0.0.1:18806 http://127.0.0.1:18807 +node tests/fake_model_browser.js http://127.0.0.1:18804 http://127.0.0.1:18805 http://127.0.0.1:18806 http://127.0.0.1:18807 http://127.0.0.1:18811 http://127.0.0.1:18803/ diff --git a/tests/llm_proxy_check.sh b/tests/llm_proxy_check.sh index a6eb5eb..ffcf312 100755 --- a/tests/llm_proxy_check.sh +++ b/tests/llm_proxy_check.sh @@ -1,6 +1,6 @@ #!/usr/bin/env bash # LLM proxy path against a local fake OpenAI-compatible server (no real key needed): -# tool call -> hydrated cards, key stays server-side, 429 fallback chain, rule-based last resort, per-IP rate limit. +# tool call -> hydrated cards, key stays server-side, 429 fallback chain, local model last, busy reply (never rules), per-IP rate limit. set -euo pipefail cd "$(dirname "$0")/.." D=$(mktemp -d); KEYF=$D/key; printf 'test-dummy-key' > $KEYF; chmod 600 $KEYF @@ -9,7 +9,16 @@ start() { # port models rpm AGENT_MARKET_PORT=$1 AGENT_MARKET_DATA=$D/d$1 AGENT_MARKET_LLM_API_KEY_FILE=$KEYF AGENT_MARKET_CHAT_RPM=$3 AGENT_MARKET_CHAT_BURST=3 \ AGENT_MARKET_LLM_BASE_URL=http://127.0.0.1:18798/v1 AGENT_MARKET_LLM_MODELS=$2 python3 server.py >/dev/null 2>&1 & echo $!; } S1=$(start 18799 busy-model,mock-model 0); S2=$(start 18800 busy-model,busy-model-2 0); S3=$(start 18801 mock-model 2); S4=$(start 18802 daily-model,mock-model 0) -trap 'kill $M $S1 $S2 $S3 $S4 2>/dev/null; rm -rf $D' EXIT +# local model (same fake server standing in for llama-server's OpenAI-compatible endpoint) +local_start() { # port hosted-models local-model timeout [nokey] + local K=$KEYF; [ -n "${5:-}" ] && K=$D/no-such-key + AGENT_MARKET_PORT=$1 AGENT_MARKET_DATA=$D/d$1 AGENT_MARKET_LLM_API_KEY_FILE=$K AGENT_MARKET_CHAT_RPM=0 \ + AGENT_MARKET_LLM_BASE_URL=http://127.0.0.1:18798/v1 AGENT_MARKET_LLM_MODELS=$2 AGENT_MARKET_LOCAL_LLM_URL=http://127.0.0.1:18798/v1 \ + AGENT_MARKET_LOCAL_LLM_MODEL=$3 AGENT_MARKET_LOCAL_LLM_TIMEOUT=$4 AGENT_MARKET_LOCAL_LLM_MIN=$4 python3 server.py >/dev/null 2>&1 & echo $!; } +S5=$(local_start 18808 daily-model local-model 30); S6=$(local_start 18809 busy-model slow-local 1); S7=$(local_start 18810 x local-model 30 nokey) +# no model at all (no key, no local model) +AGENT_MARKET_PORT=18811 AGENT_MARKET_DATA=$D/d18811 AGENT_MARKET_LLM_API_KEY_FILE=$D/no-such-key AGENT_MARKET_CHAT_RPM=0 python3 server.py >/dev/null 2>&1 & S8=$! +trap 'kill $M $S1 $S2 $S3 $S4 $S5 $S6 $S7 $S8 2>/dev/null; rm -rf $D' EXIT sleep 1.5 chat() { curl -s localhost:$1/api/chat -H 'Content-Type: application/json' -d '{"message":"show me bikes"}'; } OUT=$(chat 18799) @@ -23,10 +32,20 @@ assert all("bike" in l["title"].lower() for l in d["actions"][0]["payload"]["lis print("PASS 429 on first model -> second model answered; cards hydrated; disallowed component dropped")' if echo "$OUT" | grep -q test-dummy-key; then echo "FAIL key leaked to browser"; exit 1; else echo "PASS key never sent to browser"; fi test "$(curl -s localhost:18798/ | python3 -c 'import json,sys;print(json.load(sys.stdin)["auth_ok"])')" -ge 2 && echo "PASS key read from file and sent upstream only" -chat 18800 | python3 -c ' -import json,sys; d=json.load(sys.stdin) -assert d["agent"]=="fallback" and d["llm_error"]=="all_models_failed" and d["actions"], d -print("PASS all models rate-limited -> rule-based agent still renders", [a["type"] for a in d["actions"]])' +BUSY='import json,sys; d=json.load(sys.stdin) +assert d["agent"]=="unavailable" and d["reply"]=="The assistant is busy, please try again in a moment." and d["actions"]==[] and d["retry"] is True, d' +chat 18800 | python3 -c "$BUSY"' +assert d["llm_error"]=="all_models_failed" and [f["error"] for f in d["fallbacks"]]==["rate_limited","rate_limited"], d +print("PASS all models rate-limited -> honest busy reply with retry, no rule-based answer")' +curl -s localhost:18800/api/chat -H 'Content-Type: application/json' -d '{"message":"replace agent market to new internet on the top left corner","theme":{}}' | python3 -c "$BUSY"' +print("PASS look/text commands are never answered by rules either (busy reply)")' +curl -s localhost:18800/api/chat -H 'Content-Type: application/json' -d '{"event":{"type":"sell"}}' | python3 -c ' +import json,sys; d=json.load(sys.stdin); assert d["agent"]=="ui-event" and d["actions"][0]["type"]=="show_listing_form", d +print("PASS clicks stay deterministic while the models are busy (Sell -> listing form)")' +curl -s localhost:18800/api/chat -H 'Content-Type: application/json' -d '{"event":{"type":"landing"}}' | python3 -c ' +import json,sys; d=json.load(sys.stdin); a=d["actions"] +assert d["agent"]=="ui-event" and a[0]["type"]=="show_cards" and len(a[0]["payload"]["listings"])>=3, d +print("PASS landing view is deterministic (featured listings)")' curl -s localhost:18800/api/status | python3 -c ' import json,sys; a=json.load(sys.stdin)["agent"]; assert set(a["cooling_down"])=={"busy-model","busy-model-2"}, a; print("PASS 429ed models put on cooldown")' CODES=$(for i in 1 2 3 4 5; do curl -s -o /dev/null -w '%{http_code} ' localhost:18801/api/chat -H 'Content-Type: application/json' -H 'CF-Connecting-IP: 203.0.113.9' -d '{"message":"show me bikes"}'; done) @@ -45,9 +64,53 @@ import json,sys; d=json.load(sys.stdin); a=d["actions"] assert d["agent"]=="llm" and a and a[0]["type"]=="set_theme" and a[0]["payload"]["patch"]["copy"]=={"logo":"New Internet"}, d print("PASS model path: prompt explains top-left = logo + current page text -> copy.logo set")' curl -s localhost:18802/api/chat -H 'Content-Type: application/json' -d '{"message":"show me bikes"}' >/dev/null -curl -s localhost:18802/api/chat -H 'Content-Type: application/json' -d '{"message":"switch to light mode with a green accent"}' | python3 -c ' -import json,sys; d=json.load(sys.stdin) -assert d["agent"]=="fallback" and d["actions"][0]["type"]=="set_theme" and d["actions"][0]["payload"]["patch"]["mode"]=="light", d -print("PASS daily quota -> whole chain parked, built-in agent still restyles:", d["reply"][:80])' +curl -s localhost:18802/api/chat -H 'Content-Type: application/json' -d '{"message":"switch to light mode with a green accent"}' | python3 -c "$BUSY"' +assert d["fallbacks"]==[{"model":"*","error":"daily_quota","ms":0}], d["fallbacks"] +print("PASS daily quota, no local model -> whole chain parked, busy reply (no rules)")' curl -s localhost:18802/api/status | python3 -c ' import json,sys; a=json.load(sys.stdin)["agent"]; assert a["daily_quota_exhausted_until_utc"], a; print("PASS status reports quota reset", a["daily_quota_exhausted_until_utc"])' +calls() { curl -s localhost:18798/ | python3 -c "import json,sys;print(json.load(sys.stdin)['calls'].get('$1',0))"; } +auth() { curl -s localhost:18798/ | python3 -c "import json,sys;print(json.load(sys.stdin)['auth'].get('$1',0))"; } +C0=$(calls daily-model); A0=$(auth daily-model) +chat 18808 | python3 -c ' +import json,sys; d=json.load(sys.stdin) +assert d["agent"]=="llm" and d.get("local") and d["provider"]=="local" and d["model"]=="local-model", d +assert d["fallbacks"][0]["model"]=="daily-model" and d["fallbacks"][0]["error"]=="daily_quota", d["fallbacks"] +print("PASS hosted daily quota -> local model answered (hosted -> local)")' +[ "$(auth daily-model)" = "$((A0+1))" ] && echo "PASS hosted calls still carry the key when a local model is configured" || { echo "FAIL hosted call without key"; exit 1; } +curl -s localhost:18798/ | python3 -c ' +import json,sys; s=json.load(sys.stdin) +assert s["grammar"].get("local-model",0)>=1 and s["auth"].get("local-model",0)==0 and s["local_prompt"].get("local-model",0)>=1, s +assert s["grammar"].get("mock-model",0)==0 and s["local_prompt"].get("mock-model",0)==0, s +print("PASS local requests: JSON grammar, examples + visitor-message prompt, never the hosted key")' +chat 18808 | python3 -c ' +import json,sys; d=json.load(sys.stdin); assert d.get("local") and d["fallbacks"]==[{"model":"*","error":"daily_quota","ms":0}], d' +[ "$(calls daily-model)" = "$((C0+1))" ] && echo "PASS quota parked: next turn goes straight to the local model (no hosted call)" || { echo "FAIL hosted called again: $(calls daily-model)"; exit 1; } +curl -s localhost:18808/api/status | python3 -c ' +import json,sys; a=json.load(sys.stdin)["agent"] +assert a["local_model"]=="local-model" and a["local_ok"] is True and a["chain"]==["daily-model","local:local-model"] and a["last_local"] is True and a["daily_quota_exhausted_until_utc"], a +print("PASS status: chain hosted -> local, local server up, last answer local")' +kill $S5; sleep 0.5; S5=$(local_start 18808 daily-model local-model 30); sleep 1.2 +curl -s localhost:18808/api/status | python3 -c ' +import json,sys; a=json.load(sys.stdin)["agent"]; assert a["daily_quota_exhausted_until_utc"], a' +chat 18808 >/dev/null +[ "$(calls daily-model)" = "$((C0+1))" ] && echo "PASS quota stop survives a restart (no call spent to rediscover it)" || { echo "FAIL hosted called after restart"; exit 1; } +chat 18809 | python3 -c "$BUSY"' +assert [(f["model"], f["error"]) for f in d["fallbacks"]]==[("busy-model","rate_limited"),("slow-local","timeout")], d["fallbacks"] +print("PASS hosted 429 + local model over its own timeout -> busy reply with retry (no rules)")' +chat 18810 | python3 -c ' +import json,sys; d=json.load(sys.stdin); assert d["agent"]=="llm" and d.get("local") and d["fallbacks"]==[], d +print("PASS local model works with no hosted key configured")' +curl -s localhost:18810/api/chat -H 'Content-Type: application/json' -d '{"message":"show me bikes"}' | grep -q test-dummy-key && { echo "FAIL key"; exit 1; } || true +curl -s localhost:18810/api/chat -H 'Content-Type: application/json' -d '{"message":"replace agent market to new internet on the top left corner","theme":{}}' | python3 -c ' +import json,sys; d=json.load(sys.stdin); a=d["actions"] +assert d["agent"]=="llm" and d.get("local") and "rules_checked" not in d and a[0]["payload"]["patch"]["copy"]=={"logo":"New Internet"}, d +print("PASS local model edits page text itself (exact logo phrase; no rules involved)")' +chat 18811 | python3 -c "$BUSY"' +assert d["llm_error"]=="no_model_configured", d +print("PASS no model configured -> busy reply for typed messages")' +curl -s localhost:18811/api/chat -H 'Content-Type: application/json' -d '{"event":{"type":"landing"}}' | python3 -c ' +import json,sys; d=json.load(sys.stdin); assert d["agent"]=="ui-event" and d["actions"][0]["type"]=="show_cards", d' +curl -s localhost:18811/api/status | python3 -c ' +import json,sys; a=json.load(sys.stdin)["agent"]; assert a["mode"]=="unavailable" and a["chain"]==[], a +print("PASS no model configured: landing still works, status says unavailable")' diff --git a/tests/mock_llm.py b/tests/mock_llm.py index 2f00155..0c0aa04 100755 --- a/tests/mock_llm.py +++ b/tests/mock_llm.py @@ -1,10 +1,10 @@ #!/usr/bin/env python3 """Fake OpenAI-compatible endpoint to exercise the LLM proxy path without a real key. Round 1 asks for a search tool; round 2 returns show_cards with ids from the tool results.""" -import json, re, sys +import json, re, sys, time from http.server import ThreadingHTTPServer, BaseHTTPRequestHandler PORT = int(sys.argv[1]) if len(sys.argv) > 1 else 8798 -SEEN = {"auth_ok": 0} +SEEN = {"auth_ok": 0, "calls": {}, "auth": {}, "grammar": {}, "local_prompt": {}} class H(BaseHTTPRequestHandler): def log_message(self, *a): pass @@ -12,12 +12,21 @@ class H(BaseHTTPRequestHandler): b = json.dumps(SEEN).encode(); self.send_response(200); self.end_headers(); self.wfile.write(b) def do_POST(self): body = json.loads(self.rfile.read(int(self.headers["Content-Length"]))) + SEEN["calls"][body.get("model")] = SEEN["calls"].get(body.get("model"), 0) + 1 + if body.get("model") == "slow-local": + time.sleep(3) if body.get("model") in ("busy-model", "busy-model-2"): b = b'{"error":{"message":"rate-limited upstream","code":429}}' self.send_response(429); self.send_header("Content-Type", "application/json"); self.end_headers(); self.wfile.write(b) return + mdl = body.get("model") if self.headers.get("Authorization") == "Bearer test-dummy-key": SEEN["auth_ok"] += 1 + SEEN["auth"][mdl] = SEEN["auth"].get(mdl, 0) + 1 + if body.get("grammar"): + SEEN["grammar"][mdl] = SEEN["grammar"].get(mdl, 0) + 1 + if "EXAMPLES" in body["messages"][0]["content"] and "VISITOR MESSAGE" in body["messages"][-1]["content"]: + SEEN["local_prompt"][mdl] = SEEN["local_prompt"].get(mdl, 0) + 1 if body.get("model") == "daily-model": b = json.dumps({"error": {"message": "Rate limit exceeded: free-models-per-day.", "code": 429, "metadata": {"headers": {"X-RateLimit-Limit": "50", "X-RateLimit-Remaining": "0", "X-RateLimit-Reset": "4102444800000"}}}}).encode() @@ -42,7 +51,7 @@ class H(BaseHTTPRequestHandler): self.send_response(200); self.send_header("Content-Type", "application/json"); self.end_headers(); self.wfile.write(b) return if last.startswith("TOOL_RESULTS"): - ids = re.findall(r'"id": "(lst_[a-f0-9]+)"', last)[:3] + ids = re.findall(r'"id": "(lst_[a-f0-9]+|L\d+)"', last)[:3] # local model sees short refs (L1, L2, ...) out = {"reply": "Here are bikes I found (mock LLM). Payments are TEST MODE ONLY.", "actions": [{"type": "show_cards", "payload": {"heading": "Bikes", "listing_ids": ids}}, {"type": "not_allowed_component", "payload": {}}]} diff --git a/tests/nl_check.py b/tests/nl_check.py index 5231718..0afc553 100755 --- a/tests/nl_check.py +++ b/tests/nl_check.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 -"""Natural-language checks against the real hosted model. Usage: nl_check.py BASE_URL -Each prompt should be answered by the LLM (not the fallback) with the expected panel.""" +"""Natural-language checks against a real model (hosted, or the local model). Usage: nl_check.py BASE_URL +Each prompt should be answered by a model (not the busy reply) with the expected panel.""" import json, sys, time, urllib.request, urllib.error BASE = (sys.argv[1] if len(sys.argv) > 1 else "http://127.0.0.1:8795").rstrip("/") fee_bps = json.load(urllib.request.urlopen(BASE + "/api/status"))["fee_bps"] diff --git a/tests/test_agent_units.py b/tests/test_agent_units.py new file mode 100644 index 0000000..6999e26 --- /dev/null +++ b/tests/test_agent_units.py @@ -0,0 +1,65 @@ +#!/usr/bin/env python3 +"""Unit checks for the model-output normalisers used with the local backup model (no server, no model).""" +import os, re, sys, tempfile +sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..")) +os.environ["AGENT_MARKET_DATA"] = tempfile.mkdtemp() +for k in ("AGENT_MARKET_LLM_API_KEY_FILE", "AGENT_MARKET_LOCAL_LLM_URL"): + os.environ.pop(k, None) +import server # noqa: E402 (seeds a temp store) +from market import agent, theme # noqa: E402 + +results = [] +def check(name, ok, d=""): + results.append(bool(ok)); print(("PASS " if ok else "FAIL ") + name + (f" [{d}]" if d else "")) + +st = server.store +by = {L["title"]: L["id"] for L in st.all("listings")} +tut = next(i for t, i in by.items() if "tutoring" in t.lower()); bike = next(i for t, i in by.items() if "kids" in t.lower()) +lawn = next(i for t, i in by.items() if "lawn" in t.lower()) +f = lambda q, ids: agent._focus_results(st, q, [{"type": "show_cards", "payload": {"listing_ids": list(ids)}}])[0]["payload"]["listing_ids"] +check("focus: unrelated padding dropped when some picks match", f("show me bikes", [tut, bike]) == [bike]) +check("focus: semantic picks kept when nothing matches lexically", f("anything for the garden?", [lawn]) == [lawn]) +check("focus: no keywords -> unchanged", f("show me everything", [tut, bike]) == [tut, bike]) + +a = theme.build_theme_action({"preset": "sunset", "reset": True}) +check("theme: preset + reset = start from the preset (not a bare reset)", a and a.get("preset") == "sunset" and a.get("base") and a.get("reset"), a) +a = theme.build_theme_action({"reset": True}) +check("theme: bare reset still resets", a == {"reset": True, "patch": {}}, a) + +h = server.hydrate([{"type": "show_listing_form", "payload": {"title": "Old guitar", "price_cents": 12000, "payment_options": ["cash"]}}], "t") +d = h[0]["payload"]["defaults"] +check("listing form: flat payload + price_cents normalised", d == {"title": "Old guitar", "price": 12000, "payment_options": ["cash"]}, d) +h = server.hydrate([{"type": "show_listing_form", "payload": {"defaults": {"title": "Lamp", "price": 2500}}}], "t") +check("listing form: normal shape untouched", h[0]["payload"]["defaults"] == {"title": "Lamp", "price": 2500}) + +cfg = agent._llm_config() +check("no hosted key and no local model -> no chain (typed messages get the busy reply)", cfg is None) +os.environ.update(AGENT_MARKET_LOCAL_LLM_URL="http://127.0.0.1:9/v1", AGENT_MARKET_LOCAL_LLM_MODEL="m") +cfg = agent._llm_config() +check("local model alone is a valid chain", cfg and cfg["models"] == [] and cfg["local"]["model"] == "m" and cfg["local"]["url"].endswith("/v1/chat/completions")) +# local model: short catalog refs in the prompt, mapped back to real ids; unknown refs left for hydrate() to drop +al = agent._aliases(st) +sp = agent._system_prompt(st, "", local=True) +check("local prompt uses short refs, not hex ids", '"id": "L1"' in sp and not re.search(r"lst_[0-9a-f]{6}", sp.split("CATALOG (all active listings)")[1]) and '"L7"' in sp, len(al)) +plan = agent._unalias({"actions": [{"type": "show_cards", "payload": {"listing_ids": ["L1", "L999"]}}, + {"type": "show_checkout", "payload": {"listing_id": "L2"}}]}, al) +check("refs mapped back (unknown kept for hydrate to drop)", plan["actions"][0]["payload"]["listing_ids"] == [al["L1"], "L999"] and + plan["actions"][1]["payload"]["listing_id"] == al["L2"]) +check("hosted prompt keeps real ids", al["L1"] in agent._system_prompt(st, "", local=False)) + +# primer: re-prime after the local server restarts (llama-server /metrics counter goes back down) or is down +import http.server, threading +M = {"n": 500} +class _H(http.server.BaseHTTPRequestHandler): + def log_message(self, *a): pass + def do_GET(self): + b = f"# HELP x\nllamacpp:n_decode_total {M['n']}\n".encode(); self.send_response(200); self.end_headers(); self.wfile.write(b) +srv = http.server.ThreadingHTTPServer(("127.0.0.1", 0), _H); threading.Thread(target=srv.serve_forever, daemon=True).start() +loc = {"url": f"http://127.0.0.1:{srv.server_port}/v1/chat/completions"} +agent._PRIMED["decodes"] = None +first = agent._local_restarted(loc); M["n"] = 520; grew = agent._local_restarted(loc); M["n"] = 7; dropped = agent._local_restarted(loc) +srv.shutdown(); down = agent._local_restarted(loc) +check("primer: restart detected from /metrics, normal use is not a restart, down = re-prime", + (first, grew, dropped, down) == (False, False, True, True), (first, grew, dropped, down)) +print(f"\n{sum(results)}/{len(results)} passed") +sys.exit(0 if all(results) else 1) diff --git a/tests/test_copy_defaults.py b/tests/test_copy_defaults.py new file mode 100644 index 0000000..1b1c115 --- /dev/null +++ b/tests/test_copy_defaults.py @@ -0,0 +1,28 @@ +#!/usr/bin/env python3 +"""Page-text slots: server defaults match the React app, and the model prompt shows them. No server or model needed. +(Text edits are made by the model with set_theme copy; there is no rule-based text editing.)""" +import os, sys +sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..")) +os.environ.pop("AGENT_MARKET_LLM_API_KEY_FILE", None) +from market.copyedit import DEFAULT_COPY, current_copy +from market import agent + +results = [] +def check(name, ok, d=""): + results.append(bool(ok)); print(("PASS " if ok else "FAIL ") + name + (f" [{d}]" if d else "")) + +app = open(os.path.join(os.path.dirname(__file__), "..", "web", "src", "App.tsx"), encoding="utf-8").read() +ok = all(s in app for s in (DEFAULT_COPY["logo"], DEFAULT_COPY["headline"], DEFAULT_COPY["subhead"])) and \ + all(c["label"] in app for c in DEFAULT_COPY["chips"]) and all(b.split("{fee}% ")[-1] in app for b in DEFAULT_COPY["bullets"]) +check("DEFAULT_COPY in sync with web/src/App.tsx", ok) +check("model prompt explains top-left = logo and includes PAGE_TEXT", "TOP-LEFT" in agent.UI_SCHEMA and '"logo": "Agent Market"' in agent._page_text("{}")) +check("PAGE_TEXT follows the visitor's overrides", '"logo": "New Internet"' in agent._page_text('{"copy":{"logo":"New Internet"}}')) +check("fee in bullets follows FEE_BPS", current_copy({}, 250)["bullets"][0].startswith("2.5% ")) +check("local prompt carries examples (incl. a top-left text edit), hosted prompt doesn't", + "EXAMPLES" in agent.LOCAL_EXAMPLES and '"copy":{"logo"' in agent.LOCAL_EXAMPLES and "EXAMPLES (" not in agent.UI_SCHEMA) +gone = [n for n in ("fallback_agent", "_rules_check") if hasattr(agent, n)] +import market.theme as th, market.copyedit as ce +gone += [n for n in ("parse_look_command",) if hasattr(th, n)] + [n for n in ("parse_copy_edit",) if hasattr(ce, n)] +check("no rule-based typed-message code left", not gone, gone) +print(f"\n{sum(results)}/{len(results)} passed") +sys.exit(0 if all(results) else 1) diff --git a/tests/test_copyedit.py b/tests/test_copyedit.py deleted file mode 100644 index 7206552..0000000 --- a/tests/test_copyedit.py +++ /dev/null @@ -1,90 +0,0 @@ -#!/usr/bin/env python3 -"""Built-in agent: UI copy edits (logo/headline/subtitle/footer/chips/bullets). No server or model needed.""" -import json, os, re, sys -sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..")) -os.environ.pop("AGENT_MARKET_LLM_API_KEY_FILE", None) -from market.theme import parse_look_command -from market.copyedit import DEFAULT_COPY -from market import agent - -results = [] -def check(name, ok, d=""): - results.append(bool(ok)); print(("PASS " if ok else "FAIL ") + name + (f" [{d}]" if d else "")) - -def copy_of(text, theme=None): - r = parse_look_command(text, theme or {}) - return (r[0], (r[1] or {}).get("copy")) if r else (None, None) - -LOGO = [ - "replace agent market to new internet on the top left corner", # the user's exact phrasing - "change Agent Market to New Internet in the top left", "rename the logo to New Internet", "call the site New Internet", - "top left should say New Internet", "replace Agent Market with New Internet", "change the top left text to New Internet", - "make the logo say new internet", "I want the header to say New Internet", "logo: New Internet", "use New Internet as the site name", - "change the brand to new internet please", "rename the website to new internet", "name the site new internet", - "change agent market into new internet", "swap Agent Market for New Internet", 'replace "Agent Market" with "New Internet"', - "Please change the site name to New Internet.", "the top-left corner should read New Internet", -] -for p in LOGO: - reply, c = copy_of(p) - check(f"logo: {p}", c == {"logo": "New Internet"}, f"{c} / {reply}") - -cases = [ - ("change the title to Hello World", {"headline": "Hello World"}), - ("change the headline to Retro vibes", {"headline": "Retro vibes"}), - ("replace A marketplace you can just talk to. with Buy and sell anything nearby", {"headline": "Buy and sell anything nearby"}), - ("replace marketplace with bazaar", {"headline": "A bazaar you can just talk to."}), - ("set the subtitle to Fast and free", {"subhead": "Fast and free"}), - ("swap the tagline with Shop by chatting", {"subhead": "Shop by chatting"}), - ("swap the footer to Made for testing", {"footer": "Made for testing"}), - ("replace market with Internet in the logo", {"logo": "Agent Internet"}), - ("the top left should read NEW INTERNET", {"logo": "NEW INTERNET"}), - ("change the logo to new Internet", {"logo": "new Internet"}), # mixed case typed -> kept exactly -] -for p, want in cases: - reply, c = copy_of(p) - check(f"copy: {p}", c == want, f"{c}") -reply, c = copy_of("rename Tutoring to Lessons") -check("chip label edited, emoji + action kept", c and c["chips"][1] == {"label": "📐 Lessons", "say": "find tutoring services"} and c["chips"][3].get("event") == "sell", c and c["chips"][:4]) -reply, c = copy_of("replace $0 fee with no fee") -check("bullet edited by find-and-replace", c and c["bullets"][1].startswith("no fee when you meet"), c) -reply, c = copy_of("replace New Internet with Agent Hub", {"copy": {"logo": "New Internet"}}) -check("find-and-replace uses the visitor's current text", c == {"logo": "Agent Hub"}, c) - -# unclear -> says what it understood + closest option, never silently nothing / listings -r = parse_look_command("replace agent with robot", {}) -check("ambiguous text -> asks where", r and r[1] is None and "more than one place" in r[0], r and r[0]) -r = parse_look_command("replace agnet markt with New Internet", {}) -check("typo -> offers the closest slot", r and r[1] is None and "Closest is the top-left logo text" in r[0], r and r[0]) -r = parse_look_command("replace foo bar with baz", {}) -check("not found -> lists what can be edited", r and r[1] is None and "can't find" in r[0], r and r[0]) - -# no hijacking of look commands, search or selling -for p, key in (("change the background to a sunset gradient", "background"), ("switch to light mode with a green accent", "mode"), - ("use a serif font and make the text bigger", "font"), ("replace the background with blue", "background"), - ("move the chat to the left", "chat"), ("hide the bullet points", "sections")): - r = parse_look_command(p, {}) - check(f"look command still works: {p}", r and r[1] and key in r[1] and "copy" not in r[1], r and r[1]) -for p in ("show me bikes", "change my order to cash", "I'd like to sell my lamp for $20", "find tutoring services"): - check(f"not a copy edit: {p}", parse_look_command(p, {}) is None) -r = parse_look_command("make it look retro", {}) -check("preset still works", r and r[1] == {"preset": "retro"}) -r = parse_look_command("use the Inter font", {}) -check("font names are word-bounded (Inter font still works)", r and r[1].get("font") == "Inter") -check("'internet' is not the Inter font", "font" not in (parse_look_command("make the headline say internet rocks", {})[1] or {})) - -# end to end through the built-in agent (what /api/chat returns when no model answers) -out = agent.fallback_agent(None, "replace agent market to new internet on the top left corner", [], "t", {}) -check("built-in agent: user's exact phrasing -> set_theme copy.logo, no listings", - [a["type"] for a in out["actions"]] == ["set_theme"] and out["actions"][0]["payload"]["copy"] == {"logo": "New Internet"}, out["reply"]) -out = agent.fallback_agent(None, "replace agent with robot", [], "t", {}) -check("built-in agent: unclear edit -> explanation only", out["actions"] == [] and "agent" in out["reply"]) - -# server copy defaults match the React app's defaults -app = open(os.path.join(os.path.dirname(__file__), "..", "web", "src", "App.tsx"), encoding="utf-8").read() -ok = all(s in app for s in (DEFAULT_COPY["logo"], DEFAULT_COPY["headline"], DEFAULT_COPY["subhead"]) ) and \ - all(c["label"] in app for c in DEFAULT_COPY["chips"]) and all(b.split("{fee}% ")[-1] in app for b in DEFAULT_COPY["bullets"]) -check("DEFAULT_COPY in sync with web/src/App.tsx", ok) -check("model prompt explains top-left = logo and includes PAGE_TEXT", "TOP-LEFT" in agent.UI_SCHEMA and '"logo": "Agent Market"' in agent._page_text("{}")) - -print(f"\n{sum(results)}/{len(results)} passed") -sys.exit(0 if all(results) else 1) diff --git a/tests/theme_check.js b/tests/theme_check.js index bdd9f28..99fb066 100644 --- a/tests/theme_check.js +++ b/tests/theme_check.js @@ -24,7 +24,7 @@ const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); for (let attempt = 0; attempt < 4; attempt++) { const n = (await page.$$(".msg.bot")).length; const t0 = Date.now(); await page.type("#chatInput", text); await page.keyboard.press("Enter"); - await page.waitForFunction((n) => document.querySelectorAll(".msg.bot").length > n, { timeout: 60000 }, n); + await page.waitForFunction((n) => document.querySelectorAll(".msg.bot").length > n, { timeout: 90000 }, n); await sleep(500); const last = await page.$$eval(".msg.bot", (ns) => ns[ns.length - 1].textContent); if (/sending messages quickly/.test(last)) { const s = +(last.match(/(\d+)s/) || [0, 6])[1]; await sleep((s + 1) * 1000); continue; } @@ -40,6 +40,8 @@ const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); // Guardrails: banner, badge and checkout fee lines must stay visible and >= 4.5:1 contrast. async function guardrails(label) { // open a checkout for the first online-payable item via the deterministic click path + // (earlier runs may have sold the online-payable bikes; then ask for something payable online) + if (!(await page.$("#stage .card button.btn:not(.cash)"))) await ask("show me the bike tune-up service"); await page.evaluate(() => { const b = [...document.querySelectorAll("#stage .card button.btn:not(.cash)")][0]; if (b) b.click(); }); const hasCheckout = await page.waitForSelector("[data-component=checkout]", { timeout: 15000 }).then(() => true, () => false); const r = await page.evaluate(() => { @@ -142,8 +144,8 @@ const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); check("Reset look button restores the default", rs.ls === null && /just talk to/.test(rs.h) && rs.css === "" && /Look reset to default/.test(rs.toast), JSON.stringify(rs).slice(0, 120)); check("no internal ids in replies", replies.every((x) => !/\b(lst|pi|cash|ord)_[0-9a-f]{6,}/.test(x.reply))); - const who = [...new Set(replies.map((x) => (/AI agent/.test(x.mode) ? x.mode.split(" · ")[1] : "built-in agent")))].join(", "); - if (process.env.REQUIRE_LLM) check("all restyles answered by the real model", replies.every((x) => /AI agent/.test(x.mode)), who); + const who = [...new Set(replies.map((x) => (/AI agent|Local model/.test(x.mode) ? x.mode.split(" · ").slice(0, 2).join(" ") : "built-in agent")))].join(", "); + if (process.env.REQUIRE_LLM) check("all restyles answered by the real model", replies.every((x) => /AI agent|Local model/.test(x.mode)), who); else console.log("answered by: " + who); check("no page JS errors", errors.length === 0, errors.join("; ")); const ms = replies.map((x) => x.ms).sort((a, b) => a - b); diff --git a/web/src/App.tsx b/web/src/App.tsx index d3d4490..90e70f8 100644 --- a/web/src/App.tsx +++ b/web/src/App.tsx @@ -48,12 +48,23 @@ export default function App() { const id = ++seq; setToasts((ts) => [...ts, { id, text, level }]); setTimeout(() => setToasts((ts) => ts.filter((t) => t.id !== id)), 3500); }, []); - const say = useCallback((role: Msg["role"], text: string) => { const id = ++seq; setMsgs((m) => [...m, { id, role, text }]); return id; }, []); + const say = useCallback((role: Msg["role"], text: string, retry?: () => void) => { const id = ++seq; setMsgs((m) => [...m, { id, role, text, retry }]); return id; }, []); const unsay = useCallback((id: number) => setMsgs((m) => m.filter((x) => x.id !== id)), []); - const setModel = useCallback((model?: string, ms?: number) => { + const setModel = useCallback((model?: string, ms?: number, local = false) => { const short = (model || "").replace(/:free$/, ""); + const secs = ms ? " · " + (ms / 1000).toFixed(1) + "s" : ""; + if (local) { + // the model running on this server (answers when the free hosted models are unavailable) + setBadge({ text: `Local model (${short})`, title: `${model} running on this server (used when the free hosted models are unavailable; slower)` }); + setChatMode(`Local model · ${short}${secs}`); + return; + } setBadge({ text: "AI: " + short, title: (model || "") + " via OpenRouter (free tier)" }); - setChatMode(`AI agent · ${short}${ms ? " · " + (ms / 1000).toFixed(1) + "s" : ""}`); + setChatMode(`AI agent · ${short}${secs}`); + }, []); + const setBusy = useCallback(() => { + setBadge({ text: "AI: busy", title: "The free hosted models and the local model are all unavailable right now" }); + setChatMode("Assistant busy · try again in a moment"); }, []); // Render agent actions through the registry: effects run, panels replace the results area. @@ -80,34 +91,40 @@ export default function App() { setTimeout(() => stageRef.current?.scrollIntoView({ behavior: "smooth", block: "start" }), 0); }, [toast]); - const talk = useCallback(async ({ message, event, echo }: Talk) => { + const talk = useCallback(async ({ message, event, echo }: Talk): Promise => { setChatOpen(true); if (echo) say("user", echo); const thinking = say("sys", "agent is thinking…"); try { const out = await api("/api/chat", { message, event, history: history.current.slice(-6), actor, theme: themeRef.current }); unsay(thinking); + if (out.agent === "unavailable") { + // every model failed: an honest busy message with a retry button (no rule-based answer, nothing kept in history) + const id = say("bot", out.reply || "The assistant is busy, please try again in a moment.", () => { unsay(id); talk({ message, event }); }); + setBusy(); + return; + } if (message) history.current.push({ role: "user", content: message }); if (event) history.current.push({ role: "user", content: "UI_EVENT " + JSON.stringify(event) }); history.current.push({ role: "assistant", content: out.reply || "" }); say("bot", out.reply || "(no reply)"); - if (out.agent === "llm") setModel(out.model, out.latency_ms); - else if (out.agent === "fallback") { - setChatMode("built-in agent answered (AI model unavailable)"); - setBadge({ text: "agent: built-in (AI unavailable)", title: "The hosted model is rate-limited or out of free quota" }); - } else if (out.agent === "rate-limit") toast(out.reply || "", "warn"); + if (out.agent === "llm") setModel(out.model, out.latency_ms, !!out.local); + else if (out.agent === "rate-limit") toast(out.reply || "", "warn"); render(out.actions || []); } catch (e) { unsay(thinking); say("sys", "Agent error: " + (e as Error).message); } - }, [say, unsay, setModel, toast, render]); + }, [say, unsay, setModel, setBusy, toast, render]); // Boot: status (fee + badge), then the deterministic landing view and greeting. useEffect(() => { api("/api/status").then((s) => { setFeeBps(s.fee_bps); - if (s.agent.mode === "llm" && s.agent.daily_quota_exhausted_until_utc) - setBadge({ text: "agent: built-in (AI quota used up)", title: "Hosted model free quota resets " + s.agent.daily_quota_exhausted_until_utc }); - else if (s.agent.mode === "llm") setModel(s.agent.last_model || s.agent.model); - else setBadge({ text: "agent: rule-based fallback", title: s.agent.note || "" }); + const a = s.agent; + if (a.mode === "llm" && a.daily_quota_exhausted_until_utc && a.local_model && a.local_ok !== false) + setBadge({ text: `Local model (${a.local_model})`, title: `Hosted free quota used up (resets ${a.daily_quota_exhausted_until_utc}); a local model on this server answers` }); + else if (a.mode === "llm" && a.daily_quota_exhausted_until_utc) + setBadge({ text: "AI: quota used up", title: "Hosted model free quota resets " + a.daily_quota_exhausted_until_utc + "; typed messages may get a busy reply" }); + else if (a.mode === "llm") setModel(a.last_model || a.model, undefined, a.last_model ? !!a.last_local : !a.chain?.some((m) => !m.startsWith("local:"))); + else setBadge({ text: "AI: unavailable", title: s.agent.note || "" }); }); api("/api/chat", { event: { type: "landing" }, actor }).then((out) => { render(out.actions || []); diff --git a/web/src/components/Chat.tsx b/web/src/components/Chat.tsx index 5105d86..f04e4ce 100644 --- a/web/src/components/Chat.tsx +++ b/web/src/components/Chat.tsx @@ -1,6 +1,6 @@ import { useEffect, useRef, useState } from "react"; -export interface Msg { id: number; role: "user" | "bot" | "sys"; text: string } +export interface Msg { id: number; role: "user" | "bot" | "sys"; text: string; retry?: () => void } /** Chat popup. Typed messages go to the model via /api/chat; position/size/colours come from the theme. */ export function Chat({ open, setOpen, mode, msgs, onSend }: { @@ -19,7 +19,9 @@ export function Chat({ open, setOpen, mode, msgs, onSend }: {
- {msgs.map((m) =>
{m.text}
)} + {msgs.map((m) => m.retry + ?
{m.text}
+ :
{m.text}
)}
{ e.preventDefault(); const v = value.trim(); if (!v) return; setValue(""); onSend(v); }}> setValue(e.target.value)} /> diff --git a/web/src/styles.css b/web/src/styles.css index 908de7c..618621d 100644 --- a/web/src/styles.css +++ b/web/src/styles.css @@ -95,3 +95,4 @@ body.no-anim *::after{animation:none !important;transition:none !important} [data-component=checkout] a,[data-component=cash-meetup] a{color:var(--p-ink);text-decoration:underline} [data-component=receipt] h2.ok{color:var(--p-ink)} } +.msg.busy{border:1px dashed color-mix(in srgb,var(--chat-ink) 35%,transparent)}.retryBtn{margin-left:6px;padding:2px 9px;border-radius:9px;border:1px solid var(--chat-acc);background:transparent;color:var(--chat-ink);cursor:pointer;font:inherit;font-size:12px} diff --git a/web/src/types.ts b/web/src/types.ts index 50271cc..657049b 100644 --- a/web/src/types.ts +++ b/web/src/types.ts @@ -15,12 +15,16 @@ export interface Order { } export interface UIAction { type: string; payload?: any } export interface ChatReply { - reply?: string; actions?: UIAction[]; agent?: "llm" | "fallback" | "ui-event" | "rate-limit"; model?: string; latency_ms?: number; + reply?: string; actions?: UIAction[]; agent?: "llm" | "unavailable" | "ui-event" | "rate-limit"; model?: string; latency_ms?: number; + provider?: string; local?: boolean; busy?: boolean; retry?: boolean; } export interface ChatEvent { type: string; [k: string]: unknown } export interface Status { env: string; fee_bps: number; - agent: { mode: "llm" | "fallback"; model?: string; last_model?: string; note?: string; daily_quota_exhausted_until_utc?: string | null }; + agent: { + mode: "llm" | "unavailable"; model?: string; last_model?: string; note?: string; daily_quota_exhausted_until_utc?: string | null; + local_model?: string | null; local_ok?: boolean | null; last_local?: boolean; chain?: string[]; + }; } export const money = (c: number) => "$" + (c / 100).toFixed(2);