Typed chat: free hosted models -> local model (llama-server), no keyword rules; busy reply + Retry #7

Merged
leora merged 7 commits from feat/local-llm-backup into main 2026-10-08 22:18:21 +08:00

7 Commits

Author SHA1 Message Date
a2fd100587 Browser tests: don't assume an online-payable bike is still unsold on staging 2026-10-08 22:18:08 +08:00
6c2a1d6a40 Local model: short catalog refs (L1, L2, ...) mapped back to real ids; checkout and cash examples
A 4B model miscopied 8-hex listing ids (picked a tutoring id for a kids' bike). The local prompt now shows L-refs, the
server maps them back before tools/hydration (unknown refs are dropped as before), tool results are aliased the same
way, and refs are scrubbed from replies. Hosted models still see real ids.
2026-10-08 22:05:11 +08:00
d785091d64 Typed chat: hosted free models, then the local model, nothing else; busy reply + Retry; llama-server watchdog
- Remove the keyword-rules agent from typed chat: fallback_agent, parse_look_command (+ tables), the copy-edit parser
  and _rules_check are gone. If every model fails: 'The assistant is busy, please try again in a moment.' with a Retry
  button (agent 'unavailable', no actions). Clicks, forms and payments stay deterministic (fallback_event); the landing
  view is a deterministic featured-listings panel; the Sell button opens the empty form.
- Local model via llama.cpp llama-server directly (ops/local-llm-watchdog.sh, loopback only, restart on exit/hang,
  crash backoff, log rotation; deploy.sh ensures it runs). Lean: no vision projector, no weight repack (mmapped), ctx
  6144, 1 slot, q8 KV, 6 threads at nice 10, poll 0, 1 GiB prompt cache. Idle 0% CPU, 2.9 GB RSS (0.6 GB private).
- Local requests use a JSON GBNF grammar, worked examples in the cached system prompt, and a labelled visitor message.
  10/10 correct on the real prompt, warm 4-13 s.
- Primer re-primes after a model-server restart (llama-server /metrics counter) and otherwise stays idle.
- Fix: hosted calls kept the Authorization header off when a local model was configured (cfg['local'] is a dict on
  hosted configs); now only the local server itself skips the key. Test added.
- Badge/chat header: 'AI: busy', 'AI: quota used up', 'AI: unavailable'; no 'built-in'/'checked by rules' texts.
- Tests: proxy checks expect the busy reply (never rules), clicks still work while busy, no-model server, local grammar/
  prompt/no key; browser checks for busy + Retry; e2e_http accepts busy for typed turns unless REQUIRE_LLM=1;
  test_copyedit.py replaced by test_copy_defaults.py; README updated.
2026-10-08 21:58:41 +08:00
e4552f5649 Local model: no chat history by default; rules check for look/text edits (marked in the chat header) 2026-10-08 21:35:37 +08:00
549cd46778 Local model: focus padded search results on the request's keywords; unit tests 2026-10-08 21:25:55 +08:00
c531b427d5 Local model: prime the prompt cache at start-up and when the catalog changes; accept price_cents in listing-form defaults; 2 Ollama slots 2026-10-08 21:15:37 +08:00
55cccc4d9a Local backup model (Ollama, gemma3:4b): hosted chain -> local model -> keyword rules; own timeout, straight to local while the daily quota is used up, quota stop persisted; honest 'Local model' badge; mock + browser tests; watchdog script 2026-10-08 21:11:57 +08:00