leora/agent-market
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Rate-limit only LLM-bound chat turns; clicks and payment events stay unlimited

Browse Source
This commit is contained in:
leora 2026-10-08 19:31:04 +08:00
parent c925544fe9
commit 63dd93dde9
3 changed files with 8 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@ -60,7 +60,7 @@ key stays on the server: it never goes to the browser, never into the repo, and
| `AGENT_MARKET_LLM_MODELS` | comma-separated fallback chain, tried in order |
| `AGENT_MARKET_LLM_TIMEOUT` / `_BUDGET` | hard cap per model attempt (12 s) and for the whole chain (28 s) |
| `AGENT_MARKET_LLM_COOLDOWN` | how long a model that returned 429 gets skipped (60 s) |
| `AGENT_MARKET_CHAT_RPM` / `_BURST` | per-IP token bucket on `/api/chat` (12/min, burst 6). `CF-Connecting-IP` is trusted only from loopback, i.e. the tunnel. |
| `AGENT_MARKET_CHAT_RPM` / `_BURST` | per-IP token bucket on `/api/chat` turns that reach the LLM (12/min, burst 6). Deterministic UI events aren't limited. `CF-Connecting-IP` is trusted only from loopback, i.e. the tunnel. |
`deploy.sh` loads `~/.config/agent-market/llm.env` and `~/.config/agent-market/<env>.env` if they exist. Both
sit outside the repo. The box runs OpenRouter free models:

4
server.py
View File

@ -312,7 +312,9 @@ class H(BaseHTTPRequestHandler):
return self._json(200, self._webhook(body))
actor = self._actor(body)
if path == "/api/chat":
ok, retry = rate_ok(self._client_ip())
# Only free-text turns reach the LLM; deterministic UI events (clicks, payments) are not limited.
llm_bound = not isinstance(body.get("event"), dict) or A.LLM_EVENTS
ok, retry = rate_ok(self._client_ip()) if llm_bound else (True, 0)
if not ok:
return self._json(429, {"error": "rate_limited", "retry_after": retry, "agent": "rate-limit",
"reply": f"You're sending messages quickly. Please wait about {retry}s and try again.",

6
tests/llm_proxy_check.sh
View File

@ -29,6 +29,8 @@ assert d["agent"]=="fallback" and d["llm_error"]=="all_models_failed" and d["act
print("PASS all models rate-limited -> rule-based agent still renders", [a["type"] for a in d["actions"]])'
curl -s localhost:18800/api/status | python3 -c '
import json,sys; a=json.load(sys.stdin)["agent"]; assert set(a["cooling_down"])=={"busy-model","busy-model-2"}, a; print("PASS 429ed models put on cooldown")'
CODES=$(for i in 1 2 3 4 5; do curl -s -o /dev/null -w '%{http_code} ' localhost:18801/api/chat -H 'Content-Type: application/json' -H 'CF-Connecting-IP: 203.0.113.9' -d '{"event":{"type":"landing"}}'; done)
OTHER=$(curl -s -o /dev/null -w '%{http_code}' localhost:18801/api/chat -H 'Content-Type: application/json' -H 'CF-Connecting-IP: 203.0.113.10' -d '{"event":{"type":"landing"}}')
CODES=$(for i in 1 2 3 4 5; do curl -s -o /dev/null -w '%{http_code} ' localhost:18801/api/chat -H 'Content-Type: application/json' -H 'CF-Connecting-IP: 203.0.113.9' -d '{"message":"show me bikes"}'; done)
OTHER=$(curl -s -o /dev/null -w '%{http_code}' localhost:18801/api/chat -H 'Content-Type: application/json' -H 'CF-Connecting-IP: 203.0.113.10' -d '{"message":"show me bikes"}')
EV=$(curl -s -o /dev/null -w '%{http_code}' localhost:18801/api/chat -H 'Content-Type: application/json' -H 'CF-Connecting-IP: 203.0.113.9' -d '{"event":{"type":"landing"}}')
[[ "$EV" == "200" ]] && echo "PASS UI events (no LLM call) not rate-limited" || { echo "FAIL event limited: $EV"; exit 1; }
[[ "$CODES" == "200 200 200 429 429 " && "$OTHER" == "200" ]] && echo "PASS per-IP rate limit (burst 3: $CODES; other IP $OTHER)" || { echo "FAIL rate limit: $CODES / $OTHER"; exit 1; }