Migrating from Jev

Use this checklist before switching traffic:

1. Set TYPESAFE_BASE_URL to your Laya Cloud origin and TYPESAFE_API_KEY to a Laya key. Keep your existing systemOne or system_one call.
2. Use jev-latest (the SDK default), jev-preview, jev-1.13.0, or laya-latest for language routing. English resolves to laya-0.3.5; detected non-English resolves to laya-multilingual-0.3.9. To select a checkpoint explicitly use laya-0.3.5, laya-multilingual, or laya-multilingual-0.3.9. See models.
3. Measure every question's state room. English has a 512-token sequence; the exact room is 511 − len(head + options sequence) for that question. The simple choice and noul questions in the plan leave 476 state tokens. Multilingual has a 1024-token window and about 951–990 state tokens of room. Longer instructions and options reduce either limit.
4. Split long state into smaller, self-contained chunks and send only the relevant chunk per call. Laya Cloud rejects state that would truncate; it never silently cuts it.
5. Keep each call at or below 32 questions and the selected model's processed-token cap: 3,500 English or 6,000 multilingual. The processed cost is number of questions × longest per-question Laya sequence length, so many questions with long states hit the cap sooner. A 32-question call can exceed the SDK's 10-second default timeout on a busy CPU worker; prefer smaller batches.
6. Handle 402 monthly free allowance exhaustion by checking /app/usage and waiting for the UTC reset or contacting support. Jev does not have this response.
7. Handle 429 account rate/in-flight limits and 529 worker busy/startup with Retry-After and retry-after-ms; branch on x-laya-error-code. The SDK retries these by default, so set an overall deadline.
8. Compare outputs: Laya strips action and noul confidence. Multilingual confidence is uncalibrated, and its English quality was lower in a 12-case check. The two checkpoints use different tokenizers, so token counts and state limits differ. The model card evaluates about 50 languages; its “100+” claim is unverified.
9. Pin SDK versions while migrating: Python typesafe-sdk==0.7.1; JavaScript @typesafe-ai/[email protected]. Roll back by restoring the previous Jev base URL and Jev key; keep keys separate.

The snippet below uses the repository's laya_compat counter and the model tokenizer only; it does not load torch or model weights. Run it from the repository root after uv sync --project worker. Downloading the tokenizer requires Hugging Face access the first time.

import sys
sys.path.insert(0, "worker")
from huggingface_hub import hf_hub_download
from runtime import MODEL_ID, REVISION, Tokenizer
from laya_compat import inspect_request

tokenizer = Tokenizer(hf_hub_download(MODEL_ID, "tokenizer/tokenizer.json", revision=REVISION))
state = "I was billed twice for March."
questions = {
    "department": {"type": "choice", "instructions": "Which team should handle this?", "criteria": {"billing": "Payment issues", "technical": "Bugs"}},
}
for qid, q in questions.items():
    _, errors = inspect_request(tokenizer, "word " * 2500, {qid: q})
    room = next(e["m"] for e in errors if e["code"] == "state_too_long")
    print(qid, "state room:", room)
counts, errors = inspect_request(tokenizer, state, questions)
print("processed tokens:", counts.processed, "validation errors:", errors)

Try the migrated request locally:

curl --fail --silent "$TYPESAFE_BASE_URL/v1/systemone" -H "Authorization: Bearer $TYPESAFE_API_KEY" -H 'Content-Type: application/json' -d '{"model":"jev-latest","state":"I was billed twice for March.","questions":{"department":{"type":"choice","instructions":"Which team should handle this?","criteria":{"billing":"Payment issues","technical":"Bugs"}}}}' | python3 -c 'import json,sys; r=json.load(sys.stdin); assert r["model"] == "laya-0.3.5" and "department" in r["answers"]'